Guest post by Kaitlin Dunning from Young Rewired State Young Rewired State is a network of software developers and designers aged 18 and under. It is the philanthropic arm of Rewired State and its primary focus is to find and foster the young children and teenagers who are driven to teaching themselves how to code, […]
Careers in Computing
I realise the whole world isn’t inspired by the same things I am, and that’s fair enough. However, on Wednesday, I had the privilege of being invited to share some of my inspiration with a bunch of teenagers at the Liverpool John Moores University career day. I was asked to talk to year-10 students (14-15 […]
Getting SkilledUp
Guest Post by SkilledUp‘s Nick Gidwani. ScraperWiki is a revolutionary tool. Not just because it allows you to collect data, but because it allows anyone – including journalists who now must specialize in data – to organize and draw conclusions from vast data sets. That skill set (organizing what is now called “big data”) was […]
From future import x.scraperwiki.com
Time flies when you’re building a platform. At the start of the year, we announced the beginnings of a new, more powerful, more flexible ScraperWiki. More powerful because it exposes industry standards like SQL, SSH, and a persistent filesystem to developers, so they can scrape and crunch and export data pretty much however they like. […]
Combining survey data with scraped data to tell a story
Guest post by Dan Armstrong My last three jobs have involved telling stories with survey data. Since every story requires a conflict, I look for conflicts. For instance, I’ll go to CFOs and ask them if their marketing people are doing a good job. Then I’ll ask the marketers if they think they’re doing a […]
Tools of the trade
With the experience of a whole week of ScraperWiki, I am starting to appreciate the core tools of the professional Data Scientist. In the past I’ve written scrapers in Matlab, C# and Python. However, the house language for scraping at ScraperWiki is Python. It’s a good choice: a mature but modern language with a wide […]
WordPress Titles: scraping with search url
I’ve blogged for a few years now, and I’ve used several tools along the way. zachbeauvais.com began as a Drupal site, until I worked out that it’s a bit overkill, and switched to WordPress. Recently, I’ve been toying with the idea of using a static site generator (a lá Jekyll or Hyde), or even pulling […]
A visit from a minister
You may have heard from twitter that last Wednesday, Nick Hurd, the Cabinet Minister for Civil Society, paid a visit to ScraperWiki HQ. Nick has been looking into government data and transparency as part of his remit, and asked if he could come and have a chat with us in Liverpool. Joined by Sophie and […]
World News Media Hackday
(#DME13hack) On Saturday, 13 April: Global news media execs are gathering in London to discuss the continuing emergence of digital media at WAN-IFRA’s Digital Media Europe 2013 (#DME13) and we’d like to invite you to influence the agenda with a hack day on Saturday 13th April. WAN-IFRA covered the first Knight-Ridder tablet back in 1994. 20 years later, publishers […]
I am Ian, Ian I am*
I have an 8 year itch: I spent the first 8 years of my career as an academic ending up a lecturer in physics at UMIST. Then I was a research scientist at a large “fast moving consumer goods” company for another 8 years. On Monday I started work at ScraperWiki as Senior Data Scientist, […]