How would you go about collecting, structuring and analysing 100,000 reports on Romanian companies? You could use ScraperWiki to write and host you own computer code that carries out the scraping you need, and then use our other self-service tools to clean and analyse the data. But sometimes writing your own code is not a […]
A visit from a minister
You may have heard from twitter that last Wednesday, Nick Hurd, the Cabinet Minister for Civil Society, paid a visit to ScraperWiki HQ. Nick has been looking into government data and transparency as part of his remit, and asked if he could come and have a chat with us in Liverpool. Joined by Sophie and […]
The state of Twitter: Mitt Romney and Indonesian Politics
It’s no secret that a lot of people use ScraperWiki to search the Twitter API or download their own timelines. Our “basic_twitter_scraper” is a great starting point for anyone interested in writing code that makes data do stuff across the web. Change a single line, and you instantly get hundreds of tweets that you can […]
Fine set of graphs at the Office of National Statistics
It’s difficult to keep up. I’ve just noticed a set of interesting interactive graphs over at the Office of National Statistics (UK). If the world is about people, then the most fundamental dataset of all must be: Where are the people? And: What stage of life are they living through? A Population Pyramid is a […]
How to stop missing the good weekends
Far too often I get so stuck into the work week that I forget to monitor the weather for the weekend when I should be going off to play on my dive kayaks — an activity which is somewhat weather dependent. Luckily, help is at hand in the form of the ScraperWiki email alert system. […]
ScraperWiki scrapers: now 53% more useful!
It’s Christmas come early at ScraperWiki HQ as we deliver—like elves popping boxes under the data digging Christmas tree—a bunch of great new improvements to the ScraperWiki site. We’ve been working on these for a while, so it’s great to finally let you all use them! First up: a new look for your scrapers The […]
Solving Healthcare Problems with Open Source Software
This year, EHealth Insider brought a new feature to their annual EHI Live exhibition: a healthcare skunkworks that gave visitors the chance to ask questions about how open source software can be used to solve healthcare problems. ScraperWiki, of course had to be one of the invited guests to exhibit at the skunkworks. So as is our way, we […]
How to get along with an ASP webpage
Fingal County Council of Ireland recently published a number of sets of Open Data, in nice clean CSV, XML and KML formats. Unfortunately, the one set of Open Data that was difficult to obtain, was the list of sets of open data. That’s because the list was separated into four separate pages. The important thing […]
Meet all our users!!!!
As I have been looking for your great guys to highlight your work, passion and scraping oddities, so your time has come to explore the craven habits of the new age data digging programmers! You can now search for people on ScraperWiki!!!! Just use our regular search box. So all you ScraperWikians out there, I […]
ScraperWiki Tutorial Screencast for Non-Programmers
If you’ve been going through our first ambitious tutorial and taster session for non-coders then good for you! I hope you found it enlightening. For those of you yet to try it, here it is. It is a step-by-step guide so please give it a go and don’t just try and follow the answers as […]