Here at ScraperWiki, we’ve got a good eye for data. Not just structuring, formatting and quality, but also where data can tell stories (hence the addition of views to the site). The decision to put a scraper of the Texas Department of Criminal Justice by Noah Seger on the front page proved to be a bit […]
Knight Foundation finance ScraperWiki for journalism
ScraperWiki is the place to work together on data, and it is particularly useful for journalism. We are therefore very pleased to announce that ScraperWiki has won the Knight News Challenge! The Knight Foundation are spending $280,000 over 2 years for us to improve ScraperWiki as a platform for journalists, and to run events to bring together journalists […]
Access government in a way that makes sense to you? Surely not!
alpha.gov.uk uses Scraperwiki, a cutting edge data-gathering tool, to deliver the results that citizens want. And radically for government, rather than tossing a finished product out onto the web with a team of defenders, this is an experiment in customer engagement. If you’re looking to renew your passport, find out about student loans or how […]
ScraperWiki Datastore – The SQL.
Recently at ScraperWiki we replaced the old datastore, which was creaking under the load, with a new, lighter and faster solution – all your data is now stored in Sqlite tables as part of the move towards pluggable datastores. In addition to the new increase in performance, using Sqlite also provides some other benefits such […]
Hacks/Hackers London
First of all, the Iraq War Logs: Round One – The Cleaning Documents, records and words all hugely intimidating in their vastness. But some tools to help are MySQL, Ultraedit and Google Refine. But this stage is incredibly frustrating. Round Two – The Problem How do you tackle the types of documents? There was even […]
One-click ‘Open in Google Docs’
Data for data’s sake is no good, it’s what you do with it that matters. To make it easier for everyone, not just programmers, to reuse the data on ScraperWiki, you can now use the new experimental ‘Open in Google Docs’ feature. Clicking the link at the top right of a scraper page will load the data […]
Government data release: what’s still out there
James Ball Last week saw big steps forward in public data: on Monday, Prime Minister David Cameron wrote to all government departments, setting out a timetable for the release of a swathe of official datasets. On Wednesday, the first two (senior civil service pay and MRSA infection rates) appeared – but the real meat came […]
What Londoners worry about: Metropolitan Police priorities by area
We’ve been putting the alpha release of ScraperWiki though its paces by grabbing a few datasets and poking about with them. One of the things we have been taking a look at is police priorities across London, as set by local people. Under the Safer Neighbourhoods scheme, each council ward has a dedicated team of police […]