And when I say quick I mean ~1hr! With a couple of geeks in hand, some eager journalist types, laptops and our ever articulate CEO, Francis Irving, we set to work, well, talking about data. The fact is there are many pre-scraping steps to consider:

1. What is the general area you are interested in?
2. Can you find other people, especially geeks, with that interest?
3. When you have done so, you need to find where the data is that relates to your field of interest
4. Once you’ve got a list of interesting data, you need to look at its structure (non-programmatically) in order to decide on a hypothesis to test
5. Then you need to recruit your geek (who should be involved in all of the above steps) to start deconstructing the data i.e. seeing what can be scraped
6. At this point you all need to work together to decide the schema of the scraper datastore i.e. the headings and their attributes
7. Iterate until your data can answer your hypothesis or alter your hypothesis (it could be that you can mash the scraper with another dataset)
8. Get working on answering your hypothesis. The outcome could be a query, a visualization or an application
9. Go back to your data and iterate again so that the structure fits your outcome
10. Pat yourselves on the back, have a beer and keep in touch for your next project

This may seem a bit much but this is how you make, iterate, and mediate for the web. The Mozilla Festival proved that this is achievable and enjoyable. In that vein, we got a scraper in 1hr! So a big cheer to Alex Poderoso for winning the coveted ScraperWiki mug.

To catch up on the MozFest fun, here is  the first draft of the Data Journalism Handbook. The festival premiered an amazing HTML5 documentary called The One Millionth Tower. You can catch up with all the rest including teaching kids to code with Hackasaurus and hacking video with popcorn.js (and an octocopter!) and loads more at the Mozilla Festival website.

Both can be found at the Mozilla Festival park in London on 4-6 November. If you’re lucky you might even spot a wily firefox. There will be an inclosure on the Friday from 18:00 where our tamed digger driver, Francis Irving, can give you some driving lessons.

As part of the Data Journalism Workshop on the Saturday, 10:00-17:00, we’ll be hosting a ‘Scraping 101’ session. There will be a host of data trackers to guide you through the web wilderness including Open Knowledge Foundation‘s Jonathan Gray and the European Journalism Centre‘s Liliana Bounegru. There will be herds of other data/web beasts roaming the plains so we suggest you stay inside or close to your digger.

If you’re interested in a close encounter of the data kind sign up for the event here.

So watch out Mozilla Festival – you’re being ScraperWikied!