What’s great about this (besides preventing angry Scotsmen from rioting when their local pub gets torn down without their knowing) is that this project spawned from our Hackers and Hacks Hack day in Glasgow. The winning pair, former Guardian Local beat blogger Michael MacLeod and developer Robert McWilliam, scraped the horrendous planning applications site and went on to make a map for the Greener Leith site.

Now Robert and Michael first met at our event and we’re thrilled that they’ve continued to work together to improve their idea and make data work for the people that are most affected. Robert is actually an electronics engineer but has found himself programming of late. He currently works for BlueFlow building stuff to do automated data mining.

ScraperWiki gives a nice way to run these things without having to worry about the infrastructure. Probably more important than the tech though is the documentation and wealth of other peoples work all nicely collected together and easy to “borrow” from
– Robert McWilliam, developer at BlueFlow

If this frees up some of Robert’s time so that he can help out young, talented journalists who want to work with data, then our mission is succeeding. And for keeping in contact with your hack and helping him help others, Robert McWilliam we salute you!

PS: this has been helping us test out our realtime scraping capabilities (just so you know we’re not lazing about!)

PlanningAlerts.org.au started life as a port of the PlanningAlerts.com codebase, which scrapes development application information from councils and sends email alerts of planning applications nearby to the user. It has since significantly evolved and is now a Ruby on Rails application that has sent almost one million development applications to people throughout Australia.

“We’ve started to use ScraperWiki because it makes collaboratively fixing scraper problems much simpler”
– Henare Degan (openaustralia)

Open Australia were finding Councils update their development application pages quite regularly. As a result, almost 30 percent of their scrapers broke. By having the visibility of what scrapers are broken, OpenAustralia volunteers can quickly dive in and make the changes required to fix broken scrapers on ScraperWiki. This is where our email alerts come in to play.

“Because of the email alerts, I’ve been able to quickly correct [broken scrapers] meaning more development applications are flowing to PlanningAlerts. The real power for me is that ScraperWiki handles the regular runs of scrapers and the email alerts are a killer feature”
– Henare Degan (openaustralia)