This year’s Open Knowledge Conference takes place in Berlin. While the conference proper starts tomorrow, during the last two days ScraperWiki ran a workshop to scrape and visualise German data. It has drawn a sizeable community of people, from novices to experienced coders, working on a variety of scrapers. Julian stunned everybody by giving a tour-de-force of just how much is possible by combining ScraperWiki scrapers and views – offering help (and consolation) once the participants tried to achieve just a bit of this themselves.

meine-demokratie.de (myDemocracy)

We have been there to get more data for our project meine-demokratie.de (literally myDemocracy).  This is a not-for-profit endeavour by German NGO Netzdemokraten to make it easy for citizens to find ways on how to have their say in politics and local affairs. In its simplest form, it is a search engine for participation opportunities such as elections, MP surgeries, demonstrations, planning applications, citizen referenda, petitions, consultations, etc. But its much more than that:

1. its focused on the local level, giving citizens tangible opportunities (on a map) to make an impact in their neighbourhood
2. it aggregates various disparate sources of information to make them accessible
3. it makes the data available in a standardised format, enhanced with geographical and administrative information

Why we use ScraperWiki

We have been using SraperWiki for some time because it offers a number of key benefits to our needs:

1. its a simple way to host code and share it with other collaborators
2. we don’t have to care about how to store the data
3. it provides a simple interface to the data – exporting is easy as is creating customised output formats
4. its easy to schedule periodic scraping

There is a more detailed description of how we make use of ScraperWiki at our blog, but basically we look for scrapers to cover many more practical opportunities for citizens to get involved in politics and have their say.The scraping workshop has been a great success for us because we managed to scrape data on future election dates as well as past ones (because the platform is also meant as an archive). What is more, we now also feature information collected by German NGO Mehr Demokratie e.V. on citizen referenda (visualised here) in German states – and there is more to come.

Data on political engagement opportunities – meine-demokratie’s wishlist

However, there is much more data out there, but it is is dispersed and in multiple formats.

Can you help us to get some more of this information in a standardised form – to make it searchable and accessible via meine-demokratie.de?

Here is a list of stuff that could be useful. Its all in German but please do get in touch (on our site or via Twitter) and we bring you up to speed. Also, if you have other sources in mind we would love to hear about them!!!

It’s an all-star line up of who’s who of open data including Chris Taggart, Rufus Pollack, Paul Bradshaw and our own, ‘The Julian’. Workshops running up to the talks cover Open Spending, CKAN and ScraperWiki. It’s an open data festival if ever there was and we’re heading to the continent to park our digger and invite you along for the ride. Just check out the packed programme!

So if you can make it, sign up for the conference here. If you’re in town, our workshop is free for anyone so get the details and the tickets here. We look forward to seeing you and your data!