OpenlyLocal – ScraperWiki https://blog.scraperwiki.com Extract tables from PDFs and scrape the web Tue, 09 Aug 2016 06:10:13 +0000 en-US hourly 1 https://wordpress.org/?v=4.6 58264007 Conquering Copyright and Scaling Open Data Projects – How Chris Taggart is Counting Culture https://blog.scraperwiki.com/2011/09/how-chris-taggart-is-counting-culture/ https://blog.scraperwiki.com/2011/09/how-chris-taggart-is-counting-culture/#comments Fri, 09 Sep 2011 14:05:18 +0000 http://blog.scraperwiki.com/?p=758215374 Chris Taggart is a founder of OpenlyLocal and OpenCorporates. He says “When people ask what I do I say I open up data, sometimes whether people like it or not.” In the beginning he didn’t really expect much to come of his first scrapers “other than maybe being told off by the councils, because all the councils at that time had got things on their website saying this is copyright”.

He did it anyway with a very profound outcome:

I expected them to send me a take down notice … actually that didn’t happen. What did happen is that a couple of councils contacted us and said we like what you’re doing, will you start scraping us.

His first success spurred him on to create an even more ambitious project. Corporate data. He knew he’d be looking at a vast array of sources scattered across the web, in different languages and formats. So he made call out on ScraperWiki for OpenCorporates. It currently has information from 22 million companies across 28 jurisdictions. And it’s an alpha! I caught up with him on Skpye to find out what he’s learnt about conquering copyright and scaling open data projects.

]]>
https://blog.scraperwiki.com/2011/09/how-chris-taggart-is-counting-culture/feed/ 1 758215374
Constructing the Open Data Landscape https://blog.scraperwiki.com/2011/09/constructing-the-open-data-landscape/ https://blog.scraperwiki.com/2011/09/constructing-the-open-data-landscape/#comments Wed, 07 Sep 2011 11:01:38 +0000 http://blog.scraperwiki.com/?p=758215331 In an article in today’s Telegraph regarding Francis Maude’s Public Data Corporation, Michael Cross asks: “What makes the state think it can be at the cutting edge of the knowledge economy“. He writes in terms of market and business share, giving the example of the satnav market worth over $100bn a year yet it’s based on free data from the US Government’s GPS system.

He credits the internet revolution for transforming public sector data into ‘cashable proposition’. We, along with many other start-ups, foundations and civic coding groups, are part of this ‘geeky world’ of Open Data. So we’d like to add our piece concerning the Open Data movement.

Michael has the right to ask this question because there is this constant custodial battle being fought every day, every scrape and every script on the web for the rights to data. So let me tell you about the geeks’ take on Open Data.

[vimeo http://www.vimeo.com/21711338]

The idea(l) behind Open Data is to create sustainable Open Data projects with purpose. This has been championed in the last couple of years by civic data projects such as MySociety, Open Knowledge Foundation, Code for America, Open AustraliaOpen Development Cambodia is following me on twitter! Older, more established organizations are also being converted to the Open Data ethos. For instance, The World Bank is one major organization turning to Open Data in a big way.

However, much of the public sector data published so far has been pretty much useless. Governments, finally, are beginning to realize that data has little value unless people understand its context and provenance. They are beginning to see that opening up their data can reduce the cost and responsibility of getting it to the end point user, as the Open Declaration on European Public Services clearly says: “The needs of today’s society are too complex to be met by government alone”.

The key to a sustainable Open Data landscape lies not in the organisational heads of government bodies but in the provenance of the data they release and the ways in which it is released. The goal should be to gain the 5 stars of open linked data. For this to be achieved the data needs to be pared down to its raw ingredients. In a research paper entitled “Open Data, Open Society” (see end of post) Marco Fioretti explains:

Public data are really useful only when they are raw, really open and linked … only when data are published online in that way every citizen or organization will be able to automatically analyze and present them in easy to understand forms

This is where ScraperWiki really excels in terms of opening up data. Not only is our data open and accessible through various processes (csv, database, API), even the extraction process is open in the form of a code wiki. In terms of data, we are rawer than raw. If government ordered an open data steak they would order rare, data hubs would order raw, ours would be mooing!

We’re providing some of the heavy machinery needed to construct the Open Data landscape. What it will look like very much depends on the civic cyber-community getting involved. A leader in this community is Chris Taggart, creator of OpenlyLocal and OpenCorporates, and a prolific ScraperWiki user. So I Skyped him to see what he makes of the state thinking it can be at the cutting edge of the knowledge economy:

Speaking of the linked economy, do check out all the links in this post and all the media included here is under Creative Commons license.

If you are interested in getting more involved in the Open Data scene check out the Open Knowledge Foundation.

Open Data, Open Society

]]>
https://blog.scraperwiki.com/2011/09/constructing-the-open-data-landscape/feed/ 1 758215331