guardian local – ScraperWiki https://blog.scraperwiki.com Extract tables from PDFs and scrape the web Tue, 09 Aug 2016 06:10:13 +0000 en-US hourly 1 https://wordpress.org/?v=4.6 58264007 A reluctant goodbye to Guardian Local https://blog.scraperwiki.com/2011/04/a-reluctant-goodbye-to-guardian-local/ Thu, 28 Apr 2011 13:30:25 +0000 http://blog.scraperwiki.com/?p=758214699 ScraperWiki is sad to hear that Guardian Local is being wound down, just over a year after its public launch. We’ve had the good fortune to work with the talented Guardian Local journalists at three of our Hacks & Hackers events: in Cardiff, Leeds and Glasgow.

We would like to say a particular thank you to the project’s editor, Sarah Hartley, for her generous help. We wish Sarah, Hannah, John and Michael the very best in their new ventures, whatever they may be.

As you can see from the comments under the Guardian post announcing the sites’ closure, the beatbloggers, led by Sarah, have done amazing work for their respective communities. It’s testament to their hard work and energy that they’ve built up such a loyal following in a short space of time.

Michael MacLeod from Guardian Edinburgh at our Glasgow event (right):

]]>
758214699
Cardiff Hacks and Hackers Hacks Day https://blog.scraperwiki.com/2011/03/cardiff-hacks-and-hackers-hacks-day/ https://blog.scraperwiki.com/2011/03/cardiff-hacks-and-hackers-hacks-day/#comments Tue, 15 Mar 2011 16:12:32 +0000 http://blog.scraperwiki.com/?p=758214441 What’s occurin’? Loads in fact, at our first Welsh Hacks and Hackers Hack Day! From schools from space to catering colleges with a Food Safety Standard of 2, we had an amazing day. Check out the video by Gavin Owen:

We got five teams:

Co-Ordnance – This project aimed to be a local business tracker. They wanted to make the London Stock Exchange code into meaningful data, but alas, the stock exchange prevents scraping. So they decided to use company data from registers like the LSE and Companies House to extract business information and structure it for small businesses who need to know best place to set up and for local business activists.

The team consisted of 3 hacks (Steve Fossey, Eva Tallaksen from Intrafish and Gareth Morlais from BBC Cymru) and 3 hackers (Carey HilesCraig Marvelley and Warren Seymour, all from Box UK).

It’s a good thing they had some serious hackers as they had a serious hack on their hands. Here’s a scraper they did for the London Stock Exchange ticker. And here’s what they were able to get done in just one day!

This was just a locally hosted site but the map did allow users to search for types of businesses by region, see whether they’d been dissolved and by what date.

Open Senedd – This project aimed to be a Welsh version of TheyWorkforYou. A way for people in Wales to find out how assembly members voted in plenary meetings. It tackles the worthy task of making assembly members voting records accessible and transparent.

The team consisted of 2 hacks (Daniel Grosvenor from CLIConline and Hannah Waldram from Guardian Cardiff) and 2 hackers (Nathan Collins and Matt Dove).

They spent the day hacking away and drew up an outline for www.opensenedd.org.uk. We look forward to the birth of their project! Which may or may not look something like this (left). Minus Coke can and laptop hopefully!

They took on a lot for a one day project but devolution will not stop the ScraperWiki digger!

There’s no such thing as a free school meal – This project aimed to extract information on Welsh schools from inspection reports. This involved getting unstructure Estyn reports on all 2698 Welsh schools into ScraperWiki.

The team consisted of 1 hack (Izzy Kaminski) and 2 astronomer hackers (Edward Gomez and Stuart Lowe from LCOGT).

This small team managed to scrape Welsh schools data (which the next team stole!) and had time to make a heat map of schools in Wales. This was done using some sort of astronomical tool. Their longer term aim is to overlay the map with information on child poverty and school meals. A worthy venture and we wish them well.

Ysgoloscope – This project aimed to be a Welsh version of Schooloscope. Its aim was to make accessible and interactive information about schools for parents to explore. It used Edward’s scraper of horrible PDF Estyn inspection reports. These had different rating methodology to Ofsted (devolution is not good for data journalism!).

The team consisted of 6 hacks (Joni Ayn Alexander, Chris Bolton, Bethan James from the Stroke Association, Paul Byers, Geraldine Nichols and Rachel Howells), 1 hacker (Ben Campbell from Media Standards Trust) and 1 troublemaker (Esko Reinikainen).

Maybe it was a case of too many hacks or just trying to narrow down what area of local government to tackle, but the result was a plan. Here is their presentation and I’m sure parents all over Wales are hoping to see Ysgoloscope up and running.

Blasus – This project aimed to map food hygiene rating over Wales. They wanted to correlate this information with deprivation indices. They noticed that the Food Standards Agency site does not work. Not for this purpose which is most useful.

The team consisted of 4 hacks (Joe Goodden from the BBC, Alyson Fielding, Charlie Duff from HRZone and Sophie Paterson from the ATRiuM) and 1 hacker (Dafydd Vaughan from CF Labs).

As you can see below they created something which they presented on the day. They used this scraper and made an interactive map with food hygiene ratings, symbols and local information. Amazing for just a day’s work!

And the winners are… (drum roll please)

  • 1st Prize: Blasus
  • 2nd Prize: Open Senedd
  • 3rd Prize: Co-Ordnance
  • Best Scoop: Blasus for finding  a catering college in Merthyr with a Food Hygiene Standard rating of just 2
  • Best Scraper: Co-Ordnance

A big shout out

To our judges Glyn Mottershead from Cardiff School of Journalism, Media and Cultural Studies, Gwawr Hughes from Skillset and Sean Clarke from The Guardian.

And our sponsors Skillset, Guardian Platform, Guardian Local and Cardiff School of Journalism, Media and Cultural Studies.

Schools, businesses and eating place of Wales – you’ve been ScraperWikied!

Blasus winning first prize and Best Scoop award (prizes will be delivered, sealed with a handshake from our sponsor).


]]>
https://blog.scraperwiki.com/2011/03/cardiff-hacks-and-hackers-hacks-day/feed/ 8 758214441
New event! Hacks and Hackers Hack Day Cardiff (#hhhCar) https://blog.scraperwiki.com/2011/02/new-event-hacks-and-hackers-hack-day-cardiff-hhhcar/ https://blog.scraperwiki.com/2011/02/new-event-hacks-and-hackers-hack-day-cardiff-hhhcar/#comments Wed, 09 Feb 2011 08:11:50 +0000 http://blog.scraperwiki.com/?p=758214239 The UK Hacks & Hackers tour carries on – into 2011. Our first stop: Wales.

Scraperwiki, which provides award-winning tools for screen scraping,data mining and visualisation, will hold a one day practical hack day* at the Atrium in Cardiff on Friday 11 March, 2011.

Web developers and designers will pair up with journalists and bloggers to produce a number of projects and stories based on public data.

We would like to thank our main sponsor Skillset Cymru, our hosts the Atrium and our prize sponsors Guardian Local, Guardian Open Platform and Cardiff School of Journalism, Media and Cultural Studies for making the event possible.

“Skillset Cymru is very pleased to be supporting the Cardiff Scraperwiki Hacks and Hackers Hack Day this March,” says Gwawr Hughes, director, Skillset Cymru.

“This exciting event will bring journalists and computer programmers and designers together to explore the scraping, storage, aggregation, and distribution of public data in more useful, structured formats.

“It is at the forefront of data journalism and should be of great interest to the media industry across the board here in Wales.”

More details

Who’s it for? We hope to attract ‘hacks’ and ‘hackers’ from all different types of backgrounds: people from big media organisations, as well as individual online publishers and freelancers.

What will I get out of it?
The aim is to show journalists how to use programming and design techniques to create online news stories and features; and vice versa, to show programmers how to find, develop, and polish stories and features. To see what happened at our past events in Liverpool and Birmingham visit the ScraperWiki blog. Here’s a video showing what happened in Belfast.

How much? NOTHING! It’s absolutely free, thanks to our sponsors. Food and refreshments will be provided throughout the day. If you have special dietary requirements please email judith [at] scraperwiki.com.

What should I bring? We would encourage people to come along with ideas for local ‘datasets’ that are of interest. In addition we will create a list of suggested data sets at the introduction on the morning of the event but flexibility is key for this event. If you have a laptop, please bring this too.

So what exactly will happen on the day? Armed with their laptops and WIFI, journalists and developers will be put into teams of around four to develop their ideas, with the aim of finishing final projects that can be published and shared publicly. Each team will then present their project to the whole group. Winners will receive prizes at the end of the day.

*Not sure what a hack day is? Let’s go with the Wikipedia definition: It “an event where developers, designers and people with ideas gather to build ‘cool stuff'”…

With thanks to our sponsors:

Keep an eye on the ScraperWiki blog for details about Scraperwiki events. Hacks & Hackers Hack Day Glasgow is scheduled for March 25 2011. For additional information please contact judith [at] scraperwiki.com.

]]>
https://blog.scraperwiki.com/2011/02/new-event-hacks-and-hackers-hack-day-cardiff-hhhcar/feed/ 2 758214239
New event: Hacks and Hackers Hack Day Leeds https://blog.scraperwiki.com/2010/09/new-event-hacks-and-hackers-hack-day-leeds/ https://blog.scraperwiki.com/2010/09/new-event-hacks-and-hackers-hack-day-leeds/#comments Mon, 13 Sep 2010 08:57:42 +0000 http://blog.scraperwiki.com/?p=758213838 Tickets are now available for a new ScraperWiki event: Hacks and Hackers Hack Day Leeds, a chance for web developers and designers to pair up with journalists and bloggers to produce a number of projects and stories based on public data. We’re using the tag ‘#hhhLeeds‘ to talk about this on Twitter.

It’s the fifth stop on the ScraperWiki tour: we’ll be going to Manchester on Friday October 15th and Dublin on Tuesday 16th November as part of the Innovation Dublin festival. We’re currently negotiating venues and dates in CardiffBelfast and Glasgow.

What? Hacks and Hackers Hack Day Leeds

When? Thursday October 28th 2010 from 9.30 am to 7.30 pm

Where? At nti Leeds, Old Broadcasting House, 148 Woodhouse Lane, LS2 9EN Leeds (on Google Maps)

Who is it for? We hope to attract ‘hacks’ and ‘hackers’ from all different types of backgrounds: people from big media organisations, as well as individual online publishers and freelancers.

How much? Absolutely free, thanks to our sponsors, Guardian Open Platform and Guardian Local and Leeds Trinity Centre for Journalism for making the event possible, and our hosts nti Leeds at Old Broadcasting House.

Refreshments, lunch and evening snacks will be all be provided.

NB: We’re still seeking sponsors to get involved. If you think your organisation might be interested please contact aine [at] scraperwiki.com. Or get in touch with any other questions!

What happens on the day? The aim is to show journalists how to use programming and design techniques to create online news stories and features; and vice versa, to show programmers how to find, develop, and polish stories and features. All sorts of data was scraped and played with at our past events: in Liverpool, projects included mashes of police, libraries and courts data. Birmingham saw lots of health-related projects, as well as scraping of political party donor and leisure centre information.

This four minute video from Liverpool gives you a flavour of the kind of projects that can be developed.

Armed with their laptops and WIFI, journalists and developers will be put into teams of around four to develop their ideas, with the aim of finishing final projects that can be published and shared publicly. Each team will then present their project to the whole group. Overall winners will receive a prize at the end of the day.

With thanks to our sponsors:




Want to support us too? Contact aine [at] scraperwiki.com.

]]>
https://blog.scraperwiki.com/2010/09/new-event-hacks-and-hackers-hack-day-leeds/feed/ 2 758213838