Calling journalists, bloggers, programmers and designers in Scotland! Scraperwiki is pleased to announce another hacks & hackers hack day: in Glasgow. BBC Scotland is hosting and sponsoring the one day event, with support from BBC College of Journalism. As with our other UK hack days, Guardian Open Platform is providing the prizes. Web developers and […]
Who changed my scraper? History upgrade
We’ve upgraded the history window, to make it easier to see who’s changed your scraper, what they changed, and when it broke. This is particularly useful if you’ve just received an email alert. You can get to it from the “History” tab at the top right of each scraper or view’s page. Here’s an example from the Work4MP […]
Job advert: Product / UX lover
ScraperWiki is a Silicon Valley style startup, but based in the UK. We’re changing the world of open data, and how programming is done together on the Internet. We’re looking for a web product designer who is… Able to make design decisions to launch features by themselves. Capable of writing CSS and HTML, and some […]
Spot and Normalize Inconsistent Measures
Here’s an example of why you have to be very careful when scraping, and why your normal run-of-the-mill technology that makes assumptions won’t cut it: One of our super-users, Julian Todd, decided to scrape the Vehicle Certification Agency (VCA) website on new car fuel consumption and exhaust emissions figures. And he spotted this: And another […]
New event! Hacks and Hackers Hack Day Cardiff (#hhhCar)
The UK Hacks & Hackers tour carries on – into 2011. Our first stop: Wales. Scraperwiki, which provides award-winning tools for screen scraping,data mining and visualisation, will hold a one day practical hack day* at the Atrium in Cardiff on Friday 11 March, 2011. Web developers and designers will pair up with journalists and bloggers […]
US visa lottery winners statistics – not just the numbers
The ScraperWiki community has a mish-mash of user interests, so we have a mish-mash of data, scrapers and views. It’s actually quite fun to spend time looking around, to see what people have done and how they have approached a scrape. Samuel Chinweoke Nwaobia found some data on US visa lottery winners (or Green Card […]
Be alert! Your scrapers need alerts
It’s important to know when your scrapers have stopped working, so you can fix them. And if someone else makes a change to one of your scrapers, you need to know, so you can check it’s OK and thank them. Over the next day or two, if you have made or contributed to a scraper […]
Ruby screen scraping tutorials
Mark Chapman has been busy translating our Python web scraping tutorials into Ruby. They now cover three tutorials on how to write basic screen scrapers, plus extra ones on using .ASPX pages, Excel files and CSV files. We’ve also installed some extra Ruby modules – spreadsheet and FastCSV – to make them possible. These Ruby scraping […]
Job advert: Web designer/programmer
Care about oil spills, newspapers or lost cats? ScraperWiki is a Silicon Valley style startup, but in the North West of England, in Liverpool. We’re changing the world of open data, and how programming is done together on the Internet. We’re looking for a web designer/programmer who is… Capable of writing standards compliant CSS and […]
Views part 2 – Lincoln Council committees
(This is the second of two posts announcing ScraperWiki “views”. A new feature that Julian, Richard and Tom worked away and secretly launched a couple of months ago. Once you’ve scraped your data, how can you get it out again in just the form you want? See also: Views part 1 – Canadian weather stations.) Lincoln […]