Oil wells, marathon results, planning applications… ScraperWiki is a Silicon Valley style startup, in the North West of England, in Liverpool. We’re changing the world of open data, and how data science is done together on the Internet. We’re looking for a programmer who’d like to: Revolutionise the tools for sharing data, and code that works with […]
Lots of new libraries
We’ve had lots of requests recently for new 3rd party libraries to be accessible from within ScraperWiki. For those of you who don’t know, yes, we take requests for installing libraries! Just send us word on the feedback form and we’ll be happy to install. Also, let us know why you want them as it’s […]
Meet all our users!!!!
As I have been looking for your great guys to highlight your work, passion and scraping oddities, so your time has come to explore the craven habits of the new age data digging programmers! You can now search for people on ScraperWiki!!!! Just use our regular search box. So all you ScraperWikians out there, I […]
Tweeting the drilling
A very long time ago I discovered the easiest webscraping target: the locations of all the North Sea Oil wells. Once you webcrawl through the index pages, the entries were pretty straightforward. There were dates, water depths (in feet or metres), GPS locations and so on. The code, if you want to look at it, […]
Diggers and Dinosaurs – Scraping at the Mozilla Festival
In a complete paradigm shift of the epic battle between Godzilla and Mothra we are turning our backs on the old claymation medium and embracing the digital age where dinosaurs and diggers (yes, I am aware we are a machine and not a moth) can roam free across the lawless plains of web 2.0. Both […]
Amazing Places Scrapers Go – The Big Clean
Earlier this month, there’s been some underground scraping action happening in Central Europe. We noticed this spark of activity and upon further investigation it was revealed to be a spill over from The Big Clean. At the beginning of this moth, there was Open Scraper Challenge happening in 3 hackerspaces in Czech Republic and Slovakia, in Prague, […]
Scraping guides: Dates and times
Working with dates and times in scrapers can get really tricky. So we’ve added a brand new scraping guide to the ScraperWiki documentation page, giving you copy-and-paste code to parse dates and times, and save them in the datastore. To get to it, follow the “Dates and times guide” link on the documentation page. The […]
Scraping Government Data for the Open Government Data Camp
Come one, come all and gather ye ’round the fantastical scraping table at the Open Government Data Camp at Warsaw. Here you will see such mythical beasts the Irish man with the gift of the gab and the German obsessed with numbers and efficiency he has become part database. So head to Soho Factory in Warsaw, […]
Scraping New Frontiers
Today is Columbus Day in the US (yes, I’m working regardless). So I’ve decided to write a post about discovery. This has been my first full week in America. I have toiled Heathrow Terminal 5, battled through the baffling New York subway and scaled the mountains of food to find, well, not the promised land. […]
Hi, my name’s Zarino
So when Nicola asked me to write a Friday post introducing myself on the ScraperWiki blog, I never thought I’d be writing it during such a momentous few days. I was meant to entertain and beguile you with talk of my MSc research into Open Data at Oxford, tease and tantalise with news of how we’re […]