We’ve been teasing and tempting you with blog posts about the first few tools on the new ScraperWiki Beta for a while now. It’s time to let you try them out first-hand. As of right now, the new ScraperWiki Beta is open for you, your aunt, anyone, to sign up for a free community account: […]
From future import x.scraperwiki.com
Time flies when you’re building a platform. At the start of the year, we announced the beginnings of a new, more powerful, more flexible ScraperWiki. More powerful because it exposes industry standards like SQL, SSH, and a persistent filesystem to developers, so they can scrape and crunch and export data pretty much however they like. […]
The next evolution of ScraperWiki
Quietly, over the last few months, we’ve been rebuilding both the backend and the frontend of ScraperWiki. The new ScraperWiki has been built from the ground up to be more powerful for data scientists, and easier to use for everyone else. At its core, it’s about empowering people to take a hold of their data, […]
The state of Twitter: Mitt Romney and Indonesian Politics
It’s no secret that a lot of people use ScraperWiki to search the Twitter API or download their own timelines. Our “basic_twitter_scraper” is a great starting point for anyone interested in writing code that makes data do stuff across the web. Change a single line, and you instantly get hundreds of tweets that you can […]
Software Archaeology and the ScraperWiki Data Challenge at #europython
There’s a term in technical circles called “software archaeology” – it’s when you spend time studying and reverse-engineering badly documented code, to make it work, or make it better. Scraper writing involves a lot of this stuff. ScraperWiki’s data scientists are well accustomed with a bit of archaeology here and there. But now, we want […]
Scraping the protests with Goldsmiths
Zarino here, writing from carriage A of the 10:07 London-to-Liverpool (the wonders of the Internet!). While our new First Engineer, drj, has been getting to grips with lots of the under-the-hood changes which’ll make ScraperWiki a lot faster and more stable in the very near future, I’ve been deploying ScraperWiki out on the frontline, with […]
ScraperWiki scrapers: now 53% more useful!
It’s Christmas come early at ScraperWiki HQ as we deliver—like elves popping boxes under the data digging Christmas tree—a bunch of great new improvements to the ScraperWiki site. We’ve been working on these for a while, so it’s great to finally let you all use them! First up: a new look for your scrapers The […]
Hi, my name’s Zarino
So when Nicola asked me to write a Friday post introducing myself on the ScraperWiki blog, I never thought I’d be writing it during such a momentous few days. I was meant to entertain and beguile you with talk of my MSc research into Open Data at Oxford, tease and tantalise with news of how we’re […]