In a previous blog post, I showed how by counting the most common values in each column (like a pivot table, or “group by” in SQL), I managed to make a tool that can automatically summarise datasets. I quickly realised that there were better ways of visualising the data than just showing tables. For example, […]
Two ways you can help guide ScraperWiki’s new platform.
You will have noticed some activity over the past few weeks, as we have begun reaching out about the new ScraperWiki platform. We’ve blogged about some of the new features, and have invited the first ever users outside the office to have a poke around the beta. That initial feedback has been immeasurably helpful, and […]
Summarise #1: Grouping automatically for you
Late at night, after a long conversation in a bar (after Social Media Cafe), Zach mentioned one feature that everyone loved about Kasabi. It had an overview page, which automatically summarised each dataset. Of course, Kasabi did it using linked data – telling you how many of your triples were geographic locations, and how many […]
So web scraping is easy?
Journalists, academics and budding open data hackers often praise ScraperWiki for making web scraping easy. And while it’s true our platform and powerful APIs let you get more done, more easily, the statement still creates some head-scratching at ScraperWiki HQ. That’s because, as far as we can tell, scraping is hard, no matter what platform […]
A small matter of programming
We’re rebuilding ScraperWiki. For three years, we’ve been helping people get, clean and analyse data on the web. Our key insight was that you need to write code to do that, and we should make writing that code as easy as possible. Earlier this year, we realised that that isn’t enough. ScraperWiki Classic, as we […]
Announcing ScraperWiki Premium Accounts!
The most exciting bit about ScraperWiki is how it forms a link between two very different worlds. On the one hand, we love the public good that data liberation enables, and we’re used by everyone from journalists (did you see us on the Guardian front page last week?) to activists (like the guys behind Australian planning […]