The UK Government (via GDS) is gradually rolling out a new performance platform. It’s exciting for lots of reasons, and getting less attention than it deserves. Here are three reasons I’m excited. 1. It makes government services easier to use To me, the most exciting part is that it will make it easier for me, […]
Table Scraping Is Hard
The Problem NHS trusts have been required to publish data on their expenditure over £25,000 in a bid for greater transparency; A well known B2B publisher came to us to aggregate that data and provide them with information spanning across the hundreds of different trusts, such as: who are the biggest contractors across the NHS? […]
Book review: Tableau 8 – the official guide by George Peck
A while back I reviewed Larry Keller’s book The Tableau 8.0 Training Manual, at the same time I ordered George Peck’s book Tableau 8: the official guide. It’s just arrived. The book comes with a DVD containing bonus videos featuring George Peck’s warm, friendly tones and example workbooks. I must admit to being mildly nonplussed at […]
Git!
As software company, use of some sort of software source control system is inevitable, indeed our CEO wrote TortoiseCVS – a file system overlay for the early CVS source control system. For those uninitiated in the joys of software engineering: source control is a system for recording the history of file revisions allowing programmers to […]
A place in the country
Recently Shelter came to us asking for data on house prices across the UK to help them with some research in support of campaign on housing affordability. This is a challenge we’re well suited to address, in fact a large fraction of the ScraperWiki team have scraped property price data for our own purposes. Usually […]
Us and the UN
When you think of humanitarian crises like war, famine, earthquakes, storms and the like you probably don’t think “data”. But data is critical to humanitarian relief efforts, background questions like: What is the country we’re deploying to like in terms of population, geography, infrastructure, economic development, social structures? What resources, financial and human, are available […]
ScraperWiki needs you!
Are you excited by data? Are you impatient to download and explore new datasets? If you answered yes, to either of these questions then you might be interested in a new data scientist position which has arisen here at ScraperWiki. Don’t worry if you’ve never described yourself as a data scientist, we’re looking for people […]
Hi, I’m Sean
My name is Sean Duffy, and I’m an intern at ScraperWiki! I’ve just finished sixth form and in around a month’s time I’ll be starting my degree in Computer Systems and Software Engineering at the University of York. As well as programming, in my spare time I also enjoy electronics, reading and building/flying quadcopters. When […]
Scrape anyone’s Twitter followers
Following our popular tool which makes it easy to scrape and download tweets, we’re pleased to announce a new one to get any Twitter account’s followers. To use it, log into ScraperWiki, choose “Create a new dataset” then pick the tool Then enter the name of the user you want (with or without the @). If they […]
It’s good to share…
As you may have gathered I’m on a journey, I’ve worked as a physicist, a data scientist for 20 years and now I’ve fallen amongst software engineers. There are obvious similarities in what we do, we write code to do stuff. I write code to analyse things and the software engineers write code to do […]