Increasing Government impact and usability
with data science and data engineering
Governments everywhere are making their services easier to use, making full use of the web and mobile. This reduces hassle for citizens and businesses, and saves money for Government.
We’ve been a leader in the new Government digital space since its inception. Agile, user-centred development, combined with skilled data analysis befitting the era of Big Data, are changing how people perceive Government.
Governments are using ScraperWiki to:
- Build usable, data-rich web applications
- Engineer sophisticated analytics systems
- Empower teams to convert complex data themselves
Contact sales
Case Study: GOV.UK Performance Platform
The Government Digital Service (GDS) is tasked with transforming UK government IT services into world-class products and experiences with a strong user focus.
ScraperWiki were requested by the Performance Platform team in Government Digital Services to deliver 37 activity dashboards that provide departmental Communications Officers with statistics on how users have accessed the content for which they are responsible. We created systems to collate the data from Google Analytics and the GOV.UK comments system, and integrate it into the GDS Performance Platform.
Our work included conducting a business analysis to refine the customer requirements to technical specifications. We then implemented the dashboards to those technical specifications. It was technically challenging as the platform is bespoke and under active development. ScraperWiki are ideally suited to this type of project work since we have experience in data management and analysis, as well as an understanding of the underpinning technology for the Performance Platform.
Scraperwiki did a great job, not just in building a bunch of dashboards quickly, but in helping the Performance Platform team to understand how they could most effectively work with external partners in future.
Richard Sargeant
Director of Performance & Delivery, GDS
Case Study: Office for National Statistics – DataBaker
The Office for National Statistics (ONS) is responsible for collecting and publishing statistics related to the economy, population and society of the UK.
They came to ScraperWiki with many spreadsheets which they use to publish statistics. They wanted an effective way to convert them to the data format for their new Data Explorer site, making it easier for end users to analyse the data.
ScraperWiki created DataBaker, a “domain specific language” designed specifically for expressing the headings, subheadings, tabs and other formatting of spreadsheets, and outputting them as a structured Comma Separated Value file. This made the simple cases simple, and the complicated cases possible.
A big factor in the success of the project was embedding skills transfer. The ONS team was trained to be able to create, run and maintain data transformations themselves. The project is open source, so you can read more in the project documentation.
Who knew a little Python app spitting out CSVs could make people so happy but thank you team @ScraperWiki – great stuff 🙂
Matt Jukes
Head of Digital Transformation, ONS
Case Study: GOV.UK Data Migration
The Government Digital Service (GDS) is tasked with transforming UK government IT services into world-class products and experiences with a strong user focus.
In 2010, they came to ScraperWiki, to migrate content from dozens of government websites and portals into one single domain: www.gov.uk.
During the initial stage, GDS programmers collaborated on their own scrapers via ScraperWiki’s data hub. And in later stages, ScraperWiki’s expert data scientists were brought in to help migrate 100s of sites.
Without the dynamic approach that ScraperWiki offers we would have had to rely on writing lots of redundant code to scrape the websites and munch together the different datasets.
Normally that would have taken our developers a significant amount of time, and would have been hard to maintain.
We were delighted to use ScraperWiki. It avoided a huge headache.
Tom Loosemore
Deputy Director, GDS