Aidan McGuire – ScraperWiki https://blog.scraperwiki.com Extract tables from PDFs and scrape the web Tue, 09 Aug 2016 06:10:13 +0000 en-US hourly 1 https://wordpress.org/?v=4.6 58264007 Building the Humanitarian Data Exchange team https://blog.scraperwiki.com/2015/05/building-the-humanitarian-data-exchange-team/ Wed, 20 May 2015 13:01:05 +0000 https://blog.scraperwiki.com/?p=758222407 hdx-repo-betaWe’ve been working with the United Nations Office for the Coordination of Humanitarian Affairs (UN-OCHA) to build the Humanitarian Data Exchange (HDX, @humdata).

The aim of the HDX is to bring together data from multiple, open sources and provide it in a unified way to humanitarian workers to help them with their work in the field. As such it’s a good illustration of the type of multi-disciplinary team required to successfully execute on the delivery of an Open Data project.

HDX team at ScraperWiki officesThis project brings together a wide range of skills which we think are essential to the success of such an enterprise.

  • Project sponsor – a critical role in a large organisation, the sponsor promotes the project internally and helps bring in the support of the senior management team.
  • Project manager – holds the project together including building the team, communications, budget and outreach to external stakeholders.
  • Project technical manager – marshalling a group of software developers is a very particular task.
  • Designers – it’s no good getting together a big pile of lovely Open Data if you present it in such a way that the users don’t want to access it, this is why we have designers;
  • Data architects – how to structure data in the Exchange, and describe that data in machine readable forms is a job for a Data Architect;
  • Statisticians – these people are really serious about the provenance of data, where does it come from? how exactly was it collected?;
  • Data managers – these people manage the process of bringing new data into the system;
  • Data scientists – a blend of two skills developer and statistical analysis
  • Lead developer – managers the rest of the team and how they deliver software
  • Front-end developers – do the things relating to user interface, appearance and so forth;
  • Back-end developers – do the things relating to making the data work behind the scenes;
  • System administrators – do the things relating to making a website available and reliable on the web;

This isn’t to say all big Open Data projects need a whole person for each role but this is the range of roles which need to be covered.

Hiring this range of skills is not easy as there is a general skills shortage and stiff competition from the private sector, but as a humanitarian organisation the UN OCHA benefits from volunteer help, most recently in crowd-sourced assistance to get structured data from country situation reports. Read about it here

The Organisations involved in making the Humanitarian Data Exchange happen are:

UN OCHA own the HDX project.  ScraperWiki is providing technical implementation management and data science.
Frog Design is providing the user experience, research and design.  The OKFN is providing its CKAN software and high level strategic advice.  We are a distributed and cohesive team working all over the world doing everything possible to ensure that this amazing project is successful.

Take a little look at the video about the project

Find and share data on the Humanitarian Data Exchange. Follow @humdata on Twitter to keep up to date.

]]>
758222407