ScraperWiki is all the tools you need for Screen Scraping & Data Mining.

Make bad data good, collaborate & discover new datasets.


Tools for developers ...

  • Online code editor, write debug and run Python and PHP in your browser (Ruby coming soon!)
  • ScraperWiki code libraries make scraping, parsing and geocoding simple
  • Store data direct to the datastore
  • API access to data in XML, JSON, PHP, YAML or CSV
  • Schedule scrapers to re-run so your data is always up-to-date
  • Email alerts if your scrapers fail
  • Built in source control

... and for journalists and researchers

  • Search and browse all the datasets in ScraperWiki
  • Export data directly to a spreadsheet
  • Suggest new datasets to the community or request directly from ScraperWiki
  • View data by location

Interesting new datasets

Weekly Reported Crime New York City - For Citywide and Precinct Level

This scraper pulls the reported weekly crime listed in the PDFs on the NYPDs crime statistics page. These crimes include murder, robbery, rape, burglary, felony assault, larceny, and motor vehicle theft.

created by Andrew

PFI visualisation

PFI copied from oil wells

created by Julian_Todd

PFI database

x

created by Julian_Todd

Towed cars in Ljubljana

Towed cars in Ljubljana - taken by Traffic Controllers

created by zejn

UK offshore oil wells

duplicate of other oil well map

created by Julian_Todd

About
Data
Help & support
4iP
© ScraperWiki Ltd