ScraperWiki is a powerful cloud-based service that lets you scrape data from online documents and websites.

When you write a scraper – a script to pull information from a web resource and then parse out the bits you want – it will execute inside the ScraperWiki environment.

You can store the data that is scraped inside a data store and then access the data from outside the ScraperWiki environment using their API. Scrapers can be written in one of several different languages – Ruby, PHP and Python.

ScraperWiki and Tropo operate in a very similar way. The Tropo scripting environment allows you to write scripts in one of several different languages, including Ruby, PHP and Python (Groovy and JavaScript are also supported).

Your script executes inside the Tropo environment, which means you can make direct connections to external resources – like the ScraperWiki API – from within your executing script. There is no need for extra HTTP overhead, and the additional step of posting to a back end server to connect to other APIs or resources.

In the following screencast, I demonstrate how to use Tropo and ScraperWiki to quickly and easily build an airport information system for the Philadelphia International Airport.

All of the code for this example can be found here. If you’d like to view the actual scraper I wrote on ScraperWiki, you can find it here.

This is still a work in progress – I’d like to run this script multiple times per day (ideally, maybe once an hour) to get updates to flight information and ensure that the app has the most up to date flight status. The voice dialog could also use a little tweaking, and I’d like to offer the option of repeating the information.

But even with these refinements aside, it is evident how combining these two powerful cloud resources can generate a pretty useful application in a very short time.

Tropo and ScraperWiki are a powerful combination. Happy flying!