ScraperWiki has been working on a way to accurately extract tabular data from these sources and making it easy to export to Excel or CSV format. We have been internally testing and improving it with a seemingly infinite array of tables of different complexities. We’re now confident enough in Table Xtract’s accuracy that we are proud to put our name on it and launch a free beta version for you to try.

We are not stopping there though ,as many of you are asking for the ability to create datasets from multiple files/websites and more importantly, refresh them when the source files are updated. Look out for a Table Xtract Enterprise announcement soon!

Try it for free!

Unlike some of the big developer/data science brains we have at ScraperWiki, I have a different role – making sure what we build adds enough value that our customers are willing to pay for some of it!

I started out working life in the RAF as a trainee pilot but a hockey stick to the eye cut that short. It did allow me to start down a career with a more technical bent by completing a Masters in Information Systems and Technology at City University.

From there I was a Sales Engineer for different software vendors and moved into product management. I have worked for different sized companies from the very large (IBM) to the very small (my own startup building tools for product managers) before arriving at ScraperWiki.

Products are created to solve problems for people/companies and therefore make their lives easier, so my first task is to workout what problems our customers and others interested in data experience with ScraperWiki and the tools have today. ScraperWiki can then evolve to become the tool of choice for tomorrow 🙂

Let me know if you have any feedback or ideas – I am really interested to hear your thoughts.

My first task is to let you know about the “Search for Tweets” tool on the new ScraperWiki platform.

To understand how the Twitter tool came about, it is useful to understand some of the background.

On ScraperWiki Classic, we found that a large number of people were forking a standard Twitter scraper and simply changing the search term in the code. The search terms were incredibly varied but customers were certainly getting value from a relatively easy way of accessing Twitter data.

This behaviour influenced the tool strategy as a way of making data available to those of us that don’t have the time, inclination or skill to build our own scrapers. The “Search for Tweets” tool couldn’t get any easier: simply type your search terms and you will be presented with a table view of the last 7 or so days of tweets with that content.

Once you have used the Search for Tweets tool to to find your data you can use other tools such as the “Summarise this Data” or “Download as a Spreadsheet” tools to clean up, analyse and visualise the data. Again, these tools don’t require any coding.

One thing to note is that we can only provide access to about 1 week’s worth of data due to some limitations of the Twitter API but that should still provide many of you with some great data to analyse. And as time passes on, the tool continues gathering search results, so the longer you leave it, the more data you collect.

We have written a great little help article that shows how to use the tool. Why not try it out?

As a product manager, I am keen to hear any feedback you might have on this or any other aspects of ScraperWiki – you can contact me at ed@scraperwiki.com.