time – ScraperWiki https://blog.scraperwiki.com Extract tables from PDFs and scrape the web Tue, 09 Aug 2016 06:10:13 +0000 en-US hourly 1 https://wordpress.org/?v=4.6 58264007 What’s Twitter time zone data good for? https://blog.scraperwiki.com/2014/06/whats-twitter-time-zone-data-good-for/ https://blog.scraperwiki.com/2014/06/whats-twitter-time-zone-data-good-for/#comments Thu, 05 Jun 2014 10:08:20 +0000 https://blog.scraperwiki.com/?p=758221842 2744390812_c6e2aa449b_o

Curioso elemento el tiempo” by leoplus, available under a Creative Commons Attribution-ShareAlike license.

The Twitter friends tool has just been improved to retrieve the time zone of users. This is actually more useful than it first might sound.

If you’ve looked at Twitter profiles before, you’ve probably noticed that users can, and sometimes do, enter anything they like as their location.

Looking at @ScraperWiki‘s followers, we can see from a small snippet of users that this can sometimes give us messy data:

...Denver. & Beyond
Hyper Island | Stockholm
London
Manchester
Niteroi, Brazil
Somerset
There's a wine blog too .....
London / Berkshire...

People may enter the same location in a number of ways, and may provide data that isn’t even a location.

Locations from time zones

If we look at users’ time zones, Twitter only allows users to pick from a certain number of well-defined time zones. (There’s 141 in total; I’ve collated the entire set here.) The data this returns is much neater and we’d expect that this typically reflects the user’s home location:

...Abu Dhabi
Adelaide
Alaska
Almaty
America/Toronto
Amsterdam...

We find far fewer unique time zone data entries than unique location data for @ScraperWiki’s followers: there are 1586 different location entries, but just 106 time zones. If we wanted to discover which countries or regions our users are likely to be, the time zone data would be far simpler to work with.

Furthermore, time zone data can give us insight into the location of Twitter users who don’t specify their location if they’ve selected a time zone.

For ScraperWiki’s followers, we found 670 of them had an empty location and around the same number had an empty time zone. But, far fewer user accounts (only 255) have both of these fields empty. So, in some cases, we could have a good guess at the location for users who we couldn’t previously from the data the tool was providing.

We’re always working to improve the Twitter tools! If you have ideas for features you’d like to see, let us know!

]]>
https://blog.scraperwiki.com/2014/06/whats-twitter-time-zone-data-good-for/feed/ 2 758221842
Scraping guides: Dates and times https://blog.scraperwiki.com/2011/10/scraping-guides-dates-and-times/ Wed, 12 Oct 2011 12:58:32 +0000 http://blog.scraperwiki.com/?p=758215625 Working with dates and times in scrapers can get really tricky. So we’ve added a brand new scraping guide to the ScraperWiki documentation page, giving you copy-and-paste code to parse dates and times, and save them in the datastore.

To get to it, follow the “Dates and times guide” link on the documentation page.

The guide is available in RubyPython and PHP. Enjoy!

]]>
758215625