Comments on: The story of getting Twitter data and its “missing middle” Extract tables from PDFs and scrape the web Thu, 14 Jul 2016 16:12:42 +0000 hourly 1 By: Francis Irving Mon, 08 Sep 2014 10:24:13 +0000 Crimson Hexagon have a Twitter data programme for academics: (Thanks @EBBreese for pointing this out!)

By: API Måndag – öppna valdata från SVT Pejl, Twitter vs utvecklare & Öppna Helsingborg » Mon, 08 Sep 2014 07:40:26 +0000 […] så stora att de hade råd att köpa data av Twitter. Jag rekommenderar verkligen att ni läser ScraperWikis blogginlägg om detta, det ger god insyn i hur Twitter tänker om små […]

By: Delfin J Paris Fri, 29 Aug 2014 04:55:06 +0000 Bummer about the twitter data pull! I’m still trying to fix my site based on a hint you gave me about pulling the data. Sorry about getting suspended!

By: infographic. Data Viz News [66] - » Titel Mon, 25 Aug 2014 08:27:05 +0000 […] The story of getting Twitter data and its “missing middle” | ScraperWiki […]

By: Data Viz News [66] | Visual Loop Sat, 23 Aug 2014 17:05:05 +0000 […] The story of getting Twitter data and its “missing middle” | ScraperWiki […]

By: Jamie Riddell Fri, 22 Aug 2014 13:18:29 +0000 I’m sorry to see ScraperWiki go, this game is a tough one.

Should you still need to download Twitter data to excel then please consider BirdSong. We can provide full exports of all Twitter followers for any public account and the most recent 3,000 tweets from that same account, to .csv.

We can provide this data for any Twitter account. We also have downloads available for Instagram and Facebook.

We aren’t free but charge per report with no monthly fees or lasting subscription.

By: Andy Cotgreave Thu, 21 Aug 2014 13:44:52 +0000 Hello. I’m really sorry to hear you’ve not fixed the problem. This is a real shame. Sure, at Tableau I can and do use Datasift and it’s fantastically powerful. But you’re dead right in your blog – Tableau/Datasift is not a solution all our customers want to use. This is because Datasift has two costs: the financial one and the infrastructure one.

This is where Scraperwiki has always been a great alternative service for lighter-weight needs. I’m sorry I won’t be able to point people your way for the time being.

It does seem odd as all you are doing is doing the job of the Developer using the Free API.


By: Andrew Fielding Thu, 21 Aug 2014 11:54:31 +0000 Thanks for trying guys.

You’ve hit the nail on the head here. I fall squarely in that middle box. I want “muck about with spreadsheets” and macros and formulae to wrangle information.

I used ScraperWIki to download all the followers of an account to make a PhotoMosaic. I could never justify the time to build code to download that dataset, but ScraperWiki made it easy and I could get on with USING the data.

I’m slowly learning coding, but the barrier of entry to twitter API is now too high – which means that I am losing interest in coding (call me old fashioned but I like to learn with something that is useful to me).

I will continue to use to collect relevant tweets (but for how long?) but a list of an account’s followers? Now in the bucket marked “too hard”. Sad.

I just wish Twitter would stop trying to be like Facebook…