This is a guest blog post by Elizaveta Gaufman, a researcher and lecturer at the University of Tübingen. The theme of my PhD dissertation is enemy images in contemporary Russia. However, I am not only interested in the governmental rhetoric which is relatively easy to analyse, but also in the way enemy images are circulating on […]
Scraperwiki’s response to the Heartbleed security failure
Et tu, Heartbleed “Catastrophic” is the right word. On the scale of 1 to 10, this is an 11. ― Security expert, Bruce Schneier, responds to Heartbleed On Monday the 7th of April 2014, a software flaw was identified which exposed approximately two thirds of the web to the risk of catastrophic security failure. The flaw has […]
Book review: The Signal and the Noise by Nate Silver
Nate Silver first came to my attention during the 2008 presidential election in the US. He correctly predicted the outcome of the November results in 49 of 50 states, missing only on Indiana where Barack Obama won by just a single percentage point. This is part of a wider career in prediction: aside from a […]
Getting Twitter connections
Introducing the Get Twitter Friends tool Our Twitter followers tool is one of our most popular: enter a Twitter username and it scrapes the followers of that account. We were often asked if it’s possible not only to get the users that follow a particular account, but the users that are followed by that account […]
Digitally enhanced social research
Guest post by Dr Rebecca Sandover. The continued expansion of social media activity raises many questions of how this ever-changing digital life spreads ideas and how ‘contagious’ online events arise. Exeter University’s Contagion project has been running since September 2013, funded by the UK Economics and Social Research Council to explore how such events spread […]
Scraping Spreadsheets with XYPath
Spreadsheets are great. They’re ubiquitously available, beaten only by the web pages and the word processor documents. Like the word processor, they’re easy to use and give the user a blank page, but they divide the page up into cells to make sure that the columns and rows all line up. And unlike more complicated […]
Publish your data to Tableau with OData
We know that lots of you use data from our astonishingly simple Twitter tools in visualisation tools like Tableau. While you can download your data as a spreadsheet, getting it into Tableau is a fiddly business (especially where date formatting is concerned). And when the data updates, you’d have to do the whole thing over […]
ScraperWiki Classic retirement guide
In July last year, we announced some exciting changes to the ScraperWiki platform, and our plans to retire ScraperWiki Classic later in the year. That time has now come. If you’re a ScraperWiki Classic user, here’s what will be changing, and what it means for you: Today, we’re adding a button to all ScraperWiki Classic […]
NewsReader – one year on
ScraperWiki has been contributing to NewsReader, an EU FP7 project, for over a year now. In that time, we’ve discovered that all the TechCrunch articles would make a pile 4 metres high, and that’s just one relatively small site. The total volume of news published everyday is enormous but the tools we use to process it […]
Hi, I’m Sophie
Hi, my name is Sophie Buckley, I’m an AS student studying Computing, Maths, Physics and Chemistry who’s interested in programming, games, art and reading. I’m also the latest in a line of privileged ScraperWiki interns and although my time here has been tragically short, the ScraperWiki team have done an amazing job in making every […]