scheduler – ScraperWiki https://blog.scraperwiki.com Extract tables from PDFs and scrape the web Tue, 09 Aug 2016 06:10:13 +0000 en-US hourly 1 https://wordpress.org/?v=4.6 58264007 Scheduling: A scrape a day keeps stale data away https://blog.scraperwiki.com/2011/08/scheduling-a-scrape-a-day-keeps-stale-data-away/ Wed, 24 Aug 2011 19:03:25 +0000 http://blog.scraperwiki.com/?p=758215286 We’ve just rolled out a change to the default frequency of new scrapers.

They used to default to running once a day. Now they default to not running at all.

We’ve made this change because people often make new scrapers that aren’t ready yet. These run every day and send annoying emails saying that they’re broken.

Now, when your scraper is ready, choose “edit schedule”, and then from the drop-down select how often you want it to run.

The maximum at the moment is once a day. We plan to offer more frequent runs with premium accounts. Meanwhile, do get in touch with us if you have a particular application which needs high frequency scraping. (See the FAQ “How often does my scraper run?“)

Of course, many scrapers are one offs that don’t need scheduling. If you’re happy with the run you did in the editor, then you don’t need to worry about the scraper running itself and altering your data later.

Unless you specifically scheduled it.

]]>
758215286