We’ve just rolled out a change to the default frequency of new scrapers.

They used to default to running once a day. Now they default to not running at all.

We’ve made this change because people often make new scrapers that aren’t ready yet. These run every day and send annoying emails saying that they’re broken.

Now, when your scraper is ready, choose “edit schedule”, and then from the drop-down select how often you want it to run.

The maximum at the moment is once a day. We plan to offer more frequent runs with premium accounts. Meanwhile, do get in touch with us if you have a particular application which needs high frequency scraping. (See the FAQ “How often does my scraper run?“)

Of course, many scrapers are one offs that don’t need scheduling. If you’re happy with the run you did in the editor, then you don’t need to worry about the scraper running itself and altering your data later.

Unless you specifically scheduled it.