Starting Oct 10th, 2017 Periodic Jobs is a feature available only to Scrapy Cloud paid plans.


You can configure a Periodic Job to run a spider at specific intervals. To add a periodic job, navigate to the project and click the Periodic Jobs link on the left sidebar.



On the Periodic Jobs page click on the Add periodic job button, then provide the details of the required schedule. In the below example, the spider is being set to run every Thursday at 3.00 UTC time.



You can provide Tags and Arguments that should be used to launch the job.


For a case where you want to run a job every 5 hours, you would have to create multiple Periodic Jobs for the same spider where the time difference would be 5 hours. Like in the below example, the job is scheduled to run every 5 hours on every Monday.



Note: there's a limit of 1000 periodic jobs per project and 10000 periodic jobs per organisation.