Sometimes you got many jobs pending on "Next job" queue.


You can try using this curl command:


curl -X POST "https://storage.scrapinghub.com/jobq/<projectid>/<spiderid>/cancel?state=pending&apikey=<your api key>"


The endpoint cancels up to 50000 jobs in a request, on at a time. The timeout is 50 seconds, which is not enough for the 50k, but the command did continue successfully, even after a 504 response.

So just needs to be called multiple times after the previous one finishes (pending job counter stops dropping) to clear all the jobs.


If you need further assistance with this, don't hesitate to contact our support team.