How can we help you today?

Start a new topic
Discussions Scrapy Cloud Technical Help
Answered

Passing arguments to the spider via jobs API

I want to run a job using jobs API. 

I am passing the project id and the spider name.

I would like to pass arguments to the spider as well.

  1. How do I pass them via the REST call?
  2. How can the spider consume those arguments? Will they be passed via __init__ ?

Thanks

Avishay

Best Answer

Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:

$ curl -u APIKEY: https://app.scrapinghub.com/api/run.json -d project=123 -d spider=somespider -d spiderarg1=example
3 CommentsSorted by Oldest First

me too

Answer

Any parameter not listed here: https://doc.scrapinghub.com/api/jobs.html#run-json, will be treated as an argument. That means you can pass arguments like this:

$ curl -u APIKEY: https://app.scrapinghub.com/api/run.json -d project=123 -d spider=somespider -d spiderarg1=example
1 person likes this

how to capture the argument spiderarg1=example in this spider? 

Login to post a comment
More topics in Technical Help
See all 182 topics