Start a new topic

Environment variable in scrapinghub not detected

I want to read an API as an environment variable in ScrapingHub. 

I have saved it in Spiders > Settings (in the screenshot), and then I have tried to extract it in Python:

api = os.environ.get('OWM_API')

But when I deploy it to the scrapy cloud, the variable api is None. It doesn't detect the environment variable.

In my scrapinghub.yml I only have the project name and the pointer to the requirements file. What am I doing wrong?

1 Comment

I believe the "Raw Settings" panel is not for environment variables, but for Spider settings, similar to what you can find in "" file.

I do something similar, and here's how it works for me:

1. Add a new line to "" like so:

OWM_API = os.environ.get('OWM_API')


2. Wherever you need to get this setting, you can fetch it from the spider's 'settings' attribute: 

your_token = spider.settings['OWM_API']

## or this one, if you're within the Spider class:
your_token = self.settings['OWN_API']


Hope this helps! 

Login to post a comment