By default, any Scrapy project deployed to Scrapy Cloud runs on Python 2.7. If you are creating your spiders using Python 3 features, you have to define the Python 3 stack as the default environment for your project.
You can do this by adding a
stacks section in your
scrapinghub.yml file, such as:
projects: default: 123 stacks: default: scrapy:1.3-py3
The stack name is composed like this:
scrapy:<scrapy version>[-py3]. The lack of the
-py3 suffix in the stack name means that it's a Python 2.7 stack.
NOTE: stacks 1.1-py3 & 1.2-py3 use Python 3.5, stacks 1.3-py3 and newer use Python 3.6
scrapy:1.3-py3: Scrapy 1.3 running on Python 3.
scrapy:1.3: Scrapy 1.3 running on Python 2.7.
For more information, see Scrapy Cloud Stacks.