By default, any Scrapy project deployed to Scrapy Cloud runs on Python 2.7. If you are creating your spiders using Python 3 features, you have to define the Python 3 stack as the default environment for your project.

You can do this by adding a stacks section in your scrapinghub.yml file, such as:


projects:
    default: 123
stacks:
    default: scrapy:1.3-py3



The stack name is composed like this: scrapy:<scrapy version>[-py3]. The lack of the -py3 suffix in the stack name means that it's a Python 2.7 stack.

NOTE: stacks 1.1-py3 & 1.2-py3 use Python 3.5, stacks 1.3-py3 and newer use Python 3.6


Examples:

  • scrapy:1.3-py3: Scrapy 1.3 running on Python 3.
  • scrapy:1.3: Scrapy 1.3 running on Python 2.7.


For more information, see Scrapy Cloud Stacks.