I've spent hours trying to find a solution to this.
When trying to do a shub deploy, this happens:
Entrypoint container is created successfully >>> Checking python dependencies awscli 1.10.52 has requirement botocore==1.4.42, but you have botocore 1.10.1. Warning: Pip checks failed, please fix the conflicts. {"message": "Dependencies check exit code: 1", "details": "Pip checks failed, please fix the conflicts", "error": "requirements_error"} {"status": "error", "message": "Requirements error"} Deploy log location: /var/folders/qc/dzs6cch91lq21cb50q2c7zhm0000gn/T/shub_deploy_e0o02o4v.log Error: Deploy failed: b'{"status": "error", "message": "Requirements error"}'
My requirements.txt looks like this:
Distance==0.1.3 nltk==3.2.5 Scrapy==1.5.0 tweepy==3.5.0 scrapinghub==2.0.3 setuptools==36.5.0 gensim==3.4.0 msgpack_python==0.5.6 PyDispatcher==2.0.5
My scrapinghub.yml looks like this:
project: 285313 requirements: file: requirements.txt stacks: default: scrapy:1.1-py3
It seems that the awscli version is being overrided somehow, as the awscli version that I have installed on my computer is updated.
Pip freeze, awscli==1.15.1 (on my computer), but when I do a shub deploy, it says that I have awscli==1.10.52
It seems like dependency gensim==3.4.0 enforces downgrading botocore from 1.4.42 to 1.10.1 which breaks awscli requirements. Awscli is part of the 1.1-py3 stack (https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-1.1-py3/requirements.txt#L9)
Adding awscli==1.15.1 to your requirements.txt should fix the issue.
This worked, thank you!
I'm glad it worked.
Movere
I've spent hours trying to find a solution to this.
When trying to do a shub deploy, this happens:
My requirements.txt looks like this:
My scrapinghub.yml looks like this:
It seems that the awscli version is being overrided somehow, as the awscli version that I have installed on my computer is updated.
Pip freeze, awscli==1.15.1 (on my computer), but when I do a shub deploy, it says that I have awscli==1.10.52
It seems like dependency gensim==3.4.0 enforces downgrading botocore from 1.4.42 to 1.10.1 which breaks awscli requirements. Awscli is part of the 1.1-py3 stack (https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-1.1-py3/requirements.txt#L9)
Adding awscli==1.15.1 to your requirements.txt should fix the issue.
- Oldest First
- Popular
- Newest First
Sorted by Oldest Firstnestor
It seems like dependency gensim==3.4.0 enforces downgrading botocore from 1.4.42 to 1.10.1 which breaks awscli requirements. Awscli is part of the 1.1-py3 stack (https://github.com/scrapinghub/scrapinghub-stack-scrapy/blob/branch-1.1-py3/requirements.txt#L9)
Adding awscli==1.15.1 to your requirements.txt should fix the issue.
Movere
This worked, thank you!
nestor
I'm glad it worked.
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 180 topics