Scrapy Cloud FAQ

Which services are available on Scrapy Cloud?
Here is a diagram that shows the different components and services of the Scrapinghub platform, and how they typically interact.
Wed, 26 Apr, 2017 at 1:40 PM
Portia or Scrapy?
Let us help you decide which technology best fits you So, you just started creating a Scrapy Cloud project and got a daunting choice: "Build spider...
Thu, 4 Jan, 2018 at 3:23 PM
What is a Scrapy Cloud Unit?
Scrapy Cloud jobs run in containers. These containers can be of different sizes defined by Scrapy Cloud units. A Scrapy Cloud provides: 1 GB of RAM 2....
Wed, 26 Jul, 2017 at 3:38 PM
Why do I see only one Scrapy Cloud unit in Dashboard after purchasing an extra unit?
When you subscribe to Scrapinghub, 1 Scrapy Cloud unit is given for free. This free unit has features such as data retention for 7 days and a run time limit...
Tue, 19 Sep, 2017 at 5:40 AM
Can I use an HTTP cache on Scrapy Cloud?
Yes, you can. To do that, you have to enable Scrapy's HTTP cache extension by setting HTTPCACHE_ENABLED to True in your project settings. Th...
Fri, 2 Jun, 2017 at 11:30 PM
What are the IP addresses used by Scrapy Cloud?
Scenario: You have e.g. a MongoDB server that you would like your spiders to write to, and you would like to open access to that server only from Scrapy Clo...
Wed, 26 Apr, 2017 at 1:35 PM
Can I run Scrapy Cloud on my environment?
Scrapy Cloud is only offered as a hosted service. If you're interested in hosting your own version of Scrapy Cloud, get in touch and let us know.
Wed, 26 Apr, 2017 at 1:29 PM
What are the differences between running a spider locally and on Scrapy Cloud?
Here are a few things that work differently on Scrapy Cloud compared to a default Scrapy configuration. All these can be overridden of course:  AutoThro...
Wed, 26 Apr, 2017 at 1:34 PM
Where can I find my activation (or re-activation) link?
A common issue is not to receive your activation link when you subscribe to a new account in Scrapinghub or when you forgot your password and trying to send...
Wed, 26 Apr, 2017 at 1:37 PM
How can I migrate my Python eggs to Scrapy Cloud 2.0?
Scrapinghub will support eggs within the Scrapinghub Dashboard through the end of 2016. After that, project dependencies outside of the Scrapy Cloud stack m...
Wed, 26 Apr, 2017 at 1:33 PM