Crawlera FAQ

Does Crawlera handle cookies?
Crawlera manages cookies for you by default and retains them for up to 15 minutes since the last request. Crawlera keeps separate groups of cookies per outg...
Wed, 26 Apr, 2017 at 3:24 PM
Does Crawlera solve captchas?
Crawlera does not solve captchas. However, Crawlera will detect redirects to captcha pages (for most sites) and will retry until it hits a clean page. U...
Wed, 26 Apr, 2017 at 3:23 PM
523 Response "Domain Forbidden" - what does it mean?
Some websites have strict policies about crawling and don't allow requests for data extraction purposes. Therefore, they are not accessible through req...
Wed, 20 Sep, 2017 at 11:47 AM
Many failed requests with 429 code. What does it mean?
For each crawlera plan there is a limit to Concurrency (Requests that can be made parallel at a time). Concurrency is cumulative of all domains crawled at a...
Wed, 21 Mar, 2018 at 5:11 AM
Why do I get bad_proxy_auth error while making request using Crawlera?
To use Crawlera, we need to authenticate the Crawlera user. This authentication is done using Crawlera API key. This is different than the Scrapinghub API k...
Wed, 21 Nov, 2018 at 1:05 PM
Why do I get pages with inconsistent content from Google with Crawlera?
In most cases this is to do with the way Google pinpoints the geolocation of Crawlera proxies. We try our best to match the location, but different websites...
Thu, 6 Dec, 2018 at 1:01 PM