Crawlera FAQ

Can I use Crawlera for sites that require login?
If you're attempting to use Crawlera to avoid getting blocked, the website will still be able to track you (with the user you logged in) and Crawlera w...
Wed, 26 Apr, 2017 at 3:20 PM
Can I try Crawlera with a trial account?
With our Basic and Advanced plans you can try Crawlera for 14 days or 10,000 requests (whichever comes first). After that you will automatically be billed ...
Mon, 10 Feb, 2020 at 3:35 PM
Does Crawlera handle cookies?
In Basic and Advanced plans, Crawlera discards cookies to protect customer privacy and optimize performance to the crawlera sites, because cookies are often...
Fri, 24 Apr, 2020 at 8:11 AM
Does Crawlera solve captchas?
Crawlera does not solve captchas. However, Crawlera will detect redirects to captcha pages (for most sites) and will retry until it hits a clean page. U...
Wed, 26 Apr, 2017 at 3:23 PM
523 Response "Domain Forbidden" - what does it mean?
Some websites have strict policies about crawling and don't allow requests for data extraction purposes. Therefore, they are not accessible through req...
Wed, 20 Sep, 2017 at 11:47 AM
Many failed requests with 429 code. What does it mean?
For each crawlera plan there is a limit to Concurrency (Requests that can be made parallel at a time). Concurrency is cumulative of all domains crawled at a...
Wed, 21 Mar, 2018 at 5:11 AM
Why do I get bad_proxy_auth error while making request using Crawlera?
To use Crawlera, we need to authenticate the Crawlera user. This authentication is done using Crawlera API key. This is different than the Scrapinghub API k...
Wed, 21 Nov, 2018 at 1:05 PM
Why do I get pages with inconsistent content from Google with Crawlera?
In most cases this is to do with the way Google pinpoints the geolocation of Crawlera proxies. We try our best to match the location, but different websites...
Thu, 6 Dec, 2018 at 1:01 PM
Will my API Key change when I upgrade or downgrade Crawlera plans?
Crawlera plans can be upgraded or downgraded anytime, it would not affect the existing API key. API keys are mapped to the account and not to the plans. Any...
Fri, 28 Jun, 2019 at 10:41 AM
Why do I see duplicate requests?
If you see duplicate requests and if the first request response code is 301 with HTTP and the next one is 200 with HTTPS, then that means the requested webs...
Fri, 15 May, 2020 at 9:05 AM