Crawlera Advanced Topics
When a request fails for some reason, Crawlera will include an error code in the X-Crawlera-Error header. Please refer to the API documentation to understan...
Wed, 26 Apr, 2017 at 3:58 PM
X-Crawlera-Profile is a header that is meant to replace X-Crawlera-UA. It not only sets the User-Agent but also applies a set of headers which are actually ...
Wed, 4 Jul, 2018 at 8:40 PM
When Crawlera gets a ban from a target website, it automatically retries the request from another proxy IP. By default, Crawlera re-tries 5 times to ret...
Wed, 15 Apr, 2020 at 5:34 PM
⚠️ Note: Headless browser support is only available in Advanced and Enterprise plans. Overview Crawling with a headless browser is different from t...
Wed, 13 May, 2020 at 9:08 PM
We are changing the behavior of a custom header (X-Crawlera-Profile) that can be sent along with your request to Crawlera. This change is expected to go out...
Fri, 1 Feb, 2019 at 2:50 PM
⚠️ Note: Sessions are only available on Crawlera Advanced, C50, C100, C200 and Enterprise plans. With Crawlera, each request is routed through a differ...
Wed, 15 Apr, 2020 at 2:10 PM