Crawlera isn't perfect and sometimes it just won't have enough capacity to fetch pages from a specific domain, even after trying many outgoing nodes.


In those cases, Crawlera will return a 503 response with the following header:


X-Crawlera-Error: slavebanned


This happens more often with busy domains such as Amazon and Google.


The only thing you can do in those cases is retry. In order to not waste retries in vain, Crawlera provides a header Retry-After with a hint on how soon you should try again.