Sometimes I receive that error from Crawlera and after that I'm not able to scrape anymore. Yesterday it happened during the afternoon and I was not able to scrape again for the rest of the day. Today it was working again but after a while same thing happened.
What is the proper way for minimizing this error and for dealing with it?
Retry and wait?