My spiders return inconsistent results. I am working on the website, https://www.ifixit.com/. How can I have same number of items every time I run the spider?
Unfortunately this is not possible to guarantee. Even with the best plans offered and the most strictly monitoring, some items got missing due other parameters beyond our control. Mainly due the backend issues or changes from the target domain side.
Many of our clients consider using DotScrapy persistence add-on to have a more robust control on the items extracted.
Customer support service by UserEcho