Start a new topic

Inconsistant Scraping on Amazon Category pages

Trying to scrape an amazon.co.uk category page that has two filters applied to it (Brand + Seller) which should give me a list of all my products listed on Amazon.


I want to scrape the following data:


Product Title

Rating

Total Reviews


I have tested the code and re-wrote  a number of times but there I always seem to have some of the content showing somthing else as if it has skipped to another page element. i..e instead of the rating's (4.6 out of 5 stars) it changes to Prime and on these items the Total Reviews data field is completely missing.


The strange thing is it's not all product, just a handful that seem to be effected?

1 Comment

Testing this some more I think it comes down to the 'Selection mode' being set to 'Automatic' and the target html blocks being slightly dynamic in the amount of content they contain.

I have tried changing the Selection Mode to CSS selector but again this causes issues as some blocks have different class names, and I believe that the XPath selector is only available for paid accounts?


Not sure how to fix this as a novice/non coder.


Login to post a comment