I am trying to download comments from kickstarter campaigns by using Portia. For each campaign, there is a "show older comments" link after the most recent set of 50 comments. Selecting that link shows an additional 50 comments. I'm not a developer and have been trying to come up with a work around to download all comments with one scrape job.
I've been able to use the "configure url patterns" feature to go from 50 to 100 comments (details below). However, the crawl stops there and doesn't follow the link to the next set of 50 comments (in total there are ~600 comments). Since the "show older comments" URL doesn't change, I'm not sure why it is stopping. Any help would be much appreciated!
Regular expression for the "follow links that match this expression": (https:\/\/www\.kickstarter\.com\/projects\/1865494715\/apollo-7-worlds-most-compact-true-wireless-earphon\/comments\?cursor=14162300)+
Customer support service by UserEcho