Start a new topic

Can't crawl for pages that require log-in

Keeps getting stuck at "/cookie_usage.php"

1 Comment

Would be able to help us out with more details of the issue. For which spider you get this, you can try setting the User Agent through Settings in UI as given in https://support.scrapinghub.com/solution/articles/22000200670-customizing-scrapy-settings-in-scrapy-cloud.


Login to post a comment