If seeing the logs it's not enough, the Page Storage Addon could help seeing the responses Scrapy Cloud is getting from the job's crawl.

1 - Go to https://app.scrapinghub.com/p/PROJECT_ID/addons/page_storage , enable it and configure the settings:

Page storage mode:

  • Cache: Items expire after a month
  • Versioned Cache: Multiple copies are retained, and each one expires after a month

2 - Stored pages are found as collections and they are found:


3 - Each stored page could be downloaded as JSON object or viewed from dash. In order to check the html in a browser, the contents of body field should be saved as html in a new file and open in any browser. 

Fields available per stored page as JSON:

body: html code of the page
url: url of the response
_jobid: job id where the response came from