With the job console you can open a unix shell directly into the container where your job is running. Once in the console, you can perform tasks such as:

  • tailing the scrapy log (tail -f scrapy.log)
  • accessing the Scrapy Telnet Console with telnet localhost 6023
  • inspecting the files generated by your Scrapy process
  • use top to inspect CPU and memory consumption
  • ... and anything else you can do in a unix console

Accessing the job console is very easy, just click the Console link to switch to the Console tab, where the job console will open:

Finally, a couple of things to keep in mind:

  • the job console is only available for running jobs
  • access is restricted to the directory of your scrapy project (in other words, you don't have root access)