0
Answered
e.rawkz 2 months ago in Scrapy Cloud • updated by Pablo Vaz (Support Engineer) 2 months ago 1

IHave a pet project of which scrapes video hosting sites which then return as items the title, video Source URL( stream), and category( depending on the website being scraped) of which using scraping hubs python API client I then manually have to insert the project ID and the specific job Id to them iterate through the items to create a .m3u playlist... The purpose of the project being to agregate videos in one playlist of which one could use VLC(or choice program) .


Here's a quick write-up sample of more and less How I have been iterating to each project
...
list = conn.project_ids()
print("PROJECTS")
print("-#-" * 30)
for index, item in enumerate(list[1::]):
index = str(index)
item = str(item)
project = conn[item]
pspi = project.spiders()
jobs = project.jobs()
for x in pspi:
print("["+ index + "] | PROJECT ID " + item, x['id'], x['tags'])
....
The issue being is that I am unable to then iterate through jobs to then call each job (aware that using "list" is not recommended as it is a python fuction, this is just an example of the proccess more-or-less I go through)...


I understand also that I'm not being very clear as English is not my native language ultimately all I wish to do is 2 iterate through projects jobs to be able to call all job.items from all jobs in the given project...

Answer

Answer
Answered

Hi!


We can't provide coding support through the community forum. Perhaps the best channels to ask are Stack Overflow and Github, many of our best developers contribute there actively.


Best,


Pablo

Answer
Answered

Hi!


We can't provide coding support through the community forum. Perhaps the best channels to ask are Stack Overflow and Github, many of our best developers contribute there actively.


Best,


Pablo