How do I see if there are changes every time I scrape a website?

Start a new topic

Answered

started a topic 11 months ago

Hi everyone,

How can I see if there are changes made to websites on a daily basis?

Best Answer

vaz said 11 months ago

Hi Yizu Token Report,

You can crawl the data desired from a website and deploy a periodic job on a daily basis to compare and check if any changes were made.

A more complex but effective solution is to deploy a periodic job and also using Delta Fetch Addon, which allows you to check only for new items.

Best,

Pablo

1 Comment

said 11 months ago

Answer

Hi Yizu Token Report,

You can crawl the data desired from a website and deploy a periodic job on a daily basis to compare and check if any changes were made.

A more complex but effective solution is to deploy a periodic job and also using Delta Fetch Addon, which allows you to check only for new items.

Best,

Pablo

Scrapinghub Support Center