It seems Distil is blocking my use of Splash now. I'm not sure if the site I'm accessing just started using Distil, or if Distil just started detecting Splash.
Is there any information as to how Distil is detecting Splash? I've seen examples for Selenium needing to delete certain properties from document.window, but am unclear as to exactly how Splash is automating things, and what it might stick in the browser that make it detectable.
I did find https://www.quora.com/How-do-you-crawl-Crunchbase-and-bypass-the-bot-protection , where Pablo Hoffman (scrapinghub co-founder) suggests contacting scrapinghub to help with crawling. I'm not sure what the costs for a full consulting gig to do this are (any estimates?)
I'm already using scrapinghub/splash and pay for an instance, but if it's impossible to get through Distil, I'll just have to turn off my spaslh JS instance and remove the feature, so any pointers (public or private to email@example.com) would be appreciated!
Customer support service by UserEcho