Start a new topic
Answered

How to connect Selenium PhantomJS to Crawlera?

Hello, I encountered a problem while trying to use Selenium's PhantomJS web driver with specified proxy - crawlera. There is an article about polipo with selenium, but it won't work and anyway, most of the time I work with Windows, so polipo will be problematic. 


I tried to do it like:

service_args = [
'--proxy="proxy.crawlera.com:8010"',
'--proxy-auth="apikeyweirdcode:"'
] self.driver = webdriver.PhantomJS(service_args=service_args)
But it won't work, tried with colon and without, with double quotes, without and always when I scrape whatsmyip website with Selenium (by Selenium Middleware, I get a page then in middleware I get specific div with IP address and then print it) I get my own IP. After this Selenium printing, I get to scrapy's shell and here when I take the same div with IP, I get some Crawlera's IP, not mine. Did anyone use selenium with crawlera and it worked? Cheers.

Best Answer

Hey TeamScrapy, we have provided useful documentation to use Crawlera with different languages and browsers, please check our section: Crawlera Examples


If you still having issues to connect Crawlera, please consider to Hire our experts to provide coding assistance, set or even deploy the crawlers for you. If interested let us know through: https://scrapinghub.com/quote


Best regards,


Pablo

1 Comment

Answer

Hey TeamScrapy, we have provided useful documentation to use Crawlera with different languages and browsers, please check our section: Crawlera Examples


If you still having issues to connect Crawlera, please consider to Hire our experts to provide coding assistance, set or even deploy the crawlers for you. If interested let us know through: https://scrapinghub.com/quote


Best regards,


Pablo

Login to post a comment