Crawlera Examples

Examples of using Crawlera with different languages and technologies

Using Crawlera with Scrapy
The recommended way to use Crawlera with Scrapy is by using the Crawlera middleware which can be installed with: pip install scrapy-crawlera You can enabl...
Thu, 22 Mar, 2018 at 7:31 PM
Using Crawlera with Splash & Python requests library
You can use Splash with Crawlera to render JavaScript and proxy all requests issued from Splash. This may be necessary if your uses Splash heavily and targe...
Thu, 7 Sep, 2017 at 8:32 PM
Using Crawlera with Python Requests
Here is a code snippet that illustrates how to use Crawlera with Python Requests library: import requests url = "http://httpbin.org/ip" pr...
Thu, 28 Jun, 2018 at 11:40 AM
Using Crawlera with PHP
Making use of PHP binding for libcurl library: <?php $ch = curl_init(); $url = 'https://twitter.com/'; $proxy = 'proxy.crawlera.com:...
Wed, 20 Jun, 2018 at 8:42 PM
Using Crawlera with Ruby
Making use of curb, a Ruby binding for libcurl: require 'curb' url = "https://twitter.com" proxy = "proxy.crawlera.com:8010...
Fri, 28 Apr, 2017 at 11:41 PM
Using Crawlera with Node.js
Making use of request, an HTTP client: module.exports = require('./lib/express'); var express = require('express'); var request ...
Fri, 28 Apr, 2017 at 11:49 PM
Using Crawlera with Java
NOTE Because of HTTPCLIENT-1649 you should use version 4.5 of HttpComponents Client or later. Extending an example published at The Apache HttpCompo...
Wed, 3 May, 2017 at 10:24 PM
Using Crawlera with C#
using System; using System.IO; using System.Net; namespace ProxyRequest { class MainClass { public static void Main (string[] args)...
Mon, 8 May, 2017 at 1:04 PM
Using Crawlera with R
This is an example of using Crawlera with R. # helper functions safe_GET <- purrr::safely(httr::GET) safely_htmlParse <- purrr::safely(htmlParse...
Thu, 13 Jul, 2017 at 5:24 PM
Using Crawlera with PhantomJS and CasperJS
PhantomJS Using Crawlera in Phantom object: var page = require('webpage').create(); phantom.setProxy('proxy.crawlera.com', &#...
Mon, 16 Oct, 2017 at 8:39 PM