Crawlera Examples

Examples of using Crawlera with different languages and technologies

Using Crawlera with Scrapy
The recommended way to use Crawlera with Scrapy is by using the Crawlera middleware which can be installed with: pip install scrapy-crawlera You can enabl...
Thu, 25 Jul, 2019 at 1:35 PM
Using Crawlera with Splash & Python requests library
You can use Splash with Crawlera to render JavaScript and proxy all requests issued from Splash. This may be necessary if your uses Splash heavily and targe...
Thu, 7 Sep, 2017 at 8:32 PM
Using Crawlera with Python Requests
Here is a code snippet that illustrates how to use Crawlera with Python Requests library: import requests url = "" pr...
Mon, 19 Aug, 2019 at 7:35 PM
Using Crawlera with PHP
Making use of PHP binding for libcurl library: <?php $ch = curl_init(); $url = ''; $proxy = '
Thu, 14 Feb, 2019 at 4:00 PM
Using Crawlera with Ruby
Making use of curb, a Ruby binding for libcurl: require 'curb' url = "" proxy = "
Fri, 26 Apr, 2019 at 2:16 PM
Using Crawlera with Node.js
Making use of request, an HTTP client: module.exports = require('./lib/express'); var express = require('express'); var request = r...
Wed, 5 Jun, 2019 at 8:23 PM
Using Crawlera with Java
NOTE Because of HTTPCLIENT-1649 you should use version 4.5 of HttpComponents Client or later. Extending an example published at The Apache HttpCompo...
Wed, 5 Jun, 2019 at 8:25 PM
Using Crawlera with Java and Jsoup
import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import java.nio.file.Files; import java.nio.file.Paths; import; import java.uti...
Mon, 15 Apr, 2019 at 9:58 PM
Using Crawlera with C#
using System; using System.IO; using System.Net; namespace ProxyRequest { class MainClass { public static void Main(string[] args) ...
Tue, 26 Mar, 2019 at 9:55 PM
Using Crawlera with R
This is an example of using Crawlera with R. # helper functions safe_GET <- purrr::safely(httr::GET) safely_htmlParse <- purrr::safely(htmlParse) ...
Wed, 5 Jun, 2019 at 8:31 PM