scrapy.core.downloader.handlers.http11

module documentation

Download handlers for http and https schemes
Variable logger Undocumented
Class _​Request​Body​Producer Undocumented
Class _​Response​Reader Undocumented
Class ​HTTP11​Download​Handler No class docstring; 0/7 instance variable, 0/1 class variable, 1/3 method, 0/1 class method documented
Class ​Scrapy​Agent Undocumented
Class ​Scrapy​Proxy​Agent No class docstring; 0/1 instance variable, 1/2 method documented
Class ​Tunnel​Error An HTTP CONNECT tunnel could not be established by the proxy.
Class ​Tunneling​Agent No summary
Class ​Tunneling​TCP4​Client​Endpoint No summary
Function tunnel​_request​_data Return binary content of a CONNECT request.
logger =

Undocumented

def tunnel_request_data(host, port, proxy_auth_header=None):

Return binary content of a CONNECT request.

>>> from scrapy.utils.python import to_unicode as s
>>> s(tunnel_request_data("example.com", 8080))
'CONNECT example.com:8080 HTTP/1.1\r\nHost: example.com:8080\r\n\r\n'
>>> s(tunnel_request_data("example.com", 8080, b"123"))
'CONNECT example.com:8080 HTTP/1.1\r\nHost: example.com:8080\r\nProxy-Authorization: 123\r\n\r\n'
>>> s(tunnel_request_data(b"example.com", "8090"))
'CONNECT example.com:8090 HTTP/1.1\r\nHost: example.com:8090\r\n\r\n'