Pool Manager#

class urllib3.PoolManager(num_pools=10, headers=None, **connection_pool_kw)#

Bases: RequestMethods

Allows for arbitrary requests while transparently keeping track of necessary connection pools for you.

Parameters

num_pools (int) – Number of connection pools to cache before discarding the least recently used pool.
headers (Mapping[str, str] | None) – Headers to include with all requests, unless other headers are given explicitly.
**connection_pool_kw (Any) – Additional parameters are used to create fresh urllib3.connectionpool.ConnectionPool instances.

Example:

import urllib3

http = urllib3.PoolManager(num_pools=2)

resp1 = http.request("GET", "https://google.com/")
resp2 = http.request("GET", "https://google.com/mail")
resp3 = http.request("GET", "https://yahoo.com/")

print(len(http.pools))
# 2

clear()#

Empty our store of pools and direct them all to close.

This will not affect in-flight connections, but they will not be re-used after completion.

Return type: None

connection_from_context(request_context)#

Get a urllib3.connectionpool.ConnectionPool based on the request context.

request_context must at least contain the scheme key and its value must be a key in key_fn_by_scheme instance variable.

Parameters: request_context (dict[str, Any]) –
Return type: HTTPConnectionPool

connection_from_host(host, port=None, scheme='http', pool_kwargs=None)#

Get a urllib3.connectionpool.ConnectionPool based on the host, port, and scheme.

If port isn’t given, it will be derived from the scheme using urllib3.connectionpool.port_by_scheme. If pool_kwargs is provided, it is merged with the instance’s connection_pool_kw variable and used to create the new connection pool, if one is needed.

Parameters

host (str | None) –
port (int | None) –
scheme (str | None) –
pool_kwargs (dict[str, Any] | None) –

Return type

HTTPConnectionPool

connection_from_pool_key(pool_key, request_context)#

Get a urllib3.connectionpool.ConnectionPool based on the provided pool key.

pool_key should be a namedtuple that only contains immutable objects. At a minimum it must have the scheme, host, and port fields.

Parameters

pool_key (PoolKey) –
request_context (dict[str, Any]) –

Return type

HTTPConnectionPool

connection_from_url(url, pool_kwargs=None)#

If pool_kwargs is not provided and a new pool needs to be constructed, self.connection_pool_kw is used to initialize the urllib3.connectionpool.ConnectionPool. If pool_kwargs is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided pool_kwargs are not used.

Parameters

url (str) –
pool_kwargs (dict[str, Any] | None) –

Return type

HTTPConnectionPool

proxy: urllib3.util.url.Url | None = None#

proxy_config: urllib3._base_connection.ProxyConfig | None = None#

request(method, url, body=None, fields=None, headers=None, json=None, **urlopen_kw)#

Make a request using urlopen() with the appropriate encoding of fields based on the method used.

This is a convenience method that requires the least amount of manual effort. It can be used in most situations, while still having the option to drop down to more specific methods when necessary, such as request_encode_url(), request_encode_body(), or even the lowest level urlopen().

Parameters

method (str) –
url (str) –
body (Optional[Union[bytes, IO[Any], Iterable[bytes], str]]) –
fields (Optional[Union[Sequence[Union[Tuple[str, Union[str, bytes, Tuple[str, Union[str, bytes]], Tuple[str, Union[str, bytes], str]]], RequestField]], Mapping[str, Union[str, bytes, Tuple[str, Union[str, bytes]], Tuple[str, Union[str, bytes], str]]]]]) –
headers (Optional[Mapping[str, str]]) –
json (Any | None) –
urlopen_kw (Any) –

Return type

BaseHTTPResponse

request_encode_body(method, url, fields=None, headers=None, encode_multipart=True, multipart_boundary=None, **urlopen_kw)#

Make a request using urlopen() with the fields encoded in the body. This is useful for request methods like POST, PUT, PATCH, etc.

When encode_multipart=True (default), then urllib3.encode_multipart_formdata() is used to encode the payload with the appropriate content type. Otherwise urllib.parse.urlencode() is used with the ‘application/x-www-form-urlencoded’ content type.

Multipart encoding must be used when posting files, and it’s reasonably safe to use it in other times too. However, it may break request signing, such as with OAuth.

Supports an optional fields parameter of key/value strings AND key/filetuple. A filetuple is a (filename, data, MIME type) tuple where the MIME type is optional. For example:

fields = {
    'foo': 'bar',
    'fakefile': ('foofile.txt', 'contents of foofile'),
    'realfile': ('barfile.txt', open('realfile').read()),
    'typedfile': ('bazfile.bin', open('bazfile').read(),
                  'image/jpeg'),
    'nonamefile': 'contents of nonamefile field',
}

When uploading a file, providing a filename (the first parameter of the tuple) is optional but recommended to best mimic behavior of browsers.

Note that if headers are supplied, the ‘Content-Type’ header will be overwritten because it depends on the dynamic random boundary string which is used to compose the body of the request. The random boundary string can be explicitly set with the multipart_boundary parameter.

Parameters

method (str) –
url (str) –
fields (Optional[Union[Sequence[Union[Tuple[str, Union[str, bytes, Tuple[str, Union[str, bytes]], Tuple[str, Union[str, bytes], str]]], RequestField]], Mapping[str, Union[str, bytes, Tuple[str, Union[str, bytes]], Tuple[str, Union[str, bytes], str]]]]]) –
headers (Optional[Mapping[str, str]]) –
encode_multipart (bool) –
multipart_boundary (str | None) –
urlopen_kw (str) –

Return type

BaseHTTPResponse

request_encode_url(method, url, fields=None, headers=None, **urlopen_kw)#

Make a request using urlopen() with the fields encoded in the url. This is useful for request methods like GET, HEAD, DELETE, etc.

Parameters

method (str) –
url (str) –
fields (Optional[Union[Sequence[Tuple[str, Union[str, bytes]]], Mapping[str, Union[str, bytes]]]]) –
headers (Optional[Mapping[str, str]]) –
urlopen_kw (str) –

Return type

BaseHTTPResponse

urlopen(method, url, redirect=True, **kw)#

Same as urllib3.HTTPConnectionPool.urlopen() with custom cross-host redirect logic and only sends the request-uri portion of the url.

The given url parameter must be absolute, such that an appropriate urllib3.connectionpool.ConnectionPool can be chosen for it.

Parameters

method (str) –
url (str) –
redirect (bool) –
kw (Any) –

Return type

BaseHTTPResponse

class urllib3.ProxyManager(proxy_url, num_pools=10, headers=None, proxy_headers=None, proxy_ssl_context=None, use_forwarding_for_https=False, proxy_assert_hostname=None, proxy_assert_fingerprint=None, **connection_pool_kw)#

Bases: PoolManager

Behaves just like PoolManager, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs.

Parameters

proxy_url (str) – The URL of the proxy to be used.
proxy_headers (Mapping[str, str] | None) – A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication.
proxy_ssl_context (ssl.SSLContext | None) – The proxy SSL context is used to establish the TLS connection to the proxy when using HTTPS proxies.
use_forwarding_for_https (bool) – (Defaults to False) If set to True will forward requests to the HTTPS proxy to be made on behalf of the client instead of creating a TLS tunnel via the CONNECT method. Enabling this flag means that request and response headers and content will be visible from the HTTPS proxy whereas tunneling keeps request and response headers and content private. IP address, target hostname, SNI, and port are always visible to an HTTPS proxy even when this flag is disabled.
proxy_assert_hostname (None | str | Literal[False]) – The hostname of the certificate to verify against.
proxy_assert_fingerprint (str | None) – The fingerprint of the certificate to verify against.
num_pools (int) –
headers (Mapping[str, str] | None) –
connection_pool_kw (Any) –

Example:

import urllib3

proxy = urllib3.ProxyManager("https://localhost:3128/")

resp1 = proxy.request("GET", "https://google.com/")
resp2 = proxy.request("GET", "https://httpbin.org/")

print(len(proxy.pools))
# 1

resp3 = proxy.request("GET", "https://httpbin.org/")
resp4 = proxy.request("GET", "https://twitter.com/")

print(len(proxy.pools))
# 3

connection_from_host(host, port=None, scheme='http', pool_kwargs=None)#

Get a urllib3.connectionpool.ConnectionPool based on the host, port, and scheme.

Parameters

host (str | None) –
port (int | None) –
scheme (str | None) –
pool_kwargs (dict[str, Any] | None) –

Return type

HTTPConnectionPool

urlopen(method, url, redirect=True, **kw)#

Same as HTTP(S)ConnectionPool.urlopen, url must be absolute.

Parameters

method (str) –
url (str) –
redirect (bool) –
kw (Any) –

Return type

BaseHTTPResponse

class urllib3.poolmanager.PoolKey(key_scheme, key_host, key_port, key_timeout, key_retries, key_block, key_source_address, key_key_file, key_key_password, key_cert_file, key_cert_reqs, key_ca_certs, key_ssl_version, key_ssl_minimum_version, key_ssl_maximum_version, key_ca_cert_dir, key_ssl_context, key_maxsize, key_headers, key__proxy, key__proxy_headers, key__proxy_config, key_socket_options, key__socks_options, key_assert_hostname, key_assert_fingerprint, key_server_hostname, key_blocksize)#

Bases: NamedTuple

All known keyword arguments that could be provided to the pool manager, its pools, or the underlying connections.

All custom key schemes should include the fields in this key at a minimum.

Parameters

key_scheme (str) –
key_host (str) –
key_port (int | None) –
key_timeout (Timeout | float | int | None) –
key_retries (Retry | int | None) –
key_block (bool | None) –
key_source_address (tuple[str, int] | None) –
key_key_file (str | None) –
key_key_password (str | None) –
key_cert_file (str | None) –
key_cert_reqs (str | None) –
key_ca_certs (str | None) –
key_ssl_version (int | str | None) –
key_ssl_minimum_version (ssl.TLSVersion | None) –
key_ssl_maximum_version (ssl.TLSVersion | None) –
key_ca_cert_dir (str | None) –
key_ssl_context (ssl.SSLContext | None) –
key_maxsize (int | None) –
key_headers (frozenset[tuple[str, str]] | None) –
key__proxy (Url | None) –
key__proxy_headers (frozenset[tuple[str, str]] | None) –
key__proxy_config (ProxyConfig | None) –
key_socket_options (_TYPE_SOCKET_OPTIONS | None) –
key__socks_options (frozenset[tuple[str, str]] | None) –
key_assert_hostname (bool | str | None) –
key_assert_fingerprint (str | None) –
key_server_hostname (str | None) –
key_blocksize (int | None) –

key__proxy: Url | None#: Alias for field number 19

key__proxy_config: ProxyConfig | None#: Alias for field number 21

key__proxy_headers: frozenset[tuple[str, str]] | None#: Alias for field number 20

key__socks_options: frozenset[tuple[str, str]] | None#: Alias for field number 23

key_assert_fingerprint: str | None#: Alias for field number 25

key_assert_hostname: bool | str | None#: Alias for field number 24

key_block: bool | None#: Alias for field number 5

key_blocksize: int | None#: Alias for field number 27

key_ca_cert_dir: str | None#: Alias for field number 15

key_ca_certs: str | None#: Alias for field number 11

key_cert_file: str | None#: Alias for field number 9

key_cert_reqs: str | None#: Alias for field number 10

key_headers: frozenset[tuple[str, str]] | None#: Alias for field number 18

key_host: str#: Alias for field number 1

key_key_file: str | None#: Alias for field number 7

key_key_password: str | None#: Alias for field number 8

key_maxsize: int | None#: Alias for field number 17

key_port: int | None#: Alias for field number 2

key_retries: Retry | int | None#: Alias for field number 4

key_scheme: str#: Alias for field number 0

key_server_hostname: str | None#: Alias for field number 26

key_socket_options: _TYPE_SOCKET_OPTIONS | None#: Alias for field number 22

key_source_address: tuple[str, int] | None#: Alias for field number 6

key_ssl_context: ssl.SSLContext | None#: Alias for field number 16

key_ssl_maximum_version: ssl.TLSVersion | None#: Alias for field number 14

key_ssl_minimum_version: ssl.TLSVersion | None#: Alias for field number 13

key_ssl_version: int | str | None#: Alias for field number 12

key_timeout: Timeout | float | int | None#: Alias for field number 3