PoolManager

A pool manager is an abstraction for a collection of ConnectionPools.

If you need to make requests to multiple hosts, then you can use a PoolManager, which takes care of maintaining your pools so you don’t have to.

>>> from urllib3 import PoolManager
>>> manager = PoolManager(10)
>>> r = manager.request('GET', 'http://google.com/')
>>> r.headers['server']
'gws'
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> r.headers['server']
'YTS/1.20.0'
>>> r = manager.request('POST', 'http://google.com/mail')
>>> r = manager.request('HEAD', 'http://google.com/calendar')
>>> len(manager.pools)
2
>>> conn = manager.connection_from_host('google.com')
>>> conn.num_requests
3

The API of a PoolManager object is similar to that of a ConnectionPool, so they can be passed around interchangeably.

The PoolManager uses a Least Recently Used (LRU) policy for discarding old pools. That is, if you set the PoolManager num_pools to 10, then after making requests to 11 or more different hosts, the least recently used pools will be cleaned up eventually.

Cleanup of stale pools does not happen immediately. You can read more about the implementation and the various adjustable variables within RecentlyUsedContainer.

API

class urllib3.poolmanager.PoolManager(num_pools=10, **connection_pool_kw)

Allows for arbitrary requests while transparently keeping track of necessary connection pools for you.

Parameters:
  • num_pools – Number of connection pools to cache before discarding the least recently used pool.
  • **connection_pool_kw – Additional parameters are used to create fresh urllib3.connectionpool.ConnectionPool instances.

Example:

>>> manager = PoolManager()
>>> r = manager.urlopen("http://google.com/")
>>> r = manager.urlopen("http://google.com/mail")
>>> r = manager.urlopen("http://yahoo.com/")
>>> len(r.pools)
2
connection_from_host(host, port=80, scheme='http')

Get a ConnectionPool based on the host, port, and scheme.

Note that an appropriate port value is required here to normalize connection pools in our container most effectively.

connection_from_url(url)

Similar to urllib3.connectionpool.connection_from_url() but doesn’t pass any additional parameters to the urllib3.connectionpool.ConnectionPool constructor.

Additional parameters are taken from the PoolManager constructor.

get_url(url, fields=None, **urlopen_kw)

Deprecated since version 1.0.

Use request() instead.

post_url(url, fields=None, headers=None, **urlopen_kw)

Deprecated since version 1.0.

Use request() instead.

request(method, url, fields=None, headers=None, **urlopen_kw)

Make a request using urlopen() with the appropriate encoding of fields based on the method used.

This is a convenience method that requires the least amount of manual effort. It can be used in most situations, while still having the option to drop down to more specific methods when necessary, such as request_encode_url(), request_encode_body(), or even the lowest level urlopen().

request_encode_body(method, url, fields=None, headers=None, encode_multipart=True, multipart_boundary=None, **urlopen_kw)

Make a request using urlopen() with the fields encoded in the body. This is useful for request methods like POST, PUT, PATCH, etc.

When encode_multipart=True (default), then urllib3.filepost.encode_multipart_formdata() is used to encode the payload with the appropriate content type. Otherwise urllib.urlencode() is used with the ‘application/x-www-form-urlencoded’ content type.

Multipart encoding must be used when posting files, and it’s reasonably safe to use it in other times too. However, it may break request signing, such as with OAuth.

Supports an optional fields parameter of key/value strings AND key/filetuple. A filetuple is a (filename, data) tuple. For example:

fields = {
    'foo': 'bar',
    'fakefile': ('foofile.txt', 'contents of foofile'),
    'realfile': ('barfile.txt', open('realfile').read()),
    'nonamefile': ('contents of nonamefile field'),
}

When uploading a file, providing a filename (the first parameter of the tuple) is optional but recommended to best mimick behavior of browsers.

Note that if headers are supplied, the ‘Content-Type’ header will be overwritten because it depends on the dynamic random boundary string which is used to compose the body of the request. The random boundary string can be explicitly set with the multipart_boundary parameter.

request_encode_url(method, url, fields=None, **urlopen_kw)

Make a request using urlopen() with the fields encoded in the url. This is useful for request methods like GET, HEAD, DELETE, etc.

urlopen(method, url, **kw)

Same as urllib3.connectionpool.HTTPConnectionPool.urlopen().

url must be absolute, such that an appropriate urllib3.connectionpool.ConnectionPool can be chosen for it.

Table Of Contents

Previous topic

ConnectionPools

Next topic

Helpers

This Page