diff options
| author | Jon Wayne Parrott <jon.wayne.parrott@gmail.com> | 2016-06-28 10:43:15 -0700 |
|---|---|---|
| committer | Jon Wayne Parrott <jon.wayne.parrott@gmail.com> | 2016-06-28 10:43:15 -0700 |
| commit | ae7f533cb44838cceef79dd094018c6108d9ecc6 (patch) | |
| tree | bce93d708803f04a307ff4a9e1d881dd6720efd9 /docs/reference | |
| parent | a2bea61c8a71c7297c9020d72b11f781dab4332d (diff) | |
| download | urllib3-ae7f533cb44838cceef79dd094018c6108d9ecc6.tar.gz | |
Adding all reference docs
Diffstat (limited to 'docs/reference')
| -rw-r--r-- | docs/reference/collections.rst | 13 | ||||
| -rw-r--r-- | docs/reference/contrib.rst | 102 | ||||
| -rw-r--r-- | docs/reference/exceptions.rst | 9 | ||||
| -rw-r--r-- | docs/reference/helpers.rst | 64 | ||||
| -rw-r--r-- | docs/reference/index.rst | 88 | ||||
| -rw-r--r-- | docs/reference/managers.rst | 131 | ||||
| -rw-r--r-- | docs/reference/pools.rst | 86 | ||||
| -rw-r--r-- | docs/reference/urllib3.contrib.rst | 37 | ||||
| -rw-r--r-- | docs/reference/urllib3.util.rst | 70 |
9 files changed, 195 insertions, 405 deletions
diff --git a/docs/reference/collections.rst b/docs/reference/collections.rst deleted file mode 100644 index b3481402..00000000 --- a/docs/reference/collections.rst +++ /dev/null @@ -1,13 +0,0 @@ -Collections -=========== - -These datastructures are used to implement the behaviour of various urllib3 -components in a decoupled and application-agnostic design. - -.. automodule:: urllib3._collections - - .. autoclass:: RecentlyUsedContainer - :members: - - .. autoclass:: HTTPHeaderDict - :members: diff --git a/docs/reference/contrib.rst b/docs/reference/contrib.rst deleted file mode 100644 index c85d3aaf..00000000 --- a/docs/reference/contrib.rst +++ /dev/null @@ -1,102 +0,0 @@ -.. _contrib-modules: - -Contrib Modules -=============== - -These modules implement various extra features, that may not be ready for -prime time or that require optional third-party dependencies. - -.. _contrib-pyopenssl: - -SNI-support for Python 2 ------------------------- - -.. automodule:: urllib3.contrib.pyopenssl - - -.. _gae: - -Google App Engine ------------------ - -The :mod:`urllib3.contrib.appengine` module provides a pool manager that -uses Google App Engine's `URLFetch Service <https://cloud.google.com/appengine/docs/python/urlfetch>`_. - -Example usage:: - - from urllib3 import PoolManager - from urllib3.contrib.appengine import AppEngineManager, is_appengine_sandbox - - # This substitution will be done automagically once appengine code - # graduates from the contrib module. - if is_appengine_sandbox(): - # AppEngineManager uses AppEngine's URLFetch API behind the scenes - http = AppEngineManager() - else: - # PoolManager uses a socket-level API behind the scenes - http = PoolManager() - - # The client API should be consistent across managers, though some features are not available - # in URLFetch and you'll get warnings when you try to use them (like granular timeouts). - r = http.request('GET', 'https://google.com/') - - -There are `limitations <https://cloud.google.com/appengine/docs/python/urlfetch/#Python_Quotas_and_limits>`_ to the URLFetch service and it may not be the best choice for your application. App Engine provides three options for urllib3 users: - -1. You can use :class:`AppEngineManager` with URLFetch. URLFetch is cost-effective in many circumstances as long as your usage is within the limitations. -2. You can use a normal :class:`PoolManager` by enabling sockets. Sockets also have `limitations and restrictions <https://cloud.google.com/appengine/docs/python/sockets/#limitations-and-restrictions>`_ and have a lower free quota than URLFetch. To use sockets, be sure to specify the following in your ``app.yaml``:: - - env_variables: - GAE_USE_SOCKETS_HTTPLIB : 'true' - -3. If you are using `Managed VMs <https://cloud.google.com/appengine/docs/managed-vms/>`_, you can use the standard :class:`PoolManager` without any configuration or special environment variables. - - -.. _socks: - -SOCKS Proxies -------------- - -.. versionadded:: 1.14 - -The :mod:`urllib3.contrib.socks` module enables urllib3 to work with proxies -that use either the SOCKS4 or SOCKS5 protocols. These proxies are common in -environments that want to allow generic TCP/UDP traffic through their borders, -but don't want unrestricted traffic flows. - -To use it, either install ``PySocks`` or install urllib3 with the ``socks`` -extra, like so: - -.. code-block:: bash - - $ pip install -U urllib3[socks] - -The SOCKS module provides a -:class:`SOCKSProxyManager <urllib3.contrib.socks.SOCKSProxyManager>` that can -be used when SOCKS support is required. This class behaves very much like a -standard :class:`ProxyManager <urllib3.poolmanager.ProxyManager>`, but allows -the use of a SOCKS proxy instead. - -Using it is simple. For example, with a SOCKS5 proxy running on the local -machine, listening on port 8889: - -.. code-block:: python - - from urllib3.contrib.socks import SOCKSProxyManager - - http = SOCKSProxyManager('socks5://localhost:8889/') - r = http.request('GET', 'https://www.google.com/') - -The SOCKS implementation supports the full range of urllib3 features. It also -supports the following SOCKS features: - -- SOCKS4 -- SOCKS4a -- SOCKS5 -- Usernames and passwords for the SOCKS proxy - -The SOCKS module does have the following limitations: - -- No support for contacting a SOCKS proxy via IPv6. -- No support for reaching websites via a literal IPv6 address: domain names - must be used. diff --git a/docs/reference/exceptions.rst b/docs/reference/exceptions.rst deleted file mode 100644 index cd451be5..00000000 --- a/docs/reference/exceptions.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _exceptions: - -Exceptions -========== - -Custom exceptions defined by urllib3 - -.. automodule:: urllib3.exceptions - :members: diff --git a/docs/reference/helpers.rst b/docs/reference/helpers.rst deleted file mode 100644 index 6e8d036b..00000000 --- a/docs/reference/helpers.rst +++ /dev/null @@ -1,64 +0,0 @@ -.. _helpers: - -Helpers -======= - -Useful methods for working with :mod:`httplib`, completely decoupled from -code specific to **urllib3**. - -At the very core, just like its predecessors, :mod:`urllib3` is built on top of -:mod:`httplib` -- the lowest level HTTP library included in the Python -standard library. - -To aid the limited functionality of the :mod:`httplib` module, :mod:`urllib3` -provides various helper methods which are used with the higher level components -but can also be used independently. - -Timeouts --------- - -.. automodule:: urllib3.util.timeout - :members: - -Retries -------- - -.. automodule:: urllib3.util.retry - :members: - -URL Helpers ------------ - -.. automodule:: urllib3.util.url - :members: - -Filepost --------- - -.. automodule:: urllib3.filepost - :members: - -.. automodule:: urllib3.fields - :members: - -Request -------- - -.. automodule:: urllib3.request - :members: - -.. automodule:: urllib3.util.request - :members: - -Response --------- - -.. automodule:: urllib3.response - :members: - :undoc-members: - -SSL/TLS Helpers ---------------- - -.. automodule:: urllib3.util.ssl_ - :members: diff --git a/docs/reference/index.rst b/docs/reference/index.rst index 9a771c87..bc1b7120 100644 --- a/docs/reference/index.rst +++ b/docs/reference/index.rst @@ -1,2 +1,90 @@ Reference ========= + +.. contents:: + :local: + :backlinks: none + +Subpackages +----------- + +.. toctree:: + + urllib3.contrib + urllib3.util + +Submodules +---------- + +urllib3.connection module +------------------------- + +.. automodule:: urllib3.connection + :members: + :undoc-members: + :show-inheritance: + +urllib3.connectionpool module +----------------------------- + +.. automodule:: urllib3.connectionpool + :members: + :undoc-members: + :show-inheritance: + +urllib3.exceptions module +------------------------- + +.. automodule:: urllib3.exceptions + :members: + :undoc-members: + :show-inheritance: + +urllib3.fields module +--------------------- + +.. automodule:: urllib3.fields + :members: + :undoc-members: + :show-inheritance: + +urllib3.filepost module +----------------------- + +.. automodule:: urllib3.filepost + :members: + :undoc-members: + :show-inheritance: + +urllib3.poolmanager module +-------------------------- + +.. automodule:: urllib3.poolmanager + :members: + :undoc-members: + :show-inheritance: + +urllib3.request module +---------------------- + +.. automodule:: urllib3.request + :members: + :undoc-members: + :show-inheritance: + +urllib3.response module +----------------------- + +.. automodule:: urllib3.response + :members: + :undoc-members: + :show-inheritance: + + +Module contents +--------------- + +.. automodule:: urllib3 + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/reference/managers.rst b/docs/reference/managers.rst deleted file mode 100644 index e8e7e210..00000000 --- a/docs/reference/managers.rst +++ /dev/null @@ -1,131 +0,0 @@ -PoolManager -=========== - -.. automodule:: urllib3.poolmanager - -A pool manager is an abstraction for a collection of -:doc:`ConnectionPools <pools>`. - -If you need to make requests to multiple hosts, then you can use a -:class:`.PoolManager`, which takes care of maintaining your pools -so you don't have to. - -.. doctest :: - - >>> from urllib3 import PoolManager - >>> manager = PoolManager(10) - >>> r = manager.request('GET', 'http://example.com') - >>> r.headers['server'] - 'ECS (iad/182A)' - >>> r = manager.request('GET', 'http://httpbin.org/') - >>> r.headers['server'] - 'gunicorn/18.0' - >>> r = manager.request('POST', 'http://httpbin.org/headers') - >>> r = manager.request('HEAD', 'http://httpbin.org/cookies') - >>> len(manager.pools) - 2 - >>> conn = manager.connection_from_host('httpbin.org') - >>> conn.num_requests - 3 - -A :class:`.PoolManager` will create a new :doc:`ConnectionPool <pools>` -when no :doc:`ConnectionPools <pools>` exist with a matching pool key. -The pool key is derived using the requested URL and the current values -of the ``connection_pool_kw`` instance variable on :class:`.PoolManager`. - -The keys in ``connection_pool_kw`` used when deriving the key are -configurable. For example, by default the ``my_field`` key is not -considered. - -.. doctest :: - - >>> from urllib3.poolmanager import PoolManager - >>> manager = PoolManager(10, my_field='wheat') - >>> manager.connection_from_url('http://example.com') - >>> manager.connection_pool_kw['my_field'] = 'barley' - >>> manager.connection_from_url('http://example.com') - >>> len(manager.pools) - 1 - -To make the pool manager create new pools when the value of -``my_field`` changes, you can define a custom pool key and alter -the ``key_fn_by_scheme`` instance variable on :class:`.PoolManager`. - -.. doctest :: - - >>> import functools - >>> from collections import namedtuple - >>> from urllib3.poolmanager import PoolManager, HTTPPoolKey - >>> from urllib3.poolmanager import default_key_normalizer as normalizer - >>> CustomKey = namedtuple('CustomKey', HTTPPoolKey._fields + ('my_field',)) - >>> manager = PoolManager(10, my_field='wheat') - >>> manager.key_fn_by_scheme['http'] = functools.partial(normalizer, CustomKey) - >>> manager.connection_from_url('http://example.com') - >>> manager.connection_pool_kw['my_field'] = 'barley' - >>> manager.connection_from_url('http://example.com') - >>> len(manager.pools) - 2 - -The API of a :class:`.PoolManager` object is similar to that of a -:doc:`ConnectionPool <pools>`, so they can be passed around interchangeably. - -The PoolManager uses a Least Recently Used (LRU) policy for discarding old -pools. That is, if you set the PoolManager ``num_pools`` to 10, then after -making requests to 11 or more different hosts, the least recently used pools -will be cleaned up eventually. - -Cleanup of stale pools does not happen immediately but can be forced when used -as a context manager. - -.. doctest :: - - >>> from urllib3 import PoolManager - >>> with PoolManager(10) as manager: - ... r = manager.request('GET', 'http://example.com') - ... r = manager.request('GET', 'http://httpbin.org/') - ... len(manager.pools) - ... - 2 - >>> len(manager.pools) - 0 - -You can read more about the implementation and the various adjustable variables -within :class:`~urllib3._collections.RecentlyUsedContainer`. - -API ---- - - .. autoclass:: PoolManager - :inherited-members: - .. autoclass:: BasePoolKey - :inherited-members: - .. autoclass:: HTTPPoolKey - :inherited-members: - .. autoclass:: HTTPSPoolKey - :inherited-members: - -ProxyManager -============ - -:class:`.ProxyManager` is an HTTP proxy-aware subclass of :class:`.PoolManager`. -It produces a single -:class:`~urllib3.connectionpool.HTTPConnectionPool` instance for all HTTP -connections and individual per-server:port -:class:`~urllib3.connectionpool.HTTPSConnectionPool` instances for tunnelled -HTTPS connections. - -Example using proxy authentication: - -:: - - >>> headers = urllib3.make_headers(proxy_basic_auth='myusername:mypassword') - >>> proxy = urllib3.ProxyManager('http://localhost:3128', proxy_headers=headers) - >>> r = proxy.request('GET', 'http://example.com/') - >>> r.status - 200 - - -API ---- - .. autoclass:: ProxyManager - diff --git a/docs/reference/pools.rst b/docs/reference/pools.rst deleted file mode 100644 index 9cc2be9a..00000000 --- a/docs/reference/pools.rst +++ /dev/null @@ -1,86 +0,0 @@ -ConnectionPools -=============== - -.. automodule:: urllib3.connectionpool - -A connection pool is a container for a collection of connections to a specific -host. - -If you need to make requests to the same host repeatedly, then you should use a -:class:`.HTTPConnectionPool`. - -.. doctest :: - - >>> from urllib3 import HTTPConnectionPool - >>> pool = HTTPConnectionPool('ajax.googleapis.com', maxsize=1) - >>> r = pool.request('GET', '/ajax/services/search/web', - ... fields={'q': 'urllib3', 'v': '1.0'}) - >>> r.status - 200 - >>> r.headers['content-type'] - 'text/javascript; charset=utf-8' - >>> 'data: ' + r.data # Content of the response - 'data: ...' - >>> r = pool.request('GET', '/ajax/services/search/web', - ... fields={'q': 'python', 'v': '1.0'}) - >>> 'data: ' + r.data # Content of the response - 'data: ...' - >>> pool.num_connections - 1 - >>> pool.num_requests - 2 - -By default, the pool will cache just one connection. If you're planning on using -such a pool in a multithreaded environment, you should set the ``maxsize`` of -the pool to a higher number, such as the number of threads. You can also control -many other variables like timeout, blocking, and default headers. - -A ConnectionPool can be used as a context manager to automatically clear the -pool after usage. - -.. doctest :: - - >>> from urllib3 import HTTPConnectionPool - >>> with HTTPConnectionPool('ajax.googleapis.com', maxsize=1) as pool: - ... r = pool.request('GET', '/ajax/services/search/web', - ... fields={'q': 'urllib3', 'v': '1.0'}) - ... print(pool.pool) - ... - <queue.LifoQueue object at 0x7f67367dfcf8> - >>> print(pool.pool) - None - -Helpers -------- - -There are various helper functions provided for instantiating these -ConnectionPools more easily: - - .. autofunction:: connection_from_url - -API ---- - -:mod:`urllib3.connectionpool` comes with two connection pools: - - .. autoclass:: HTTPConnectionPool - :members: - :inherited-members: - - .. autoclass:: HTTPSConnectionPool - - -All of these pools inherit from a common base class: - - .. autoclass:: ConnectionPool - -.. module:: urllib3.connection - -Related Classes ---------------- - -urllib3 implements its own :class:`HTTPConnection` object to allow for more -flexibility than the standard library's implementation. - -.. autoclass:: HTTPConnection - :members: diff --git a/docs/reference/urllib3.contrib.rst b/docs/reference/urllib3.contrib.rst new file mode 100644 index 00000000..af6f9461 --- /dev/null +++ b/docs/reference/urllib3.contrib.rst @@ -0,0 +1,37 @@ +urllib3.contrib package +======================= + +These modules implement various extra features, that may not be ready for +prime time or that require optional third-party dependencies. + +urllib3.contrib.appengine module +-------------------------------- + +.. automodule:: urllib3.contrib.appengine + :members: + :undoc-members: + :show-inheritance: + +urllib3.contrib.ntlmpool module +------------------------------- + +.. automodule:: urllib3.contrib.ntlmpool + :members: + :undoc-members: + :show-inheritance: + +urllib3.contrib.pyopenssl module +-------------------------------- + +.. automodule:: urllib3.contrib.pyopenssl + :members: + :undoc-members: + :show-inheritance: + +urllib3.contrib.socks module +---------------------------- + +.. automodule:: urllib3.contrib.socks + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/reference/urllib3.util.rst b/docs/reference/urllib3.util.rst new file mode 100644 index 00000000..f8b20a98 --- /dev/null +++ b/docs/reference/urllib3.util.rst @@ -0,0 +1,70 @@ +urllib3.util package +==================== + +Useful methods for working with :mod:`httplib`, completely decoupled from +code specific to **urllib3**. + +At the very core, just like its predecessors, :mod:`urllib3` is built on top of +:mod:`httplib` -- the lowest level HTTP library included in the Python +standard library. + +To aid the limited functionality of the :mod:`httplib` module, :mod:`urllib3` +provides various helper methods which are used with the higher level components +but can also be used independently. + +urllib3.util.connection module +------------------------------ + +.. automodule:: urllib3.util.connection + :members: + :undoc-members: + :show-inheritance: + +urllib3.util.request module +--------------------------- + +.. automodule:: urllib3.util.request + :members: + :undoc-members: + :show-inheritance: + +urllib3.util.response module +---------------------------- + +.. automodule:: urllib3.util.response + :members: + :undoc-members: + :show-inheritance: + +urllib3.util.retry module +------------------------- + +.. automodule:: urllib3.util.retry + :members: + :undoc-members: + :show-inheritance: + +urllib3.util.timeout module +--------------------------- + +.. automodule:: urllib3.util.timeout + :members: + :undoc-members: + :show-inheritance: + +urllib3.util.url module +----------------------- + +.. automodule:: urllib3.util.url + :members: + :undoc-members: + :show-inheritance: + + +Module contents +--------------- + +.. automodule:: urllib3.util + :members: + :undoc-members: + :show-inheritance: |
