summaryrefslogtreecommitdiff
path: root/docs/reference
diff options
context:
space:
mode:
authorJon Wayne Parrott <jon.wayne.parrott@gmail.com>2016-06-28 10:43:15 -0700
committerJon Wayne Parrott <jon.wayne.parrott@gmail.com>2016-06-28 10:43:15 -0700
commitae7f533cb44838cceef79dd094018c6108d9ecc6 (patch)
treebce93d708803f04a307ff4a9e1d881dd6720efd9 /docs/reference
parenta2bea61c8a71c7297c9020d72b11f781dab4332d (diff)
downloadurllib3-ae7f533cb44838cceef79dd094018c6108d9ecc6.tar.gz
Adding all reference docs
Diffstat (limited to 'docs/reference')
-rw-r--r--docs/reference/collections.rst13
-rw-r--r--docs/reference/contrib.rst102
-rw-r--r--docs/reference/exceptions.rst9
-rw-r--r--docs/reference/helpers.rst64
-rw-r--r--docs/reference/index.rst88
-rw-r--r--docs/reference/managers.rst131
-rw-r--r--docs/reference/pools.rst86
-rw-r--r--docs/reference/urllib3.contrib.rst37
-rw-r--r--docs/reference/urllib3.util.rst70
9 files changed, 195 insertions, 405 deletions
diff --git a/docs/reference/collections.rst b/docs/reference/collections.rst
deleted file mode 100644
index b3481402..00000000
--- a/docs/reference/collections.rst
+++ /dev/null
@@ -1,13 +0,0 @@
-Collections
-===========
-
-These datastructures are used to implement the behaviour of various urllib3
-components in a decoupled and application-agnostic design.
-
-.. automodule:: urllib3._collections
-
- .. autoclass:: RecentlyUsedContainer
- :members:
-
- .. autoclass:: HTTPHeaderDict
- :members:
diff --git a/docs/reference/contrib.rst b/docs/reference/contrib.rst
deleted file mode 100644
index c85d3aaf..00000000
--- a/docs/reference/contrib.rst
+++ /dev/null
@@ -1,102 +0,0 @@
-.. _contrib-modules:
-
-Contrib Modules
-===============
-
-These modules implement various extra features, that may not be ready for
-prime time or that require optional third-party dependencies.
-
-.. _contrib-pyopenssl:
-
-SNI-support for Python 2
-------------------------
-
-.. automodule:: urllib3.contrib.pyopenssl
-
-
-.. _gae:
-
-Google App Engine
------------------
-
-The :mod:`urllib3.contrib.appengine` module provides a pool manager that
-uses Google App Engine's `URLFetch Service <https://cloud.google.com/appengine/docs/python/urlfetch>`_.
-
-Example usage::
-
- from urllib3 import PoolManager
- from urllib3.contrib.appengine import AppEngineManager, is_appengine_sandbox
-
- # This substitution will be done automagically once appengine code
- # graduates from the contrib module.
- if is_appengine_sandbox():
- # AppEngineManager uses AppEngine's URLFetch API behind the scenes
- http = AppEngineManager()
- else:
- # PoolManager uses a socket-level API behind the scenes
- http = PoolManager()
-
- # The client API should be consistent across managers, though some features are not available
- # in URLFetch and you'll get warnings when you try to use them (like granular timeouts).
- r = http.request('GET', 'https://google.com/')
-
-
-There are `limitations <https://cloud.google.com/appengine/docs/python/urlfetch/#Python_Quotas_and_limits>`_ to the URLFetch service and it may not be the best choice for your application. App Engine provides three options for urllib3 users:
-
-1. You can use :class:`AppEngineManager` with URLFetch. URLFetch is cost-effective in many circumstances as long as your usage is within the limitations.
-2. You can use a normal :class:`PoolManager` by enabling sockets. Sockets also have `limitations and restrictions <https://cloud.google.com/appengine/docs/python/sockets/#limitations-and-restrictions>`_ and have a lower free quota than URLFetch. To use sockets, be sure to specify the following in your ``app.yaml``::
-
- env_variables:
- GAE_USE_SOCKETS_HTTPLIB : 'true'
-
-3. If you are using `Managed VMs <https://cloud.google.com/appengine/docs/managed-vms/>`_, you can use the standard :class:`PoolManager` without any configuration or special environment variables.
-
-
-.. _socks:
-
-SOCKS Proxies
--------------
-
-.. versionadded:: 1.14
-
-The :mod:`urllib3.contrib.socks` module enables urllib3 to work with proxies
-that use either the SOCKS4 or SOCKS5 protocols. These proxies are common in
-environments that want to allow generic TCP/UDP traffic through their borders,
-but don't want unrestricted traffic flows.
-
-To use it, either install ``PySocks`` or install urllib3 with the ``socks``
-extra, like so:
-
-.. code-block:: bash
-
- $ pip install -U urllib3[socks]
-
-The SOCKS module provides a
-:class:`SOCKSProxyManager <urllib3.contrib.socks.SOCKSProxyManager>` that can
-be used when SOCKS support is required. This class behaves very much like a
-standard :class:`ProxyManager <urllib3.poolmanager.ProxyManager>`, but allows
-the use of a SOCKS proxy instead.
-
-Using it is simple. For example, with a SOCKS5 proxy running on the local
-machine, listening on port 8889:
-
-.. code-block:: python
-
- from urllib3.contrib.socks import SOCKSProxyManager
-
- http = SOCKSProxyManager('socks5://localhost:8889/')
- r = http.request('GET', 'https://www.google.com/')
-
-The SOCKS implementation supports the full range of urllib3 features. It also
-supports the following SOCKS features:
-
-- SOCKS4
-- SOCKS4a
-- SOCKS5
-- Usernames and passwords for the SOCKS proxy
-
-The SOCKS module does have the following limitations:
-
-- No support for contacting a SOCKS proxy via IPv6.
-- No support for reaching websites via a literal IPv6 address: domain names
- must be used.
diff --git a/docs/reference/exceptions.rst b/docs/reference/exceptions.rst
deleted file mode 100644
index cd451be5..00000000
--- a/docs/reference/exceptions.rst
+++ /dev/null
@@ -1,9 +0,0 @@
-.. _exceptions:
-
-Exceptions
-==========
-
-Custom exceptions defined by urllib3
-
-.. automodule:: urllib3.exceptions
- :members:
diff --git a/docs/reference/helpers.rst b/docs/reference/helpers.rst
deleted file mode 100644
index 6e8d036b..00000000
--- a/docs/reference/helpers.rst
+++ /dev/null
@@ -1,64 +0,0 @@
-.. _helpers:
-
-Helpers
-=======
-
-Useful methods for working with :mod:`httplib`, completely decoupled from
-code specific to **urllib3**.
-
-At the very core, just like its predecessors, :mod:`urllib3` is built on top of
-:mod:`httplib` -- the lowest level HTTP library included in the Python
-standard library.
-
-To aid the limited functionality of the :mod:`httplib` module, :mod:`urllib3`
-provides various helper methods which are used with the higher level components
-but can also be used independently.
-
-Timeouts
---------
-
-.. automodule:: urllib3.util.timeout
- :members:
-
-Retries
--------
-
-.. automodule:: urllib3.util.retry
- :members:
-
-URL Helpers
------------
-
-.. automodule:: urllib3.util.url
- :members:
-
-Filepost
---------
-
-.. automodule:: urllib3.filepost
- :members:
-
-.. automodule:: urllib3.fields
- :members:
-
-Request
--------
-
-.. automodule:: urllib3.request
- :members:
-
-.. automodule:: urllib3.util.request
- :members:
-
-Response
---------
-
-.. automodule:: urllib3.response
- :members:
- :undoc-members:
-
-SSL/TLS Helpers
----------------
-
-.. automodule:: urllib3.util.ssl_
- :members:
diff --git a/docs/reference/index.rst b/docs/reference/index.rst
index 9a771c87..bc1b7120 100644
--- a/docs/reference/index.rst
+++ b/docs/reference/index.rst
@@ -1,2 +1,90 @@
Reference
=========
+
+.. contents::
+ :local:
+ :backlinks: none
+
+Subpackages
+-----------
+
+.. toctree::
+
+ urllib3.contrib
+ urllib3.util
+
+Submodules
+----------
+
+urllib3.connection module
+-------------------------
+
+.. automodule:: urllib3.connection
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.connectionpool module
+-----------------------------
+
+.. automodule:: urllib3.connectionpool
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.exceptions module
+-------------------------
+
+.. automodule:: urllib3.exceptions
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.fields module
+---------------------
+
+.. automodule:: urllib3.fields
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.filepost module
+-----------------------
+
+.. automodule:: urllib3.filepost
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.poolmanager module
+--------------------------
+
+.. automodule:: urllib3.poolmanager
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.request module
+----------------------
+
+.. automodule:: urllib3.request
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.response module
+-----------------------
+
+.. automodule:: urllib3.response
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+
+Module contents
+---------------
+
+.. automodule:: urllib3
+ :members:
+ :undoc-members:
+ :show-inheritance:
diff --git a/docs/reference/managers.rst b/docs/reference/managers.rst
deleted file mode 100644
index e8e7e210..00000000
--- a/docs/reference/managers.rst
+++ /dev/null
@@ -1,131 +0,0 @@
-PoolManager
-===========
-
-.. automodule:: urllib3.poolmanager
-
-A pool manager is an abstraction for a collection of
-:doc:`ConnectionPools <pools>`.
-
-If you need to make requests to multiple hosts, then you can use a
-:class:`.PoolManager`, which takes care of maintaining your pools
-so you don't have to.
-
-.. doctest ::
-
- >>> from urllib3 import PoolManager
- >>> manager = PoolManager(10)
- >>> r = manager.request('GET', 'http://example.com')
- >>> r.headers['server']
- 'ECS (iad/182A)'
- >>> r = manager.request('GET', 'http://httpbin.org/')
- >>> r.headers['server']
- 'gunicorn/18.0'
- >>> r = manager.request('POST', 'http://httpbin.org/headers')
- >>> r = manager.request('HEAD', 'http://httpbin.org/cookies')
- >>> len(manager.pools)
- 2
- >>> conn = manager.connection_from_host('httpbin.org')
- >>> conn.num_requests
- 3
-
-A :class:`.PoolManager` will create a new :doc:`ConnectionPool <pools>`
-when no :doc:`ConnectionPools <pools>` exist with a matching pool key.
-The pool key is derived using the requested URL and the current values
-of the ``connection_pool_kw`` instance variable on :class:`.PoolManager`.
-
-The keys in ``connection_pool_kw`` used when deriving the key are
-configurable. For example, by default the ``my_field`` key is not
-considered.
-
-.. doctest ::
-
- >>> from urllib3.poolmanager import PoolManager
- >>> manager = PoolManager(10, my_field='wheat')
- >>> manager.connection_from_url('http://example.com')
- >>> manager.connection_pool_kw['my_field'] = 'barley'
- >>> manager.connection_from_url('http://example.com')
- >>> len(manager.pools)
- 1
-
-To make the pool manager create new pools when the value of
-``my_field`` changes, you can define a custom pool key and alter
-the ``key_fn_by_scheme`` instance variable on :class:`.PoolManager`.
-
-.. doctest ::
-
- >>> import functools
- >>> from collections import namedtuple
- >>> from urllib3.poolmanager import PoolManager, HTTPPoolKey
- >>> from urllib3.poolmanager import default_key_normalizer as normalizer
- >>> CustomKey = namedtuple('CustomKey', HTTPPoolKey._fields + ('my_field',))
- >>> manager = PoolManager(10, my_field='wheat')
- >>> manager.key_fn_by_scheme['http'] = functools.partial(normalizer, CustomKey)
- >>> manager.connection_from_url('http://example.com')
- >>> manager.connection_pool_kw['my_field'] = 'barley'
- >>> manager.connection_from_url('http://example.com')
- >>> len(manager.pools)
- 2
-
-The API of a :class:`.PoolManager` object is similar to that of a
-:doc:`ConnectionPool <pools>`, so they can be passed around interchangeably.
-
-The PoolManager uses a Least Recently Used (LRU) policy for discarding old
-pools. That is, if you set the PoolManager ``num_pools`` to 10, then after
-making requests to 11 or more different hosts, the least recently used pools
-will be cleaned up eventually.
-
-Cleanup of stale pools does not happen immediately but can be forced when used
-as a context manager.
-
-.. doctest ::
-
- >>> from urllib3 import PoolManager
- >>> with PoolManager(10) as manager:
- ... r = manager.request('GET', 'http://example.com')
- ... r = manager.request('GET', 'http://httpbin.org/')
- ... len(manager.pools)
- ...
- 2
- >>> len(manager.pools)
- 0
-
-You can read more about the implementation and the various adjustable variables
-within :class:`~urllib3._collections.RecentlyUsedContainer`.
-
-API
----
-
- .. autoclass:: PoolManager
- :inherited-members:
- .. autoclass:: BasePoolKey
- :inherited-members:
- .. autoclass:: HTTPPoolKey
- :inherited-members:
- .. autoclass:: HTTPSPoolKey
- :inherited-members:
-
-ProxyManager
-============
-
-:class:`.ProxyManager` is an HTTP proxy-aware subclass of :class:`.PoolManager`.
-It produces a single
-:class:`~urllib3.connectionpool.HTTPConnectionPool` instance for all HTTP
-connections and individual per-server:port
-:class:`~urllib3.connectionpool.HTTPSConnectionPool` instances for tunnelled
-HTTPS connections.
-
-Example using proxy authentication:
-
-::
-
- >>> headers = urllib3.make_headers(proxy_basic_auth='myusername:mypassword')
- >>> proxy = urllib3.ProxyManager('http://localhost:3128', proxy_headers=headers)
- >>> r = proxy.request('GET', 'http://example.com/')
- >>> r.status
- 200
-
-
-API
----
- .. autoclass:: ProxyManager
-
diff --git a/docs/reference/pools.rst b/docs/reference/pools.rst
deleted file mode 100644
index 9cc2be9a..00000000
--- a/docs/reference/pools.rst
+++ /dev/null
@@ -1,86 +0,0 @@
-ConnectionPools
-===============
-
-.. automodule:: urllib3.connectionpool
-
-A connection pool is a container for a collection of connections to a specific
-host.
-
-If you need to make requests to the same host repeatedly, then you should use a
-:class:`.HTTPConnectionPool`.
-
-.. doctest ::
-
- >>> from urllib3 import HTTPConnectionPool
- >>> pool = HTTPConnectionPool('ajax.googleapis.com', maxsize=1)
- >>> r = pool.request('GET', '/ajax/services/search/web',
- ... fields={'q': 'urllib3', 'v': '1.0'})
- >>> r.status
- 200
- >>> r.headers['content-type']
- 'text/javascript; charset=utf-8'
- >>> 'data: ' + r.data # Content of the response
- 'data: ...'
- >>> r = pool.request('GET', '/ajax/services/search/web',
- ... fields={'q': 'python', 'v': '1.0'})
- >>> 'data: ' + r.data # Content of the response
- 'data: ...'
- >>> pool.num_connections
- 1
- >>> pool.num_requests
- 2
-
-By default, the pool will cache just one connection. If you're planning on using
-such a pool in a multithreaded environment, you should set the ``maxsize`` of
-the pool to a higher number, such as the number of threads. You can also control
-many other variables like timeout, blocking, and default headers.
-
-A ConnectionPool can be used as a context manager to automatically clear the
-pool after usage.
-
-.. doctest ::
-
- >>> from urllib3 import HTTPConnectionPool
- >>> with HTTPConnectionPool('ajax.googleapis.com', maxsize=1) as pool:
- ... r = pool.request('GET', '/ajax/services/search/web',
- ... fields={'q': 'urllib3', 'v': '1.0'})
- ... print(pool.pool)
- ...
- <queue.LifoQueue object at 0x7f67367dfcf8>
- >>> print(pool.pool)
- None
-
-Helpers
--------
-
-There are various helper functions provided for instantiating these
-ConnectionPools more easily:
-
- .. autofunction:: connection_from_url
-
-API
----
-
-:mod:`urllib3.connectionpool` comes with two connection pools:
-
- .. autoclass:: HTTPConnectionPool
- :members:
- :inherited-members:
-
- .. autoclass:: HTTPSConnectionPool
-
-
-All of these pools inherit from a common base class:
-
- .. autoclass:: ConnectionPool
-
-.. module:: urllib3.connection
-
-Related Classes
----------------
-
-urllib3 implements its own :class:`HTTPConnection` object to allow for more
-flexibility than the standard library's implementation.
-
-.. autoclass:: HTTPConnection
- :members:
diff --git a/docs/reference/urllib3.contrib.rst b/docs/reference/urllib3.contrib.rst
new file mode 100644
index 00000000..af6f9461
--- /dev/null
+++ b/docs/reference/urllib3.contrib.rst
@@ -0,0 +1,37 @@
+urllib3.contrib package
+=======================
+
+These modules implement various extra features, that may not be ready for
+prime time or that require optional third-party dependencies.
+
+urllib3.contrib.appengine module
+--------------------------------
+
+.. automodule:: urllib3.contrib.appengine
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.contrib.ntlmpool module
+-------------------------------
+
+.. automodule:: urllib3.contrib.ntlmpool
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.contrib.pyopenssl module
+--------------------------------
+
+.. automodule:: urllib3.contrib.pyopenssl
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.contrib.socks module
+----------------------------
+
+.. automodule:: urllib3.contrib.socks
+ :members:
+ :undoc-members:
+ :show-inheritance:
diff --git a/docs/reference/urllib3.util.rst b/docs/reference/urllib3.util.rst
new file mode 100644
index 00000000..f8b20a98
--- /dev/null
+++ b/docs/reference/urllib3.util.rst
@@ -0,0 +1,70 @@
+urllib3.util package
+====================
+
+Useful methods for working with :mod:`httplib`, completely decoupled from
+code specific to **urllib3**.
+
+At the very core, just like its predecessors, :mod:`urllib3` is built on top of
+:mod:`httplib` -- the lowest level HTTP library included in the Python
+standard library.
+
+To aid the limited functionality of the :mod:`httplib` module, :mod:`urllib3`
+provides various helper methods which are used with the higher level components
+but can also be used independently.
+
+urllib3.util.connection module
+------------------------------
+
+.. automodule:: urllib3.util.connection
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.util.request module
+---------------------------
+
+.. automodule:: urllib3.util.request
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.util.response module
+----------------------------
+
+.. automodule:: urllib3.util.response
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.util.retry module
+-------------------------
+
+.. automodule:: urllib3.util.retry
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.util.timeout module
+---------------------------
+
+.. automodule:: urllib3.util.timeout
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+urllib3.util.url module
+-----------------------
+
+.. automodule:: urllib3.util.url
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+
+Module contents
+---------------
+
+.. automodule:: urllib3.util
+ :members:
+ :undoc-members:
+ :show-inheritance: