path: root/pkg_resources.txt

=============================================================
Package Discovery and Resource Access using ``pkg_resources``
=============================================================

The ``pkg_resources`` module distributed with ``setuptools`` provides an API
for Python libraries to access their resource files, and for extensible
applications and frameworks to automatically discover plugins.  It also
provides runtime support for using C extensions that are inside zipfile-format
eggs, support for merging packages that have separately-distributed modules or
subpackages, and APIs for managing Python's current "working set" of active
packages.


.. contents:: **Table of Contents**


--------
Overview
--------

XXX


-----------------
Developer's Guide
-----------------

    Accessing Resources
    Finding and Activating Package Distributions
        get_provider()
        require()
        WorkingSet
        iter_distributions
    Running Scripts
    Configuration
    Namespace Packages
    Extensible Applications and Frameworks
        Locating entry points
        Activation listeners
        Metadata access
        Extended Discovery and Installation
    Supporting Custom PEP 302 Implementations


-------------
API Reference
-------------

    'require', 'run_script',

Namespace Package Support
=========================

declare_namespace, fixup_namespace_packages, register_namespace_handler


``WorkingSet`` Objects
======================

Listeners


``Environment`` Objects
=======================

XXX


``Requirement`` Objects
=======================

XXX Syntax, parse_requirments, Requirement.parse, etc.




Entry Points
============

Entry points are a simple way for distributions to "advertise" Python objects
(such as functions or classes) for use by other distributions.  Extensible
applications and frameworks can search for entry points with a particular name
or group, either from a specific distribution or from all active distributions
on sys.path, and then inspect or load the advertised objects at will.

Entry points belong to "groups" which are named with a dotted name similar to
a Python package or module name.  For example, the ``setuptools`` package uses
an entry point named ``distutils.commands`` in order to find commands defined
by distutils extensions.  ``setuptools`` treats the names of entry points
defined in that group as the acceptable commands for a setup script.

In a similar way, other packages can define their own entry point groups,
either using dynamic names within the group (like ``distutils.commands``), or
possibly using predefined names within the group.  For example, a blogging
framework that offers various pre- or post-publishing hooks might define an
entry point group and look for entry points named "pre_process" and
"post_process" within that group.

To advertise an entry point, a project needs to use ``setuptools`` and provide
an ``entry_points`` argument to ``setup()`` in its setup script, so that the
entry points will be included in the distribution's metadata.  For more
details, see the ``setuptools`` documentation.  (XXX link here to setuptools)

Each project distribution can advertise at most one entry point of a given
name within the same entry point group.  For example, a distutils extension
could advertise two different ``distutils.commands`` entry points, as long as
they had different names.  However, there is nothing that prevents *different*
projects from advertising entry points of the same name in the same group.  In
some cases, this is a desirable thing, since the application or framework that
uses the entry points may be calling them as hooks, or in some other way
combining them.  It is up to the application or framework to decide what to do
if multiple distributions advertise an entry point; some possibilities include
using both entry points, displaying an error message, using the first one found
in sys.path order, etc.


Convenience API
---------------

In the following functions, the `dist` argument can be a ``Distribution``
instance, a ``Requirement`` instance, or a string specifying a requirement
(i.e. project name, version, etc.).  If the argument is a string or
``Requirement``, the specified distribution is located (and added to sys.path
if not already present).  An error will be raised if a matching distribution is
not available.

The `group` argument should be a string containing a dotted identifier,
identifying an entry point group.  If you are defining an entry point group,
you should include some portion of your package's name in the group name so as
to avoid collision with other packages' entry point groups.

``load_entry_point(dist, group, name)``
    Load the named entry point from the specified distribution, or raise
    ``ImportError``.

``get_entry_info(dist, group, name)``
    Return an ``EntryPoint`` object for the given `group` and `name` from
    the specified distribution.  Returns ``None`` if the distribution has not
    advertised a matching entry point.

``get_entry_map(dist, group=None)
    Return the distribution's entry point map for `group`, or the full entry
    map for the distribution.  This function always returns a dictionary,
    even if the distribution advertises no entry points.  If `group` is given,
    the dictionary maps entry point names to the corresponding ``EntryPoint``
    object.  If `group` is None, the dictionary maps group names to
    dictionaries that then map entry point names to the corresponding
    ``EntryPoint`` instance in that group.

``iter_entry_points(group, name=None)``
    Yield entry point objects from `group` matching `name`

    If `name` is None, yields all entry points in `group` from all
    distributions in the working set on sys.path, otherwise only ones matching
    both `group` and `name` are yielded.  Entry points are yielded from
    the active distributions in the order that the distributions appear on
    sys.path.  (Within entry points for a particular distribution, however,
    there is no particular ordering.)


Creating and Parsing
--------------------

``EntryPoint(name, module_name, attrs=(), extras=(), dist=None)``
    Create an ``EntryPoint`` instance.  `name` is the entry point name.  The
    `module_name` is the (dotted) name of the module containing the advertised
    object.  `attrs` is an optional tuple of names to look up from the
    module to obtain the advertised object.  For example, an `attrs` of
    ``("foo","bar")`` and a `module_name` of ``"baz"`` would mean that the
    advertised object could be obtained by the following code::

        import baz
        advertised_object = baz.foo.bar
    
    The `extras` are an optional tuple of "extra feature" names that the
    distribution needs in order to provide this entry point.  When the
    entry point is loaded, these extra features are looked up in the `dist`
    argument to find out what other distributions may need to be activated
    on sys.path; see the ``load()`` method for more details.  The `extras`
    argument is only meaningful if `dist` is specified.  `dist` must be
    a ``Distribution`` instance.
    
``EntryPoint.parse(src, dist=None)`` (classmethod)
    Parse a single entry point from string `src`

    Entry point syntax follows the form::

        name = some.module:some.attr [extra1,extra2]

    The entry name and module name are required, but the ``:attrs`` and
    ``[extras]`` parts are optional, as is the whitespace shown between
    some of the items.  The `dist` argument is passed through to the
    ``EntryPoint()`` constructor, along with the other values parsed from
    `src`.

``EntryPoint.parse_group(group, lines, dist=None)`` (classmethod)
    Parse `lines` (a string or sequence of lines) to create a dictionary
    mapping entry point names to ``EntryPoint`` objects.  ``ValueError`` is
    raised if entry point names are duplicated, if `group` is not a valid
    entry point group name, or if there are any syntax errors.  (Note: the
    `group` parameter is used only for validation and to create more
    informative error messages.)  If `dist` is provided, it will be used to
    set the ``dist`` attribute of the created ``EntryPoint`` objects.

``EntryPoint.parse_map(data, dist=None)`` (classmethod)
    Parse `data` into a dictionary mapping group names to dictionaries mapping
    entry point names to ``EntryPoint`` objects.  If `data` is a dictionary,
    then the keys are used as group names and the values are passed to
    ``parse_group()`` as the `lines` argument.  If `data` is a string or
    sequence of lines, it is first split into .ini-style sections (using
    the ``split_sections()`` utility function) and the section names are used
    as group names.  In either case, the `dist` argument is passed through to
    ``parse_group()`` so that the entry points will be linked to the specified
    distribution.


``EntryPoint`` Objects
----------------------

For simple introspection, ``EntryPoint`` objects have attributes that
correspond exactly to the constructor argument names: ``name``,
``module_name``, ``attrs``, ``extras``, and ``dist`` are all available.  In
addition, the following methods are provided:

``load(require=True, env=None, installer=None)``
    Load the entry point, returning the advertised Python object, or raise
    ``ImportError`` if it cannot be obtained.  If `require` is a true value,
    then ``require(env, installer)`` is called before attempting the import.

``require(env=None, installer=None)``
    Ensure that any "extras" needed by the entry point are available on
    sys.path.  ``UnknownExtra`` is raised if the ``EntryPoint`` has ``extras``,
    but no ``dist``, or if the named extras are not defined by the
    distribution.  If `env` is supplied, it must be an ``Environment``, and it
    will be used to search for needed distributions if they are not already
    present on sys.path.  If `installer` is supplied, it must be a callable
    taking a ``Requirement`` instance and returning a matching importable
    ``Distribution`` instance or None.


``Distribution`` Objects
========================

Factories: get_provider, get_distribution, find_distributions; see also
WorkingSet and Environment APIs.

register_finder
register_loader_type
'load_entry_point', 'get_entry_map', 'get_entry_info'


``ResourceManager`` API
=======================

The ``ResourceManager`` class provides uniform access to package resources,
whether those resources exist as files and directories or are compressed in
an archive of some kind.

Normally, you do not need to create or explicitly manage ``ResourceManager``
instances, as the ``pkg_resources`` module creates a global instance for you,
and makes most of its methods available as top-level names in the
``pkg_resources`` module namespace.  So, for example, this code actually
calls the ``resource_string()`` method of the global ``ResourceManager``::

    import pkg_resources
    my_data = pkg_resources.resource_string(__name__, "foo.dat")

Thus, you can use the APIs below without needing an explicit
``ResourceManager`` instance; just import and use them as needed.


Basic Resource Access
---------------------

In the following methods, the `package_or_requirement` argument may be either
a Python package/module name (e.g. ``foo.bar``) or a ``Requirement`` instance.
If it is a package or module name, the named module or package must be
importable (i.e., be in a distribution or directory on ``sys.path``), and the
`resource_name` argument is interpreted relative to the named package.  (Note
that if a module name is used, then the resource name is relative to the
package immediately containing the named module.)

If it is a ``Requirement``, then the requirement is automatically resolved
(searching the current ``Environment`` if necessary) and a matching
distribution is added to the ``WorkingSet`` and ``sys.path`` if one was not
already present.  (Unless the ``Requirement`` can't be satisfied, in which
case an exception is raised.)  The `resource_name` argument is then interpreted
relative to the root of the identified distribution; i.e. its first path
segment will be treated as a peer of the top-level modules or packages in the
distribution.

Note that resource names must be ``/``-separated paths and cannot be absolute
(i.e. no leading ``/``) or contain relative names like ``..``.  Do *not* use
``os.path`` routines to manipulate resource paths, as they are *not* filesystem
paths.

``resource_exists(package_or_requirement, resource_name)``
    Does the named resource exist?  Return ``True`` or ``False`` accordingly.

``resource_stream(package_or_requirement, resource_name)``
    Return a readable file-like object for the specified resource; it may be
    an actual file, a ``StringIO``, or some similar object.  The stream is
    in "binary mode", in the sense that whatever bytes are in the resource
    will be read as-is.

``resource_string(package_or_requirement, resource_name)``
    Return the specified resource as a string.  The resource is read in
    binary fashion, such that the returned string contains exactly the bytes
    that are stored in the resource.

``resource_isdir(package_or_requirement, resource_name)``
    Is the named resource a directory?  Return ``True`` or ``False``
    accordingly.

``resource_listdir(package_or_requirement, resource_name)``
    List the contents of the named resource directory, just like ``os.listdir``
    except that it works even if the resource is in a zipfile.

Note that only ``resource_exists()`` and ``resource_isdir()`` are insensitive
as to the resource type.  You cannot use ``resource_listdir()`` on a file
resource, and you can't use ``resource_string()`` or ``resource_stream()`` on
directory resources.  Using an inappropriate method for the resource type may
result in an exception or undefined behavior, depending on the platform and
distribution format involved.


Resource Extraction
-------------------

``resource_filename(package_or_requirement, resource_name)``
    Sometimes, it is not sufficient to access a resource in string or stream
    form, and a true filesystem filename is needed.  In such cases, you can
    use this method (or module-level function) to obtain a filename for a
    resource.  If the resource is in an archive distribution (such as a zipped
    egg), it will be extracted to a cache directory, and the filename within
    the cache will be returned.  If the named resource is a directory, then
    all resources within that directory (including subdirectories) are also
    extracted.  If the named resource is a C extension or "eager resource"
    (see the ``setuptools`` documentation for details), then all C extensions
    and eager resources are extracted at the same time.

    Archived resources are extracted to a cache location that can be managed by
    the following two methods:

``set_extraction_path(path)``
    Set the base path where resources will be extracted to, if needed.

    If you do not call this routine before any extractions take place, the
    path defaults to the return value of ``get_default_cache()``.  (Which is
    based on the ``PYTHON_EGG_CACHE`` environment variable, with various
    platform-specific fallbacks.  See that routine's documentation for more
    details.)

    Resources are extracted to subdirectories of this path based upon
    information given by the ``IResourceProvider``.  You may set this to a
    temporary directory, but then you must call ``cleanup_resources()`` to
    delete the extracted files when done.  There is no guarantee that
    ``cleanup_resources()`` will be able to remove all extracted files.

    (Note: you may not change the extraction path for a given resource
    manager once resources have been extracted, unless you first call
    ``cleanup_resources()``.)

``cleanup_resources(force=False)``
    Delete all extracted resource files and directories, returning a list
    of the file and directory names that could not be successfully removed.
    This function does not have any concurrency protection, so it should
    generally only be called when the extraction path is a temporary
    directory exclusive to a single process.  This method is not
    automatically called; you must call it explicitly or register it as an
    ``atexit`` function if you wish to ensure cleanup of a temporary
    directory used for extractions.


"Provider" Interface
--------------------

If you are implementing an ``IResourceProvider`` and/or ``IMetadataProvider``
for a new distribution archive format, you may need to use the following
``ResourceManager`` methods to co-ordinate extraction of resources to the
filesystem.  If you're not implementing an archive format, however, you have
no need to use these methods.  Unlike the other methods listed above, they are
*not* available as top-level functions tied to the global ``ResourceManager``;
you must therefore have an explicit ``ResourceManager`` instance to use them.

``get_cache_path(archive_name, names=())``
    Return absolute location in cache for `archive_name` and `names`

    The parent directory of the resulting path will be created if it does
    not already exist.  `archive_name` should be the base filename of the
    enclosing egg (which may not be the name of the enclosing zipfile!),
    including its ".egg" extension.  `names`, if provided, should be a
    sequence of path name parts "under" the egg's extraction location.

    This method should only be called by resource providers that need to
    obtain an extraction location, and only for names they intend to
    extract, as it tracks the generated names for possible cleanup later.

``postprocess(tempname, filename)``
    Perform any platform-specific postprocessing of `tempname`.
    Resource providers should call this method ONLY after successfully
    extracting a compressed resource.  They must NOT call it on resources
    that are already in the filesystem.

    `tempname` is the current (temporary) name of the file, and `filename`
    is the name it will be renamed to by the caller after this routine
    returns.


Metadata API
============

The metadata API is used to access metadata resources bundled in a pluggable
distribution.  Metadata resources are virtual files or directories containing
information about the distribution, such as might be used by an extensible
application or framework to connect "plugins".  Like other kinds of resources,
metadata resource names are ``/``-separated and should not contain ``..`` or
begin with a ``/``.  You should not use ``os.path`` routines to manipulate
resource paths.

The metadata API is provided by objects implementing the ``IMetadataProvider``
interface.  ``Distribution`` objects created with a ``metadata`` setting
implement this interface, as do objects returned by the ``get_provider()``
function:

``get_provider(package_or_requirement)``
    If a package name is supplied, return an ``IMetadataProvider`` for the
    package.  If a ``Requirement`` is supplied, resolve it by returning a
    ``Distribution`` from the current working set (searching the current
    ``Environment`` if necessary and adding the newly found ``Distribution``
    to the working set).  If the named package can't be imported, or the
    ``Requirement`` can't be satisfied, an exception is raised.

    Note also that if you supply a package name, and the package is not part
    of a pluggable distribution (i.e., it has no metadata), then you will still
    get an ``IMetadataProvider`` object, but it will just be an empty one that
    answers ``False`` when asked if any given metadata resource file or
    directory exists.

The methods provided by ``IMetadataProvider`` (and ``Distribution`` objects
with a ``metadata`` attribute set) are:

``has_metadata(name)``
    Does the named metadata resource exist?

``metadata_isdir(name)``
    Is the named metadata resource a directory?

``metadata_listdir(name)``
    List of metadata names in the directory (like ``os.listdir()``)

``get_metadata(name)``
    Return the named metadata resource as a string.  The data is read in binary
    mode; i.e., the exact bytes of the resource file are returned.

``get_metadata_lines(name)``
    Yield named metadata resource as list of non-blank non-comment lines.  This
    is short for calling ``yield_lines(provider.get_metadata(name))``.  See the
    section on `yield_lines()`_ below for more information on the syntax it
    recognizes.

``run_script(script_name, namespace)``
    Execute the named script in the supplied namespace dictionary.  Raises
    ``ResolutionError`` if there is no script by that name in the ``scripts``
    metadata directory.  `namespace` should be a Python dictionary, usually
    a module dictionary if the script is being run as a module.


Exceptions
==========

``pkg_resources`` provides a simple exception hierarchy for problems that may
occur when processing requests to locate and activate packages::

    ResolutionError
        DistributionNotFound
        VersionConflict
        UnknownExtra

``ResolutionError``
    This class is used as a base class for the other three exceptions, so that
    you can catch all of them with a single "except" clause.  It is also raised
    directly for miscellaneous requirement-resolution problems like trying to
    run a script that doesn't exist in the distribution it was requested from.

``DistributionNotFound``
    A distribution needed to fulfill a requirement could not be found.

``VersionConflict``
    The requested version of a project conflicts with an already-activated
    version of the same project.

``UnknownExtra``
    One of the "extras" requested was not recognized by the distribution it
    was requested from.


Utility Functions
=================

In addition to its high-level APIs, ``pkg_resources`` also includes several
generally-useful utility routines.  These routines are used to implement the
high-level APIs, but can also be quite useful by themselves.


Parsing Utilities
-----------------

``parse_version(version)``
    Parse a project's version string, returning a value that can be used to
    compare versions by chronological order.  Semantically, the format is a
    rough cross between distutils' ``StrictVersion`` and ``LooseVersion``
    classes; if you give it versions that would work with ``StrictVersion``,
    then they will compare the same way.  Otherwise, comparisons are more like
    a "smarter" form of ``LooseVersion``.  It is *possible* to create
    pathological version coding schemes that will fool this parser, but they
    should be very rare in practice.

    The returned value will be a tuple of strings.  Numeric portions of the
    version are padded to 8 digits so they will compare numerically, but
    without relying on how numbers compare relative to strings.  Dots are
    dropped, but dashes are retained.  Trailing zeros between alpha segments
    or dashes are suppressed, so that e.g. "2.4.0" is considered the same as
    "2.4". Alphanumeric parts are lower-cased.

    The algorithm assumes that strings like "-" and any alpha string that
    alphabetically follows "final"  represents a "patch level".  So, "2.4-1"
    is assumed to be a branch or patch of "2.4", and therefore "2.4.1" is
    considered newer than "2.4-1", whic in turn is newer than "2.4".

    Strings like "a", "b", "c", "alpha", "beta", "candidate" and so on (that
    come before "final" alphabetically) are assumed to be pre-release versions,
    so that the version "2.4" is considered newer than "2.4a1".

    Finally, to handle miscellaneous cases, the strings "pre", "preview", and
    "rc" are treated as if they were "c", i.e. as though they were release
    candidates, and therefore are not as new as a version string that does not
    contain them.

.. _yield_lines():

``yield_lines(strs)``
    Yield non-empty/non-comment lines from a string/unicode or a possibly-
    nested sequence thereof.  If `strs` is an instance of ``basestring``, it
    is split into lines, and each non-blank, non-comment line is yielded after
    stripping leading and trailing whitespace.  (Lines whose first non-blank
    character is ``#`` are considered comment lines.)

    If `strs` is not an instance of ``basestring``, it is iterated over, and
    each item is passed recursively to ``yield_lines()``, so that an arbitarily
    nested sequence of strings, or sequences of sequences of strings can be
    flattened out to the lines contained therein.  So for example, passing
    a file object or a list of strings to ``yield_lines`` will both work.
    (Note that between each string in a sequence of strings there is assumed to
    be an implicit line break, so lines cannot bridge two strings in a
    sequence.)

    This routine is used extensively by ``pkg_resources`` to parse metadata
    and file formats of various kinds, and most other ``pkg_resources``
    parsing functions that yield multiple values will use it to break up their
    input.  However, this routine is idempotent, so calling ``yield_lines()``
    on the output of another call to ``yield_lines()`` is completely harmless.

``split_sections(strs)``
    Split a string (or possibly-nested iterable thereof), yielding ``(section,
    content)`` pairs found using an ``.ini``-like syntax.  Each ``section`` is
    a whitespace-stripped version of the section name ("``[section]``")
    and each ``content`` is a list of stripped lines excluding blank lines and
    comment-only lines.  If there are any non-blank, non-comment lines before
    the first section header, they're yielded in a first ``section`` of
    ``None``.

    This routine uses ``yield_lines()`` as its front end, so you can pass in
    anything that ``yield_lines()`` accepts, such as an open text file, string,
    or sequence of strings.  ``ValueError`` is raised if a malformed section
    header is found (i.e. a line starting with ``[`` but not ending with
    ``]``).

    Note that this simplistic parser assumes that any line whose first nonblank
    character is ``[`` is a section heading, so it can't support .ini format
    variations that allow ``[`` as the first nonblank character on other lines.

``safe_name(name)``
    Return a "safe" form of a project's name, suitable for use in a
    ``Requirement`` string, as a distribution name, or a PyPI project name.
    All non-alphanumeric runs are condensed to single "-" characters, such that
    a name like "The $$$ Tree" becomes "The-Tree".  Note that if you are
    generating a filename from this value you should replace the "-" characters
    with underscores ("_") because setuptools and the distutils

``safe_version(version)``
    Similar to ``safe_name()`` except that spaces in the input become dots, and
    dots are allowed to exist in the output.  As with ``safe_name()``, if you
    are generating a filename from this you should replace any "-" characters
    in the output with underscores.


Platform Utilities
------------------

``get_platform()``
    Return this platform's identifier string.  For Windows, the return value
    is ``"win32"``, and for Mac OS X it is a string of the form
    ``"macosx-10.4-ppc"``.  All other platforms return the same uname-based
    string that the ``distutils.util.get_platform()`` function returns.

``compatible_platforms(provided, required)``
    Return true if a distribution built on the `provided` platform may be used
    on the `required` platform.  If either platform value is ``None``, it is
    considered a wildcard, and the platforms are therefore compatible.
    Likewise, if the platform strings are equal, they're also considered
    compatible, and ``True`` is returned.  Currently, the only non-equal
    platform strings that are considered compatible are Mac OS X platform
    strings with the same hardware type (e.g. ``ppc``) and major version
    (e.g. ``10``) with the `provided` platform's minor version being less than
    or equal to the `required` platform's minor version.

``get_default_cache()``
    Determine the default cache location for extracting resources from zipped
    eggs.  This routine returns the ``PYTHON_EGG_CACHE`` environment variable,
    if set.  Otherwise, on Windows, it returns a "Python-Eggs" subdirectory of
    the user's "Application Data" directory.  On all other systems, it returns
    ``os.path.expanduser("~/.python-eggs")`` if ``PYTHON_EGG_CACHE`` is not
    set.


PEP 302 Utilities
-----------------

``get_importer(path_item)``
    Retrieve a PEP 302 "importer" for the given path item (which need not
    actually be on ``sys.path``).  This routine simulates the PEP 302 protocol
    for obtaining an "importer" object.  It first checks for an importer for
    the path item in ``sys.path_importer_cache``, and if not found it calls
    each of the ``sys.path_hooks`` and caches the result if a good importer is
    found.  If no importer is found, this routine returns an ``ImpWrapper``
    instance that wraps the builtin import machinery as a PEP 302-compliant
    "importer" object.  This ``ImpWrapper`` is *not* cached; instead a new
    instance is returned each time.


File/Path Utilities
-------------------

``ensure_directory(path)``
    Ensure that the parent directory (``os.path.dirname``) of `path` actually
    exists, using ``os.makedirs()`` if necessary.

``normalize_path(path)``
    Return a "normalized" version of `path`, such that two paths represent
    the same filesystem location if they have equal ``normalized_path()``
    values.  Specifically, this is a shortcut for calling ``os.path.realpath``
    and ``os.path.normcase`` on `path`.  Unfortunately, on certain platforms
    (notably Cygwin and Mac OS X) the ``normcase`` function does not accurately
    reflect the platform's case-sensitivity, so there is always the possibility
    of two apparently-different paths being equal on such platforms.