diff options
Diffstat (limited to 'docs/setuptools.txt')
| -rw-r--r-- | docs/setuptools.txt | 3230 |
1 files changed, 3230 insertions, 0 deletions
diff --git a/docs/setuptools.txt b/docs/setuptools.txt new file mode 100644 index 00000000..2504dcda --- /dev/null +++ b/docs/setuptools.txt @@ -0,0 +1,3230 @@ +================================================== +Building and Distributing Packages with Setuptools +================================================== + +``Setuptools`` is a collection of enhancements to the Python ``distutils`` +(for Python 2.3.5 and up on most platforms; 64-bit platforms require a minimum +of Python 2.4) that allow you to more easily build and distribute Python +packages, especially ones that have dependencies on other packages. + +Packages built and distributed using ``setuptools`` look to the user like +ordinary Python packages based on the ``distutils``. Your users don't need to +install or even know about setuptools in order to use them, and you don't +have to include the entire setuptools package in your distributions. By +including just a single `bootstrap module`_ (an 8K .py file), your package will +automatically download and install ``setuptools`` if the user is building your +package from source and doesn't have a suitable version already installed. + +.. _bootstrap module: http://bitbucket.org/jaraco/setuptools/downloads/ez_setup.py + +Feature Highlights: + +* Automatically find/download/install/upgrade dependencies at build time using + the `EasyInstall tool <http://peak.telecommunity.com/DevCenter/EasyInstall>`_, + which supports downloading via HTTP, FTP, Subversion, and SourceForge, and + automatically scans web pages linked from PyPI to find download links. (It's + the closest thing to CPAN currently available for Python.) + +* Create `Python Eggs <http://peak.telecommunity.com/DevCenter/PythonEggs>`_ - + a single-file importable distribution format + +* Include data files inside your package directories, where your code can + actually use them. (Python 2.4 distutils also supports this feature, but + setuptools provides the feature for Python 2.3 packages also, and supports + accessing data files in zipped packages too.) + +* Automatically include all packages in your source tree, without listing them + individually in setup.py + +* Automatically include all relevant files in your source distributions, + without needing to create a ``MANIFEST.in`` file, and without having to force + regeneration of the ``MANIFEST`` file when your source tree changes. + +* Automatically generate wrapper scripts or Windows (console and GUI) .exe + files for any number of "main" functions in your project. (Note: this is not + a py2exe replacement; the .exe files rely on the local Python installation.) + +* Transparent Pyrex support, so that your setup.py can list ``.pyx`` files and + still work even when the end-user doesn't have Pyrex installed (as long as + you include the Pyrex-generated C in your source distribution) + +* Command aliases - create project-specific, per-user, or site-wide shortcut + names for commonly used commands and options + +* PyPI upload support - upload your source distributions and eggs to PyPI + +* Deploy your project in "development mode", such that it's available on + ``sys.path``, yet can still be edited directly from its source checkout. + +* Easily extend the distutils with new commands or ``setup()`` arguments, and + distribute/reuse your extensions for multiple projects, without copying code. + +* Create extensible applications and frameworks that automatically discover + extensions, using simple "entry points" declared in a project's setup script. + +In addition to the PyPI downloads, the development version of ``setuptools`` +is available from the `Python SVN sandbox`_, and in-development versions of the +`0.6 branch`_ are available as well. + +.. _0.6 branch: http://svn.python.org/projects/sandbox/branches/setuptools-0.6/#egg=setuptools-dev06 + +.. _Python SVN sandbox: http://svn.python.org/projects/sandbox/trunk/setuptools/#egg=setuptools-dev + +.. contents:: **Table of Contents** + +.. _distribute_setup.py: `bootstrap module`_ + + +----------------- +Developer's Guide +----------------- + + +Installing ``setuptools`` +========================= + +Please follow the `EasyInstall Installation Instructions`_ to install the +current stable version of setuptools. In particular, be sure to read the +section on `Custom Installation Locations`_ if you are installing anywhere +other than Python's ``site-packages`` directory. + +.. _EasyInstall Installation Instructions: http://peak.telecommunity.com/DevCenter/EasyInstall#installation-instructions + +.. _Custom Installation Locations: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations + +If you want the current in-development version of setuptools, you should first +install a stable version, and then run:: + + distribute_setup.py setuptools==dev + +This will download and install the latest development (i.e. unstable) version +of setuptools from the Python Subversion sandbox. + + +Basic Use +========= + +For basic use of setuptools, just import things from setuptools instead of +the distutils. Here's a minimal setup script using setuptools:: + + from setuptools import setup, find_packages + setup( + name = "HelloWorld", + version = "0.1", + packages = find_packages(), + ) + +As you can see, it doesn't take much to use setuptools in a project. +Just by doing the above, this project will be able to produce eggs, upload to +PyPI, and automatically include all packages in the directory where the +setup.py lives. See the `Command Reference`_ section below to see what +commands you can give to this setup script. + +Of course, before you release your project to PyPI, you'll want to add a bit +more information to your setup script to help people find or learn about your +project. And maybe your project will have grown by then to include a few +dependencies, and perhaps some data files and scripts:: + + from setuptools import setup, find_packages + setup( + name = "HelloWorld", + version = "0.1", + packages = find_packages(), + scripts = ['say_hello.py'], + + # Project uses reStructuredText, so ensure that the docutils get + # installed or upgraded on the target machine + install_requires = ['docutils>=0.3'], + + package_data = { + # If any package contains *.txt or *.rst files, include them: + '': ['*.txt', '*.rst'], + # And include any *.msg files found in the 'hello' package, too: + 'hello': ['*.msg'], + }, + + # metadata for upload to PyPI + author = "Me", + author_email = "me@example.com", + description = "This is an Example Package", + license = "PSF", + keywords = "hello world example examples", + url = "http://example.com/HelloWorld/", # project home page, if any + + # could also include long_description, download_url, classifiers, etc. + ) + +In the sections that follow, we'll explain what most of these ``setup()`` +arguments do (except for the metadata ones), and the various ways you might use +them in your own project(s). + + +Specifying Your Project's Version +--------------------------------- + +Setuptools can work well with most versioning schemes; there are, however, a +few special things to watch out for, in order to ensure that setuptools and +EasyInstall can always tell what version of your package is newer than another +version. Knowing these things will also help you correctly specify what +versions of other projects your project depends on. + +A version consists of an alternating series of release numbers and pre-release +or post-release tags. A release number is a series of digits punctuated by +dots, such as ``2.4`` or ``0.5``. Each series of digits is treated +numerically, so releases ``2.1`` and ``2.1.0`` are different ways to spell the +same release number, denoting the first subrelease of release 2. But ``2.10`` +is the *tenth* subrelease of release 2, and so is a different and newer release +from ``2.1`` or ``2.1.0``. Leading zeros within a series of digits are also +ignored, so ``2.01`` is the same as ``2.1``, and different from ``2.0.1``. + +Following a release number, you can have either a pre-release or post-release +tag. Pre-release tags make a version be considered *older* than the version +they are appended to. So, revision ``2.4`` is *newer* than revision ``2.4c1``, +which in turn is newer than ``2.4b1`` or ``2.4a1``. Postrelease tags make +a version be considered *newer* than the version they are appended to. So, +revisions like ``2.4-1`` and ``2.4pl3`` are newer than ``2.4``, but are *older* +than ``2.4.1`` (which has a higher release number). + +A pre-release tag is a series of letters that are alphabetically before +"final". Some examples of prerelease tags would include ``alpha``, ``beta``, +``a``, ``c``, ``dev``, and so on. You do not have to place a dot or dash +before the prerelease tag if it's immediately after a number, but it's okay to +do so if you prefer. Thus, ``2.4c1`` and ``2.4.c1`` and ``2.4-c1`` all +represent release candidate 1 of version ``2.4``, and are treated as identical +by setuptools. + +In addition, there are three special prerelease tags that are treated as if +they were the letter ``c``: ``pre``, ``preview``, and ``rc``. So, version +``2.4rc1``, ``2.4pre1`` and ``2.4preview1`` are all the exact same version as +``2.4c1``, and are treated as identical by setuptools. + +A post-release tag is either a series of letters that are alphabetically +greater than or equal to "final", or a dash (``-``). Post-release tags are +generally used to separate patch numbers, port numbers, build numbers, revision +numbers, or date stamps from the release number. For example, the version +``2.4-r1263`` might denote Subversion revision 1263 of a post-release patch of +version ``2.4``. Or you might use ``2.4-20051127`` to denote a date-stamped +post-release. + +Notice that after each pre or post-release tag, you are free to place another +release number, followed again by more pre- or post-release tags. For example, +``0.6a9.dev-r41475`` could denote Subversion revision 41475 of the in- +development version of the ninth alpha of release 0.6. Notice that ``dev`` is +a pre-release tag, so this version is a *lower* version number than ``0.6a9``, +which would be the actual ninth alpha of release 0.6. But the ``-r41475`` is +a post-release tag, so this version is *newer* than ``0.6a9.dev``. + +For the most part, setuptools' interpretation of version numbers is intuitive, +but here are a few tips that will keep you out of trouble in the corner cases: + +* Don't stick adjoining pre-release tags together without a dot or number + between them. Version ``1.9adev`` is the ``adev`` prerelease of ``1.9``, + *not* a development pre-release of ``1.9a``. Use ``.dev`` instead, as in + ``1.9a.dev``, or separate the prerelease tags with a number, as in + ``1.9a0dev``. ``1.9a.dev``, ``1.9a0dev``, and even ``1.9.a.dev`` are + identical versions from setuptools' point of view, so you can use whatever + scheme you prefer. + +* If you want to be certain that your chosen numbering scheme works the way + you think it will, you can use the ``pkg_resources.parse_version()`` function + to compare different version numbers:: + + >>> from pkg_resources import parse_version + >>> parse_version('1.9.a.dev') == parse_version('1.9a0dev') + True + >>> parse_version('2.1-rc2') < parse_version('2.1') + True + >>> parse_version('0.6a9dev-r41475') < parse_version('0.6a9') + True + +Once you've decided on a version numbering scheme for your project, you can +have setuptools automatically tag your in-development releases with various +pre- or post-release tags. See the following sections for more details: + +* `Tagging and "Daily Build" or "Snapshot" Releases`_ +* `Managing "Continuous Releases" Using Subversion`_ +* The `egg_info`_ command + + +New and Changed ``setup()`` Keywords +==================================== + +The following keyword arguments to ``setup()`` are added or changed by +``setuptools``. All of them are optional; you do not have to supply them +unless you need the associated ``setuptools`` feature. + +``include_package_data`` + If set to ``True``, this tells ``setuptools`` to automatically include any + data files it finds inside your package directories, that are either under + CVS or Subversion control, or which are specified by your ``MANIFEST.in`` + file. For more information, see the section below on `Including Data + Files`_. + +``exclude_package_data`` + A dictionary mapping package names to lists of glob patterns that should + be *excluded* from your package directories. You can use this to trim back + any excess files included by ``include_package_data``. For a complete + description and examples, see the section below on `Including Data Files`_. + +``package_data`` + A dictionary mapping package names to lists of glob patterns. For a + complete description and examples, see the section below on `Including + Data Files`_. You do not need to use this option if you are using + ``include_package_data``, unless you need to add e.g. files that are + generated by your setup script and build process. (And are therefore not + in source control or are files that you don't want to include in your + source distribution.) + +``zip_safe`` + A boolean (True or False) flag specifying whether the project can be + safely installed and run from a zip file. If this argument is not + supplied, the ``bdist_egg`` command will have to analyze all of your + project's contents for possible problems each time it buids an egg. + +``install_requires`` + A string or list of strings specifying what other distributions need to + be installed when this one is. See the section below on `Declaring + Dependencies`_ for details and examples of the format of this argument. + +``entry_points`` + A dictionary mapping entry point group names to strings or lists of strings + defining the entry points. Entry points are used to support dynamic + discovery of services or plugins provided by a project. See `Dynamic + Discovery of Services and Plugins`_ for details and examples of the format + of this argument. In addition, this keyword is used to support `Automatic + Script Creation`_. + +``extras_require`` + A dictionary mapping names of "extras" (optional features of your project) + to strings or lists of strings specifying what other distributions must be + installed to support those features. See the section below on `Declaring + Dependencies`_ for details and examples of the format of this argument. + +``setup_requires`` + A string or list of strings specifying what other distributions need to + be present in order for the *setup script* to run. ``setuptools`` will + attempt to obtain these (even going so far as to download them using + ``EasyInstall``) before processing the rest of the setup script or commands. + This argument is needed if you are using distutils extensions as part of + your build process; for example, extensions that process setup() arguments + and turn them into EGG-INFO metadata files. + + (Note: projects listed in ``setup_requires`` will NOT be automatically + installed on the system where the setup script is being run. They are + simply downloaded to the setup directory if they're not locally available + already. If you want them to be installed, as well as being available + when the setup script is run, you should add them to ``install_requires`` + **and** ``setup_requires``.) + +``dependency_links`` + A list of strings naming URLs to be searched when satisfying dependencies. + These links will be used if needed to install packages specified by + ``setup_requires`` or ``tests_require``. They will also be written into + the egg's metadata for use by tools like EasyInstall to use when installing + an ``.egg`` file. + +``namespace_packages`` + A list of strings naming the project's "namespace packages". A namespace + package is a package that may be split across multiple project + distributions. For example, Zope 3's ``zope`` package is a namespace + package, because subpackages like ``zope.interface`` and ``zope.publisher`` + may be distributed separately. The egg runtime system can automatically + merge such subpackages into a single parent package at runtime, as long + as you declare them in each project that contains any subpackages of the + namespace package, and as long as the namespace package's ``__init__.py`` + does not contain any code other than a namespace declaration. See the + section below on `Namespace Packages`_ for more information. + +``test_suite`` + A string naming a ``unittest.TestCase`` subclass (or a package or module + containing one or more of them, or a method of such a subclass), or naming + a function that can be called with no arguments and returns a + ``unittest.TestSuite``. If the named suite is a module, and the module + has an ``additional_tests()`` function, it is called and the results are + added to the tests to be run. If the named suite is a package, any + submodules and subpackages are recursively added to the overall test suite. + + Specifying this argument enables use of the `test`_ command to run the + specified test suite, e.g. via ``setup.py test``. See the section on the + `test`_ command below for more details. + +``tests_require`` + If your project's tests need one or more additional packages besides those + needed to install it, you can use this option to specify them. It should + be a string or list of strings specifying what other distributions need to + be present for the package's tests to run. When you run the ``test`` + command, ``setuptools`` will attempt to obtain these (even going + so far as to download them using ``EasyInstall``). Note that these + required projects will *not* be installed on the system where the tests + are run, but only downloaded to the project's setup directory if they're + not already installed locally. + +.. _test_loader: + +``test_loader`` + If you would like to use a different way of finding tests to run than what + setuptools normally uses, you can specify a module name and class name in + this argument. The named class must be instantiable with no arguments, and + its instances must support the ``loadTestsFromNames()`` method as defined + in the Python ``unittest`` module's ``TestLoader`` class. Setuptools will + pass only one test "name" in the `names` argument: the value supplied for + the ``test_suite`` argument. The loader you specify may interpret this + string in any way it likes, as there are no restrictions on what may be + contained in a ``test_suite`` string. + + The module name and class name must be separated by a ``:``. The default + value of this argument is ``"setuptools.command.test:ScanningLoader"``. If + you want to use the default ``unittest`` behavior, you can specify + ``"unittest:TestLoader"`` as your ``test_loader`` argument instead. This + will prevent automatic scanning of submodules and subpackages. + + The module and class you specify here may be contained in another package, + as long as you use the ``tests_require`` option to ensure that the package + containing the loader class is available when the ``test`` command is run. + +``eager_resources`` + A list of strings naming resources that should be extracted together, if + any of them is needed, or if any C extensions included in the project are + imported. This argument is only useful if the project will be installed as + a zipfile, and there is a need to have all of the listed resources be + extracted to the filesystem *as a unit*. Resources listed here + should be '/'-separated paths, relative to the source root, so to list a + resource ``foo.png`` in package ``bar.baz``, you would include the string + ``bar/baz/foo.png`` in this argument. + + If you only need to obtain resources one at a time, or you don't have any C + extensions that access other files in the project (such as data files or + shared libraries), you probably do NOT need this argument and shouldn't + mess with it. For more details on how this argument works, see the section + below on `Automatic Resource Extraction`_. + +``use_2to3`` + Convert the source code from Python 2 to Python 3 with 2to3 during the + build process. See :doc:`python3` for more details. + +``convert_2to3_doctests`` + List of doctest source files that need to be converted with 2to3. + See :doc:`python3` for more details. + +``use_2to3_fixers`` + A list of modules to search for additional fixers to be used during + the 2to3 conversion. See :doc:`python3` for more details. + + +Using ``find_packages()`` +------------------------- + +For simple projects, it's usually easy enough to manually add packages to +the ``packages`` argument of ``setup()``. However, for very large projects +(Twisted, PEAK, Zope, Chandler, etc.), it can be a big burden to keep the +package list updated. That's what ``setuptools.find_packages()`` is for. + +``find_packages()`` takes a source directory, and a list of package names or +patterns to exclude. If omitted, the source directory defaults to the same +directory as the setup script. Some projects use a ``src`` or ``lib`` +directory as the root of their source tree, and those projects would of course +use ``"src"`` or ``"lib"`` as the first argument to ``find_packages()``. (And +such projects also need something like ``package_dir = {'':'src'}`` in their +``setup()`` arguments, but that's just a normal distutils thing.) + +Anyway, ``find_packages()`` walks the target directory, and finds Python +packages by looking for ``__init__.py`` files. It then filters the list of +packages using the exclusion patterns. + +Exclusion patterns are package names, optionally including wildcards. For +example, ``find_packages(exclude=["*.tests"])`` will exclude all packages whose +last name part is ``tests``. Or, ``find_packages(exclude=["*.tests", +"*.tests.*"])`` will also exclude any subpackages of packages named ``tests``, +but it still won't exclude a top-level ``tests`` package or the children +thereof. In fact, if you really want no ``tests`` packages at all, you'll need +something like this:: + + find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]) + +in order to cover all the bases. Really, the exclusion patterns are intended +to cover simpler use cases than this, like excluding a single, specified +package and its subpackages. + +Regardless of the target directory or exclusions, the ``find_packages()`` +function returns a list of package names suitable for use as the ``packages`` +argument to ``setup()``, and so is usually the easiest way to set that +argument in your setup script. Especially since it frees you from having to +remember to modify your setup script whenever your project grows additional +top-level packages or subpackages. + + +Automatic Script Creation +========================= + +Packaging and installing scripts can be a bit awkward with the distutils. For +one thing, there's no easy way to have a script's filename match local +conventions on both Windows and POSIX platforms. For another, you often have +to create a separate file just for the "main" script, when your actual "main" +is a function in a module somewhere. And even in Python 2.4, using the ``-m`` +option only works for actual ``.py`` files that aren't installed in a package. + +``setuptools`` fixes all of these problems by automatically generating scripts +for you with the correct extension, and on Windows it will even create an +``.exe`` file so that users don't have to change their ``PATHEXT`` settings. +The way to use this feature is to define "entry points" in your setup script +that indicate what function the generated script should import and run. For +example, to create two console scripts called ``foo`` and ``bar``, and a GUI +script called ``baz``, you might do something like this:: + + setup( + # other arguments here... + entry_points = { + 'console_scripts': [ + 'foo = my_package.some_module:main_func', + 'bar = other_module:some_func', + ], + 'gui_scripts': [ + 'baz = my_package_gui.start_func', + ] + } + ) + +When this project is installed on non-Windows platforms (using "setup.py +install", "setup.py develop", or by using EasyInstall), a set of ``foo``, +``bar``, and ``baz`` scripts will be installed that import ``main_func`` and +``some_func`` from the specified modules. The functions you specify are called +with no arguments, and their return value is passed to ``sys.exit()``, so you +can return an errorlevel or message to print to stderr. + +On Windows, a set of ``foo.exe``, ``bar.exe``, and ``baz.exe`` launchers are +created, alongside a set of ``foo.py``, ``bar.py``, and ``baz.pyw`` files. The +``.exe`` wrappers find and execute the right version of Python to run the +``.py`` or ``.pyw`` file. + +You may define as many "console script" and "gui script" entry points as you +like, and each one can optionally specify "extras" that it depends on, that +will be added to ``sys.path`` when the script is run. For more information on +"extras", see the section below on `Declaring Extras`_. For more information +on "entry points" in general, see the section below on `Dynamic Discovery of +Services and Plugins`_. + + +"Eggsecutable" Scripts +---------------------- + +Occasionally, there are situations where it's desirable to make an ``.egg`` +file directly executable. You can do this by including an entry point such +as the following:: + + setup( + # other arguments here... + entry_points = { + 'setuptools.installation': [ + 'eggsecutable = my_package.some_module:main_func', + ] + } + ) + +Any eggs built from the above setup script will include a short excecutable +prelude that imports and calls ``main_func()`` from ``my_package.some_module``. +The prelude can be run on Unix-like platforms (including Mac and Linux) by +invoking the egg with ``/bin/sh``, or by enabling execute permissions on the +``.egg`` file. For the executable prelude to run, the appropriate version of +Python must be available via the ``PATH`` environment variable, under its +"long" name. That is, if the egg is built for Python 2.3, there must be a +``python2.3`` executable present in a directory on ``PATH``. + +This feature is primarily intended to support distribute_setup the installation of +setuptools itself on non-Windows platforms, but may also be useful for other +projects as well. + +IMPORTANT NOTE: Eggs with an "eggsecutable" header cannot be renamed, or +invoked via symlinks. They *must* be invoked using their original filename, in +order to ensure that, once running, ``pkg_resources`` will know what project +and version is in use. The header script will check this and exit with an +error if the ``.egg`` file has been renamed or is invoked via a symlink that +changes its base name. + + +Declaring Dependencies +====================== + +``setuptools`` supports automatically installing dependencies when a package is +installed, and including information about dependencies in Python Eggs (so that +package management tools like EasyInstall can use the information). + +``setuptools`` and ``pkg_resources`` use a common syntax for specifying a +project's required dependencies. This syntax consists of a project's PyPI +name, optionally followed by a comma-separated list of "extras" in square +brackets, optionally followed by a comma-separated list of version +specifiers. A version specifier is one of the operators ``<``, ``>``, ``<=``, +``>=``, ``==`` or ``!=``, followed by a version identifier. Tokens may be +separated by whitespace, but any whitespace or nonstandard characters within a +project name or version identifier must be replaced with ``-``. + +Version specifiers for a given project are internally sorted into ascending +version order, and used to establish what ranges of versions are acceptable. +Adjacent redundant conditions are also consolidated (e.g. ``">1, >2"`` becomes +``">1"``, and ``"<2,<3"`` becomes ``"<3"``). ``"!="`` versions are excised from +the ranges they fall within. A project's version is then checked for +membership in the resulting ranges. (Note that providing conflicting conditions +for the same version (e.g. "<2,>=2" or "==2,!=2") is meaningless and may +therefore produce bizarre results.) + +Here are some example requirement specifiers:: + + docutils >= 0.3 + + # comment lines and \ continuations are allowed in requirement strings + BazSpam ==1.1, ==1.2, ==1.3, ==1.4, ==1.5, \ + ==1.6, ==1.7 # and so are line-end comments + + PEAK[FastCGI, reST]>=0.5a4 + + setuptools==0.5a7 + +The simplest way to include requirement specifiers is to use the +``install_requires`` argument to ``setup()``. It takes a string or list of +strings containing requirement specifiers. If you include more than one +requirement in a string, each requirement must begin on a new line. + +This has three effects: + +1. When your project is installed, either by using EasyInstall, ``setup.py + install``, or ``setup.py develop``, all of the dependencies not already + installed will be located (via PyPI), downloaded, built (if necessary), + and installed. + +2. Any scripts in your project will be installed with wrappers that verify + the availability of the specified dependencies at runtime, and ensure that + the correct versions are added to ``sys.path`` (e.g. if multiple versions + have been installed). + +3. Python Egg distributions will include a metadata file listing the + dependencies. + +Note, by the way, that if you declare your dependencies in ``setup.py``, you do +*not* need to use the ``require()`` function in your scripts or modules, as +long as you either install the project or use ``setup.py develop`` to do +development work on it. (See `"Development Mode"`_ below for more details on +using ``setup.py develop``.) + + +Dependencies that aren't in PyPI +-------------------------------- + +If your project depends on packages that aren't registered in PyPI, you may +still be able to depend on them, as long as they are available for download +as: + +- an egg, in the standard distutils ``sdist`` format, +- a single ``.py`` file, or +- a VCS repository (Subversion, Mercurial, or Git). + +You just need to add some URLs to the ``dependency_links`` argument to +``setup()``. + +The URLs must be either: + +1. direct download URLs, +2. the URLs of web pages that contain direct download links, or +3. the repository's URL + +In general, it's better to link to web pages, because it is usually less +complex to update a web page than to release a new version of your project. +You can also use a SourceForge ``showfiles.php`` link in the case where a +package you depend on is distributed via SourceForge. + +If you depend on a package that's distributed as a single ``.py`` file, you +must include an ``"#egg=project-version"`` suffix to the URL, to give a project +name and version number. (Be sure to escape any dashes in the name or version +by replacing them with underscores.) EasyInstall will recognize this suffix +and automatically create a trivial ``setup.py`` to wrap the single ``.py`` file +as an egg. + +In the case of a VCS checkout, you should also append ``#egg=project-version`` +in order to identify for what package that checkout should be used. You can +append ``@REV`` to the URL's path (before the fragment) to specify a revision. +Additionally, you can also force the VCS being used by prepending the URL with +a certain prefix. Currently available are: + +- ``svn+URL`` for Subversion, +- ``git+URL`` for Git, and +- ``hg+URL`` for Mercurial + +A more complete example would be: + + ``vcs+proto://host/path@revision#egg=project-version`` + +Be careful with the version. It should match the one inside the project files. +If you want do disregard the version, you have to omit it both in the +``requires`` and in the URL's fragment. + +This will do a checkout (or a clone, in Git and Mercurial parlance) to a +temporary folder and run ``setup.py bdist_egg``. + +The ``dependency_links`` option takes the form of a list of URL strings. For +example, the below will cause EasyInstall to search the specified page for +eggs or source distributions, if the package's dependencies aren't already +installed:: + + setup( + ... + dependency_links = [ + "http://peak.telecommunity.com/snapshots/" + ], + ) + + +.. _Declaring Extras: + + +Declaring "Extras" (optional features with their own dependencies) +------------------------------------------------------------------ + +Sometimes a project has "recommended" dependencies, that are not required for +all uses of the project. For example, a project might offer optional PDF +output if ReportLab is installed, and reStructuredText support if docutils is +installed. These optional features are called "extras", and setuptools allows +you to define their requirements as well. In this way, other projects that +require these optional features can force the additional requirements to be +installed, by naming the desired extras in their ``install_requires``. + +For example, let's say that Project A offers optional PDF and reST support:: + + setup( + name="Project-A", + ... + extras_require = { + 'PDF': ["ReportLab>=1.2", "RXP"], + 'reST': ["docutils>=0.3"], + } + ) + +As you can see, the ``extras_require`` argument takes a dictionary mapping +names of "extra" features, to strings or lists of strings describing those +features' requirements. These requirements will *not* be automatically +installed unless another package depends on them (directly or indirectly) by +including the desired "extras" in square brackets after the associated project +name. (Or if the extras were listed in a requirement spec on the EasyInstall +command line.) + +Extras can be used by a project's `entry points`_ to specify dynamic +dependencies. For example, if Project A includes a "rst2pdf" script, it might +declare it like this, so that the "PDF" requirements are only resolved if the +"rst2pdf" script is run:: + + setup( + name="Project-A", + ... + entry_points = { + 'console_scripts': + ['rst2pdf = project_a.tools.pdfgen [PDF]'], + ['rst2html = project_a.tools.htmlgen'], + # more script entry points ... + } + ) + +Projects can also use another project's extras when specifying dependencies. +For example, if project B needs "project A" with PDF support installed, it +might declare the dependency like this:: + + setup( + name="Project-B", + install_requires = ["Project-A[PDF]"], + ... + ) + +This will cause ReportLab to be installed along with project A, if project B is +installed -- even if project A was already installed. In this way, a project +can encapsulate groups of optional "downstream dependencies" under a feature +name, so that packages that depend on it don't have to know what the downstream +dependencies are. If a later version of Project A builds in PDF support and +no longer needs ReportLab, or if it ends up needing other dependencies besides +ReportLab in order to provide PDF support, Project B's setup information does +not need to change, but the right packages will still be installed if needed. + +Note, by the way, that if a project ends up not needing any other packages to +support a feature, it should keep an empty requirements list for that feature +in its ``extras_require`` argument, so that packages depending on that feature +don't break (due to an invalid feature name). For example, if Project A above +builds in PDF support and no longer needs ReportLab, it could change its +setup to this:: + + setup( + name="Project-A", + ... + extras_require = { + 'PDF': [], + 'reST': ["docutils>=0.3"], + } + ) + +so that Package B doesn't have to remove the ``[PDF]`` from its requirement +specifier. + + +Including Data Files +==================== + +The distutils have traditionally allowed installation of "data files", which +are placed in a platform-specific location. However, the most common use case +for data files distributed with a package is for use *by* the package, usually +by including the data files in the package directory. + +Setuptools offers three ways to specify data files to be included in your +packages. First, you can simply use the ``include_package_data`` keyword, +e.g.:: + + from setuptools import setup, find_packages + setup( + ... + include_package_data = True + ) + +This tells setuptools to install any data files it finds in your packages. +The data files must be under CVS or Subversion control, or else they must be +specified via the distutils' ``MANIFEST.in`` file. (They can also be tracked +by another revision control system, using an appropriate plugin. See the +section below on `Adding Support for Other Revision Control Systems`_ for +information on how to write such plugins.) + +If the data files are not under version control, or are not in a supported +version control system, or if you want finer-grained control over what files +are included (for example, if you have documentation files in your package +directories and want to exclude them from installation), then you can also use +the ``package_data`` keyword, e.g.:: + + from setuptools import setup, find_packages + setup( + ... + package_data = { + # If any package contains *.txt or *.rst files, include them: + '': ['*.txt', '*.rst'], + # And include any *.msg files found in the 'hello' package, too: + 'hello': ['*.msg'], + } + ) + +The ``package_data`` argument is a dictionary that maps from package names to +lists of glob patterns. The globs may include subdirectory names, if the data +files are contained in a subdirectory of the package. For example, if the +package tree looks like this:: + + setup.py + src/ + mypkg/ + __init__.py + mypkg.txt + data/ + somefile.dat + otherdata.dat + +The setuptools setup file might look like this:: + + from setuptools import setup, find_packages + setup( + ... + packages = find_packages('src'), # include all packages under src + package_dir = {'':'src'}, # tell distutils packages are under src + + package_data = { + # If any package contains *.txt files, include them: + '': ['*.txt'], + # And include any *.dat files found in the 'data' subdirectory + # of the 'mypkg' package, also: + 'mypkg': ['data/*.dat'], + } + ) + +Notice that if you list patterns in ``package_data`` under the empty string, +these patterns are used to find files in every package, even ones that also +have their own patterns listed. Thus, in the above example, the ``mypkg.txt`` +file gets included even though it's not listed in the patterns for ``mypkg``. + +Also notice that if you use paths, you *must* use a forward slash (``/``) as +the path separator, even if you are on Windows. Setuptools automatically +converts slashes to appropriate platform-specific separators at build time. + +(Note: although the ``package_data`` argument was previously only available in +``setuptools``, it was also added to the Python ``distutils`` package as of +Python 2.4; there is `some documentation for the feature`__ available on the +python.org website. If using the setuptools-specific ``include_package_data`` +argument, files specified by ``package_data`` will *not* be automatically +added to the manifest unless they are tracked by a supported version control +system, or are listed in the MANIFEST.in file.) + +__ http://docs.python.org/dist/node11.html + +Sometimes, the ``include_package_data`` or ``package_data`` options alone +aren't sufficient to precisely define what files you want included. For +example, you may want to include package README files in your revision control +system and source distributions, but exclude them from being installed. So, +setuptools offers an ``exclude_package_data`` option as well, that allows you +to do things like this:: + + from setuptools import setup, find_packages + setup( + ... + packages = find_packages('src'), # include all packages under src + package_dir = {'':'src'}, # tell distutils packages are under src + + include_package_data = True, # include everything in source control + + # ...but exclude README.txt from all packages + exclude_package_data = { '': ['README.txt'] }, + ) + +The ``exclude_package_data`` option is a dictionary mapping package names to +lists of wildcard patterns, just like the ``package_data`` option. And, just +as with that option, a key of ``''`` will apply the given pattern(s) to all +packages. However, any files that match these patterns will be *excluded* +from installation, even if they were listed in ``package_data`` or were +included as a result of using ``include_package_data``. + +In summary, the three options allow you to: + +``include_package_data`` + Accept all data files and directories matched by ``MANIFEST.in`` or found + in source control. + +``package_data`` + Specify additional patterns to match files and directories that may or may + not be matched by ``MANIFEST.in`` or found in source control. + +``exclude_package_data`` + Specify patterns for data files and directories that should *not* be + included when a package is installed, even if they would otherwise have + been included due to the use of the preceding options. + +NOTE: Due to the way the distutils build process works, a data file that you +include in your project and then stop including may be "orphaned" in your +project's build directories, requiring you to run ``setup.py clean --all`` to +fully remove them. This may also be important for your users and contributors +if they track intermediate revisions of your project using Subversion; be sure +to let them know when you make changes that remove files from inclusion so they +can run ``setup.py clean --all``. + + +Accessing Data Files at Runtime +------------------------------- + +Typically, existing programs manipulate a package's ``__file__`` attribute in +order to find the location of data files. However, this manipulation isn't +compatible with PEP 302-based import hooks, including importing from zip files +and Python Eggs. It is strongly recommended that, if you are using data files, +you should use the `Resource Management API`_ of ``pkg_resources`` to access +them. The ``pkg_resources`` module is distributed as part of setuptools, so if +you're using setuptools to distribute your package, there is no reason not to +use its resource management API. See also `Accessing Package Resources`_ for +a quick example of converting code that uses ``__file__`` to use +``pkg_resources`` instead. + +.. _Resource Management API: http://peak.telecommunity.com/DevCenter/PythonEggs#resource-management +.. _Accessing Package Resources: http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources + + +Non-Package Data Files +---------------------- + +The ``distutils`` normally install general "data files" to a platform-specific +location (e.g. ``/usr/share``). This feature intended to be used for things +like documentation, example configuration files, and the like. ``setuptools`` +does not install these data files in a separate location, however. They are +bundled inside the egg file or directory, alongside the Python modules and +packages. The data files can also be accessed using the `Resource Management +API`_, by specifying a ``Requirement`` instead of a package name:: + + from pkg_resources import Requirement, resource_filename + filename = resource_filename(Requirement.parse("MyProject"),"sample.conf") + +The above code will obtain the filename of the "sample.conf" file in the data +root of the "MyProject" distribution. + +Note, by the way, that this encapsulation of data files means that you can't +actually install data files to some arbitrary location on a user's machine; +this is a feature, not a bug. You can always include a script in your +distribution that extracts and copies your the documentation or data files to +a user-specified location, at their discretion. If you put related data files +in a single directory, you can use ``resource_filename()`` with the directory +name to get a filesystem directory that then can be copied with the ``shutil`` +module. (Even if your package is installed as a zipfile, calling +``resource_filename()`` on a directory will return an actual filesystem +directory, whose contents will be that entire subtree of your distribution.) + +(Of course, if you're writing a new package, you can just as easily place your +data files or directories inside one of your packages, rather than using the +distutils' approach. However, if you're updating an existing application, it +may be simpler not to change the way it currently specifies these data files.) + + +Automatic Resource Extraction +----------------------------- + +If you are using tools that expect your resources to be "real" files, or your +project includes non-extension native libraries or other files that your C +extensions expect to be able to access, you may need to list those files in +the ``eager_resources`` argument to ``setup()``, so that the files will be +extracted together, whenever a C extension in the project is imported. + +This is especially important if your project includes shared libraries *other* +than distutils-built C extensions, and those shared libraries use file +extensions other than ``.dll``, ``.so``, or ``.dylib``, which are the +extensions that setuptools 0.6a8 and higher automatically detects as shared +libraries and adds to the ``native_libs.txt`` file for you. Any shared +libraries whose names do not end with one of those extensions should be listed +as ``eager_resources``, because they need to be present in the filesystem when +he C extensions that link to them are used. + +The ``pkg_resources`` runtime for compressed packages will automatically +extract *all* C extensions and ``eager_resources`` at the same time, whenever +*any* C extension or eager resource is requested via the ``resource_filename()`` +API. (C extensions are imported using ``resource_filename()`` internally.) +This ensures that C extensions will see all of the "real" files that they +expect to see. + +Note also that you can list directory resource names in ``eager_resources`` as +well, in which case the directory's contents (including subdirectories) will be +extracted whenever any C extension or eager resource is requested. + +Please note that if you're not sure whether you need to use this argument, you +don't! It's really intended to support projects with lots of non-Python +dependencies and as a last resort for crufty projects that can't otherwise +handle being compressed. If your package is pure Python, Python plus data +files, or Python plus C, you really don't need this. You've got to be using +either C or an external program that needs "real" files in your project before +there's any possibility of ``eager_resources`` being relevant to your project. + + +Extensible Applications and Frameworks +====================================== + + +.. _Entry Points: + +Dynamic Discovery of Services and Plugins +----------------------------------------- + +``setuptools`` supports creating libraries that "plug in" to extensible +applications and frameworks, by letting you register "entry points" in your +project that can be imported by the application or framework. + +For example, suppose that a blogging tool wants to support plugins +that provide translation for various file types to the blog's output format. +The framework might define an "entry point group" called ``blogtool.parsers``, +and then allow plugins to register entry points for the file extensions they +support. + +This would allow people to create distributions that contain one or more +parsers for different file types, and then the blogging tool would be able to +find the parsers at runtime by looking up an entry point for the file +extension (or mime type, or however it wants to). + +Note that if the blogging tool includes parsers for certain file formats, it +can register these as entry points in its own setup script, which means it +doesn't have to special-case its built-in formats. They can just be treated +the same as any other plugin's entry points would be. + +If you're creating a project that plugs in to an existing application or +framework, you'll need to know what entry points or entry point groups are +defined by that application or framework. Then, you can register entry points +in your setup script. Here are a few examples of ways you might register an +``.rst`` file parser entry point in the ``blogtool.parsers`` entry point group, +for our hypothetical blogging tool:: + + setup( + # ... + entry_points = {'blogtool.parsers': '.rst = some_module:SomeClass'} + ) + + setup( + # ... + entry_points = {'blogtool.parsers': ['.rst = some_module:a_func']} + ) + + setup( + # ... + entry_points = """ + [blogtool.parsers] + .rst = some.nested.module:SomeClass.some_classmethod [reST] + """, + extras_require = dict(reST = "Docutils>=0.3.5") + ) + +The ``entry_points`` argument to ``setup()`` accepts either a string with +``.ini``-style sections, or a dictionary mapping entry point group names to +either strings or lists of strings containing entry point specifiers. An +entry point specifier consists of a name and value, separated by an ``=`` +sign. The value consists of a dotted module name, optionally followed by a +``:`` and a dotted identifier naming an object within the module. It can +also include a bracketed list of "extras" that are required for the entry +point to be used. When the invoking application or framework requests loading +of an entry point, any requirements implied by the associated extras will be +passed to ``pkg_resources.require()``, so that an appropriate error message +can be displayed if the needed package(s) are missing. (Of course, the +invoking app or framework can ignore such errors if it wants to make an entry +point optional if a requirement isn't installed.) + + +Defining Additional Metadata +---------------------------- + +Some extensible applications and frameworks may need to define their own kinds +of metadata to include in eggs, which they can then access using the +``pkg_resources`` metadata APIs. Ordinarily, this is done by having plugin +developers include additional files in their ``ProjectName.egg-info`` +directory. However, since it can be tedious to create such files by hand, you +may want to create a distutils extension that will create the necessary files +from arguments to ``setup()``, in much the same way that ``setuptools`` does +for many of the ``setup()`` arguments it adds. See the section below on +`Creating distutils Extensions`_ for more details, especially the subsection on +`Adding new EGG-INFO Files`_. + + +"Development Mode" +================== + +Under normal circumstances, the ``distutils`` assume that you are going to +build a distribution of your project, not use it in its "raw" or "unbuilt" +form. If you were to use the ``distutils`` that way, you would have to rebuild +and reinstall your project every time you made a change to it during +development. + +Another problem that sometimes comes up with the ``distutils`` is that you may +need to do development on two related projects at the same time. You may need +to put both projects' packages in the same directory to run them, but need to +keep them separate for revision control purposes. How can you do this? + +Setuptools allows you to deploy your projects for use in a common directory or +staging area, but without copying any files. Thus, you can edit each project's +code in its checkout directory, and only need to run build commands when you +change a project's C extensions or similarly compiled files. You can even +deploy a project into another project's checkout directory, if that's your +preferred way of working (as opposed to using a common independent staging area +or the site-packages directory). + +To do this, use the ``setup.py develop`` command. It works very similarly to +``setup.py install`` or the EasyInstall tool, except that it doesn't actually +install anything. Instead, it creates a special ``.egg-link`` file in the +deployment directory, that links to your project's source code. And, if your +deployment directory is Python's ``site-packages`` directory, it will also +update the ``easy-install.pth`` file to include your project's source code, +thereby making it available on ``sys.path`` for all programs using that Python +installation. + +If you have enabled the ``use_2to3`` flag, then of course the ``.egg-link`` +will not link directly to your source code when run under Python 3, since +that source code would be made for Python 2 and not work under Python 3. +Instead the ``setup.py develop`` will build Python 3 code under the ``build`` +directory, and link there. This means that after doing code changes you will +have to run ``setup.py build`` before these changes are picked up by your +Python 3 installation. + +In addition, the ``develop`` command creates wrapper scripts in the target +script directory that will run your in-development scripts after ensuring that +all your ``install_requires`` packages are available on ``sys.path``. + +You can deploy the same project to multiple staging areas, e.g. if you have +multiple projects on the same machine that are sharing the same project you're +doing development work. + +When you're done with a given development task, you can remove the project +source from a staging area using ``setup.py develop --uninstall``, specifying +the desired staging area if it's not the default. + +There are several options to control the precise behavior of the ``develop`` +command; see the section on the `develop`_ command below for more details. + +Note that you can also apply setuptools commands to non-setuptools projects, +using commands like this:: + + python -c "import setuptools; execfile('setup.py')" develop + +That is, you can simply list the normal setup commands and options following +the quoted part. + + +Distributing a ``setuptools``-based project +=========================================== + +Using ``setuptools``... Without bundling it! +--------------------------------------------- + +Your users might not have ``setuptools`` installed on their machines, or even +if they do, it might not be the right version. Fixing this is easy; just +download `distribute_setup.py`_, and put it in the same directory as your ``setup.py`` +script. (Be sure to add it to your revision control system, too.) Then add +these two lines to the very top of your setup script, before the script imports +anything from setuptools: + +.. code-block:: python + + import distribute_setup + distribute_setup.use_setuptools() + +That's it. The ``distribute_setup`` module will automatically download a matching +version of ``setuptools`` from PyPI, if it isn't present on the target system. +Whenever you install an updated version of setuptools, you should also update +your projects' ``distribute_setup.py`` files, so that a matching version gets installed +on the target machine(s). + +By the way, setuptools supports the new PyPI "upload" command, so you can use +``setup.py sdist upload`` or ``setup.py bdist_egg upload`` to upload your +source or egg distributions respectively. Your project's current version must +be registered with PyPI first, of course; you can use ``setup.py register`` to +do that. Or you can do it all in one step, e.g. ``setup.py register sdist +bdist_egg upload`` will register the package, build source and egg +distributions, and then upload them both to PyPI, where they'll be easily +found by other projects that depend on them. + +(By the way, if you need to distribute a specific version of ``setuptools``, +you can specify the exact version and base download URL as parameters to the +``use_setuptools()`` function. See the function's docstring for details.) + + +What Your Users Should Know +--------------------------- + +In general, a setuptools-based project looks just like any distutils-based +project -- as long as your users have an internet connection and are installing +to ``site-packages``, that is. But for some users, these conditions don't +apply, and they may become frustrated if this is their first encounter with +a setuptools-based project. To keep these users happy, you should review the +following topics in your project's installation instructions, if they are +relevant to your project and your target audience isn't already familiar with +setuptools and ``easy_install``. + +Network Access + If your project is using ``distribute_setup``, you should inform users of the + need to either have network access, or to preinstall the correct version of + setuptools using the `EasyInstall installation instructions`_. Those + instructions also have tips for dealing with firewalls as well as how to + manually download and install setuptools. + +Custom Installation Locations + You should inform your users that if they are installing your project to + somewhere other than the main ``site-packages`` directory, they should + first install setuptools using the instructions for `Custom Installation + Locations`_, before installing your project. + +Your Project's Dependencies + If your project depends on other projects that may need to be downloaded + from PyPI or elsewhere, you should list them in your installation + instructions, or tell users how to find out what they are. While most + users will not need this information, any users who don't have unrestricted + internet access may have to find, download, and install the other projects + manually. (Note, however, that they must still install those projects + using ``easy_install``, or your project will not know they are installed, + and your setup script will try to download them again.) + + If you want to be especially friendly to users with limited network access, + you may wish to build eggs for your project and its dependencies, making + them all available for download from your site, or at least create a page + with links to all of the needed eggs. In this way, users with limited + network access can manually download all the eggs to a single directory, + then use the ``-f`` option of ``easy_install`` to specify the directory + to find eggs in. Users who have full network access can just use ``-f`` + with the URL of your download page, and ``easy_install`` will find all the + needed eggs using your links directly. This is also useful when your + target audience isn't able to compile packages (e.g. most Windows users) + and your package or some of its dependencies include C code. + +Subversion or CVS Users and Co-Developers + Users and co-developers who are tracking your in-development code using + CVS, Subversion, or some other revision control system should probably read + this manual's sections regarding such development. Alternately, you may + wish to create a quick-reference guide containing the tips from this manual + that apply to your particular situation. For example, if you recommend + that people use ``setup.py develop`` when tracking your in-development + code, you should let them know that this needs to be run after every update + or commit. + + Similarly, if you remove modules or data files from your project, you + should remind them to run ``setup.py clean --all`` and delete any obsolete + ``.pyc`` or ``.pyo``. (This tip applies to the distutils in general, not + just setuptools, but not everybody knows about them; be kind to your users + by spelling out your project's best practices rather than leaving them + guessing.) + +Creating System Packages + Some users want to manage all Python packages using a single package + manager, and sometimes that package manager isn't ``easy_install``! + Setuptools currently supports ``bdist_rpm``, ``bdist_wininst``, and + ``bdist_dumb`` formats for system packaging. If a user has a locally- + installed "bdist" packaging tool that internally uses the distutils + ``install`` command, it should be able to work with ``setuptools``. Some + examples of "bdist" formats that this should work with include the + ``bdist_nsi`` and ``bdist_msi`` formats for Windows. + + However, packaging tools that build binary distributions by running + ``setup.py install`` on the command line or as a subprocess will require + modification to work with setuptools. They should use the + ``--single-version-externally-managed`` option to the ``install`` command, + combined with the standard ``--root`` or ``--record`` options. + See the `install command`_ documentation below for more details. The + ``bdist_deb`` command is an example of a command that currently requires + this kind of patching to work with setuptools. + + If you or your users have a problem building a usable system package for + your project, please report the problem via the mailing list so that + either the "bdist" tool in question or setuptools can be modified to + resolve the issue. + + + +Managing Multiple Projects +-------------------------- + +If you're managing several projects that need to use ``distribute_setup``, and you +are using Subversion as your revision control system, you can use the +"svn:externals" property to share a single copy of ``distribute_setup`` between +projects, so that it will always be up-to-date whenever you check out or update +an individual project, without having to manually update each project to use +a new version. + +However, because Subversion only supports using directories as externals, you +have to turn ``distribute_setup.py`` into ``distribute_setup/__init__.py`` in order +to do this, then create "externals" definitions that map the ``distribute_setup`` +directory into each project. Also, if any of your projects use +``find_packages()`` on their setup directory, you will need to exclude the +resulting ``distribute_setup`` package, to keep it from being included in your +distributions, e.g.:: + + setup( + ... + packages = find_packages(exclude=['distribute_setup']), + ) + +Of course, the ``distribute_setup`` package will still be included in your +packages' source distributions, as it needs to be. + +For your convenience, you may use the following external definition, which will +track the latest version of setuptools:: + + ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup + +You can set this by executing this command in your project directory:: + + svn propedit svn:externals . + +And then adding the line shown above to the file that comes up for editing. + + +Setting the ``zip_safe`` flag +----------------------------- + +For maximum performance, Python packages are best installed as zip files. +Not all packages, however, are capable of running in compressed form, because +they may expect to be able to access either source code or data files as +normal operating system files. So, ``setuptools`` can install your project +as a zipfile or a directory, and its default choice is determined by the +project's ``zip_safe`` flag. + +You can pass a True or False value for the ``zip_safe`` argument to the +``setup()`` function, or you can omit it. If you omit it, the ``bdist_egg`` +command will analyze your project's contents to see if it can detect any +conditions that would prevent it from working in a zipfile. It will output +notices to the console about any such conditions that it finds. + +Currently, this analysis is extremely conservative: it will consider the +project unsafe if it contains any C extensions or datafiles whatsoever. This +does *not* mean that the project can't or won't work as a zipfile! It just +means that the ``bdist_egg`` authors aren't yet comfortable asserting that +the project *will* work. If the project contains no C or data files, and does +no ``__file__`` or ``__path__`` introspection or source code manipulation, then +there is an extremely solid chance the project will work when installed as a +zipfile. (And if the project uses ``pkg_resources`` for all its data file +access, then C extensions and other data files shouldn't be a problem at all. +See the `Accessing Data Files at Runtime`_ section above for more information.) + +However, if ``bdist_egg`` can't be *sure* that your package will work, but +you've checked over all the warnings it issued, and you are either satisfied it +*will* work (or if you want to try it for yourself), then you should set +``zip_safe`` to ``True`` in your ``setup()`` call. If it turns out that it +doesn't work, you can always change it to ``False``, which will force +``setuptools`` to install your project as a directory rather than as a zipfile. + +Of course, the end-user can still override either decision, if they are using +EasyInstall to install your package. And, if you want to override for testing +purposes, you can just run ``setup.py easy_install --zip-ok .`` or ``setup.py +easy_install --always-unzip .`` in your project directory. to install the +package as a zipfile or directory, respectively. + +In the future, as we gain more experience with different packages and become +more satisfied with the robustness of the ``pkg_resources`` runtime, the +"zip safety" analysis may become less conservative. However, we strongly +recommend that you determine for yourself whether your project functions +correctly when installed as a zipfile, correct any problems if you can, and +then make an explicit declaration of ``True`` or ``False`` for the ``zip_safe`` +flag, so that it will not be necessary for ``bdist_egg`` or ``EasyInstall`` to +try to guess whether your project can work as a zipfile. + + +Namespace Packages +------------------ + +Sometimes, a large package is more useful if distributed as a collection of +smaller eggs. However, Python does not normally allow the contents of a +package to be retrieved from more than one location. "Namespace packages" +are a solution for this problem. When you declare a package to be a namespace +package, it means that the package has no meaningful contents in its +``__init__.py``, and that it is merely a container for modules and subpackages. + +The ``pkg_resources`` runtime will then automatically ensure that the contents +of namespace packages that are spread over multiple eggs or directories are +combined into a single "virtual" package. + +The ``namespace_packages`` argument to ``setup()`` lets you declare your +project's namespace packages, so that they will be included in your project's +metadata. The argument should list the namespace packages that the egg +participates in. For example, the ZopeInterface project might do this:: + + setup( + # ... + namespace_packages = ['zope'] + ) + +because it contains a ``zope.interface`` package that lives in the ``zope`` +namespace package. Similarly, a project for a standalone ``zope.publisher`` +would also declare the ``zope`` namespace package. When these projects are +installed and used, Python will see them both as part of a "virtual" ``zope`` +package, even though they will be installed in different locations. + +Namespace packages don't have to be top-level packages. For example, Zope 3's +``zope.app`` package is a namespace package, and in the future PEAK's +``peak.util`` package will be too. + +Note, by the way, that your project's source tree must include the namespace +packages' ``__init__.py`` files (and the ``__init__.py`` of any parent +packages), in a normal Python package layout. These ``__init__.py`` files +*must* contain the line:: + + __import__('pkg_resources').declare_namespace(__name__) + +This code ensures that the namespace package machinery is operating and that +the current package is registered as a namespace package. + +You must NOT include any other code and data in a namespace package's +``__init__.py``. Even though it may appear to work during development, or when +projects are installed as ``.egg`` files, it will not work when the projects +are installed using "system" packaging tools -- in such cases the +``__init__.py`` files will not be installed, let alone executed. + +You must include the ``declare_namespace()`` line in the ``__init__.py`` of +*every* project that has contents for the namespace package in question, in +order to ensure that the namespace will be declared regardless of which +project's copy of ``__init__.py`` is loaded first. If the first loaded +``__init__.py`` doesn't declare it, it will never *be* declared, because no +other copies will ever be loaded!) + + +TRANSITIONAL NOTE +~~~~~~~~~~~~~~~~~ + +Setuptools 0.6a automatically calls ``declare_namespace()`` for you at runtime, +but the 0.7a versions will *not*. This is because the automatic declaration +feature has some negative side effects, such as needing to import all namespace +packages during the initialization of the ``pkg_resources`` runtime, and also +the need for ``pkg_resources`` to be explicitly imported before any namespace +packages work at all. Beginning with the 0.7a releases, you'll be responsible +for including your own declaration lines, and the automatic declaration feature +will be dropped to get rid of the negative side effects. + +During the remainder of the 0.6 development cycle, therefore, setuptools will +warn you about missing ``declare_namespace()`` calls in your ``__init__.py`` +files, and you should correct these as soon as possible before setuptools 0.7a1 +is released. Namespace packages without declaration lines will not work +correctly once a user has upgraded to setuptools 0.7a1, so it's important that +you make this change now in order to avoid having your code break in the field. +Our apologies for the inconvenience, and thank you for your patience. + + + +Tagging and "Daily Build" or "Snapshot" Releases +------------------------------------------------ + +When a set of related projects are under development, it may be important to +track finer-grained version increments than you would normally use for e.g. +"stable" releases. While stable releases might be measured in dotted numbers +with alpha/beta/etc. status codes, development versions of a project often +need to be tracked by revision or build number or even build date. This is +especially true when projects in development need to refer to one another, and +therefore may literally need an up-to-the-minute version of something! + +To support these scenarios, ``setuptools`` allows you to "tag" your source and +egg distributions by adding one or more of the following to the project's +"official" version identifier: + +* A manually-specified pre-release tag, such as "build" or "dev", or a + manually-specified post-release tag, such as a build or revision number + (``--tag-build=STRING, -bSTRING``) + +* A "last-modified revision number" string generated automatically from + Subversion's metadata (assuming your project is being built from a Subversion + "working copy") (``--tag-svn-revision, -r``) + +* An 8-character representation of the build date (``--tag-date, -d``), as + a postrelease tag + +You can add these tags by adding ``egg_info`` and the desired options to +the command line ahead of the ``sdist`` or ``bdist`` commands that you want +to generate a daily build or snapshot for. See the section below on the +`egg_info`_ command for more details. + +(Also, before you release your project, be sure to see the section above on +`Specifying Your Project's Version`_ for more information about how pre- and +post-release tags affect how setuptools and EasyInstall interpret version +numbers. This is important in order to make sure that dependency processing +tools will know which versions of your project are newer than others.) + +Finally, if you are creating builds frequently, and either building them in a +downloadable location or are copying them to a distribution server, you should +probably also check out the `rotate`_ command, which lets you automatically +delete all but the N most-recently-modified distributions matching a glob +pattern. So, you can use a command line like:: + + setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3 + +to build an egg whose version info includes 'DEV-rNNNN' (where NNNN is the +most recent Subversion revision that affected the source tree), and then +delete any egg files from the distribution directory except for the three +that were built most recently. + +If you have to manage automated builds for multiple packages, each with +different tagging and rotation policies, you may also want to check out the +`alias`_ command, which would let each package define an alias like ``daily`` +that would perform the necessary tag, build, and rotate commands. Then, a +simpler script or cron job could just run ``setup.py daily`` in each project +directory. (And, you could also define sitewide or per-user default versions +of the ``daily`` alias, so that projects that didn't define their own would +use the appropriate defaults.) + + +Generating Source Distributions +------------------------------- + +``setuptools`` enhances the distutils' default algorithm for source file +selection, so that all files managed by CVS or Subversion in your project tree +are included in any source distribution you build. This is a big improvement +over having to manually write a ``MANIFEST.in`` file and try to keep it in +sync with your project. So, if you are using CVS or Subversion, and your +source distributions only need to include files that you're tracking in +revision control, don't create a a ``MANIFEST.in`` file for your project. +(And, if you already have one, you might consider deleting it the next time +you would otherwise have to change it.) + +(NOTE: other revision control systems besides CVS and Subversion can be +supported using plugins; see the section below on `Adding Support for Other +Revision Control Systems`_ for information on how to write such plugins.) + +If you need to include automatically generated files, or files that are kept in +an unsupported revision control system, you'll need to create a ``MANIFEST.in`` +file to specify any files that the default file location algorithm doesn't +catch. See the distutils documentation for more information on the format of +the ``MANIFEST.in`` file. + +But, be sure to ignore any part of the distutils documentation that deals with +``MANIFEST`` or how it's generated from ``MANIFEST.in``; setuptools shields you +from these issues and doesn't work the same way in any case. Unlike the +distutils, setuptools regenerates the source distribution manifest file +every time you build a source distribution, and it builds it inside the +project's ``.egg-info`` directory, out of the way of your main project +directory. You therefore need not worry about whether it is up-to-date or not. + +Indeed, because setuptools' approach to determining the contents of a source +distribution is so much simpler, its ``sdist`` command omits nearly all of +the options that the distutils' more complex ``sdist`` process requires. For +all practical purposes, you'll probably use only the ``--formats`` option, if +you use any option at all. + +(By the way, if you're using some other revision control system, you might +consider creating and publishing a `revision control plugin for setuptools`_.) + + +.. _revision control plugin for setuptools: `Adding Support for Other Revision Control Systems`_ + + +Making your package available for EasyInstall +--------------------------------------------- + +If you use the ``register`` command (``setup.py register``) to register your +package with PyPI, that's most of the battle right there. (See the +`docs for the register command`_ for more details.) + +.. _docs for the register command: http://docs.python.org/dist/package-index.html + +If you also use the `upload`_ command to upload actual distributions of your +package, that's even better, because EasyInstall will be able to find and +download them directly from your project's PyPI page. + +However, there may be reasons why you don't want to upload distributions to +PyPI, and just want your existing distributions (or perhaps a Subversion +checkout) to be used instead. + +So here's what you need to do before running the ``register`` command. There +are three ``setup()`` arguments that affect EasyInstall: + +``url`` and ``download_url`` + These become links on your project's PyPI page. EasyInstall will examine + them to see if they link to a package ("primary links"), or whether they are + HTML pages. If they're HTML pages, EasyInstall scans all HREF's on the + page for primary links + +``long_description`` + EasyInstall will check any URLs contained in this argument to see if they + are primary links. + +A URL is considered a "primary link" if it is a link to a .tar.gz, .tgz, .zip, +.egg, .egg.zip, .tar.bz2, or .exe file, or if it has an ``#egg=project`` or +``#egg=project-version`` fragment identifier attached to it. EasyInstall +attempts to determine a project name and optional version number from the text +of a primary link *without* downloading it. When it has found all the primary +links, EasyInstall will select the best match based on requested version, +platform compatibility, and other criteria. + +So, if your ``url`` or ``download_url`` point either directly to a downloadable +source distribution, or to HTML page(s) that have direct links to such, then +EasyInstall will be able to locate downloads automatically. If you want to +make Subversion checkouts available, then you should create links with either +``#egg=project`` or ``#egg=project-version`` added to the URL. You should +replace ``project`` and ``version`` with the values they would have in an egg +filename. (Be sure to actually generate an egg and then use the initial part +of the filename, rather than trying to guess what the escaped form of the +project name and version number will be.) + +Note that Subversion checkout links are of lower precedence than other kinds +of distributions, so EasyInstall will not select a Subversion checkout for +downloading unless it has a version included in the ``#egg=`` suffix, and +it's a higher version than EasyInstall has seen in any other links for your +project. + +As a result, it's a common practice to use mark checkout URLs with a version of +"dev" (i.e., ``#egg=projectname-dev``), so that users can do something like +this:: + + easy_install --editable projectname==dev + +in order to check out the in-development version of ``projectname``. + + +Managing "Continuous Releases" Using Subversion +----------------------------------------------- + +If you expect your users to track in-development versions of your project via +Subversion, there are a few additional steps you should take to ensure that +things work smoothly with EasyInstall. First, you should add the following +to your project's ``setup.cfg`` file: + +.. code-block:: ini + + [egg_info] + tag_build = .dev + tag_svn_revision = 1 + +This will tell ``setuptools`` to generate package version numbers like +``1.0a1.dev-r1263``, which will be considered to be an *older* release than +``1.0a1``. Thus, when you actually release ``1.0a1``, the entire egg +infrastructure (including ``setuptools``, ``pkg_resources`` and EasyInstall) +will know that ``1.0a1`` supersedes any interim snapshots from Subversion, and +handle upgrades accordingly. + +(Note: the project version number you specify in ``setup.py`` should always be +the *next* version of your software, not the last released version. +Alternately, you can leave out the ``tag_build=.dev``, and always use the +*last* release as a version number, so that your post-1.0 builds are labelled +``1.0-r1263``, indicating a post-1.0 patchlevel. Most projects so far, +however, seem to prefer to think of their project as being a future version +still under development, rather than a past version being patched. It is of +course possible for a single project to have both situations, using +post-release numbering on release branches, and pre-release numbering on the +trunk. But you don't have to make things this complex if you don't want to.) + +Commonly, projects releasing code from Subversion will include a PyPI link to +their checkout URL (as described in the previous section) with an +``#egg=projectname-dev`` suffix. This allows users to request EasyInstall +to download ``projectname==dev`` in order to get the latest in-development +code. Note that if your project depends on such in-progress code, you may wish +to specify your ``install_requires`` (or other requirements) to include +``==dev``, e.g.: + +.. code-block:: python + + install_requires = ["OtherProject>=0.2a1.dev-r143,==dev"] + +The above example says, "I really want at least this particular development +revision number, but feel free to follow and use an ``#egg=OtherProject-dev`` +link if you find one". This avoids the need to have actual source or binary +distribution snapshots of in-development code available, just to be able to +depend on the latest and greatest a project has to offer. + +A final note for Subversion development: if you are using SVN revision tags +as described in this section, it's a good idea to run ``setup.py develop`` +after each Subversion checkin or update, because your project's version number +will be changing, and your script wrappers need to be updated accordingly. + +Also, if the project's requirements have changed, the ``develop`` command will +take care of fetching the updated dependencies, building changed extensions, +etc. Be sure to also remind any of your users who check out your project +from Subversion that they need to run ``setup.py develop`` after every update +in order to keep their checkout completely in sync. + + +Making "Official" (Non-Snapshot) Releases +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When you make an official release, creating source or binary distributions, +you will need to override the tag settings from ``setup.cfg``, so that you +don't end up registering versions like ``foobar-0.7a1.dev-r34832``. This is +easy to do if you are developing on the trunk and using tags or branches for +your releases - just make the change to ``setup.cfg`` after branching or +tagging the release, so the trunk will still produce development snapshots. + +Alternately, if you are not branching for releases, you can override the +default version options on the command line, using something like:: + + python setup.py egg_info -RDb "" sdist bdist_egg register upload + +The first part of this command (``egg_info -RDb ""``) will override the +configured tag information, before creating source and binary eggs, registering +the project with PyPI, and uploading the files. Thus, these commands will use +the plain version from your ``setup.py``, without adding the Subversion +revision number or build designation string. + +Of course, if you will be doing this a lot, you may wish to create a personal +alias for this operation, e.g.:: + + python setup.py alias -u release egg_info -RDb "" + +You can then use it like this:: + + python setup.py release sdist bdist_egg register upload + +Or of course you can create more elaborate aliases that do all of the above. +See the sections below on the `egg_info`_ and `alias`_ commands for more ideas. + + + +Distributing Extensions compiled with Pyrex +------------------------------------------- + +``setuptools`` includes transparent support for building Pyrex extensions, as +long as you define your extensions using ``setuptools.Extension``, *not* +``distutils.Extension``. You must also not import anything from Pyrex in +your setup script. + +If you follow these rules, you can safely list ``.pyx`` files as the source +of your ``Extension`` objects in the setup script. ``setuptools`` will detect +at build time whether Pyrex is installed or not. If it is, then ``setuptools`` +will use it. If not, then ``setuptools`` will silently change the +``Extension`` objects to refer to the ``.c`` counterparts of the ``.pyx`` +files, so that the normal distutils C compilation process will occur. + +Of course, for this to work, your source distributions must include the C +code generated by Pyrex, as well as your original ``.pyx`` files. This means +that you will probably want to include current ``.c`` files in your revision +control system, rebuilding them whenever you check changes in for the ``.pyx`` +source files. This will ensure that people tracking your project in CVS or +Subversion will be able to build it even if they don't have Pyrex installed, +and that your source releases will be similarly usable with or without Pyrex. + + +----------------- +Command Reference +----------------- + +.. _alias: + +``alias`` - Define shortcuts for commonly used commands +======================================================= + +Sometimes, you need to use the same commands over and over, but you can't +necessarily set them as defaults. For example, if you produce both development +snapshot releases and "stable" releases of a project, you may want to put +the distributions in different places, or use different ``egg_info`` tagging +options, etc. In these cases, it doesn't make sense to set the options in +a distutils configuration file, because the values of the options changed based +on what you're trying to do. + +Setuptools therefore allows you to define "aliases" - shortcut names for +an arbitrary string of commands and options, using ``setup.py alias aliasname +expansion``, where aliasname is the name of the new alias, and the remainder of +the command line supplies its expansion. For example, this command defines +a sitewide alias called "daily", that sets various ``egg_info`` tagging +options:: + + setup.py alias --global-config daily egg_info --tag-svn-revision \ + --tag-build=development + +Once the alias is defined, it can then be used with other setup commands, +e.g.:: + + setup.py daily bdist_egg # generate a daily-build .egg file + setup.py daily sdist # generate a daily-build source distro + setup.py daily sdist bdist_egg # generate both + +The above commands are interpreted as if the word ``daily`` were replaced with +``egg_info --tag-svn-revision --tag-build=development``. + +Note that setuptools will expand each alias *at most once* in a given command +line. This serves two purposes. First, if you accidentally create an alias +loop, it will have no effect; you'll instead get an error message about an +unknown command. Second, it allows you to define an alias for a command, that +uses that command. For example, this (project-local) alias:: + + setup.py alias bdist_egg bdist_egg rotate -k1 -m.egg + +redefines the ``bdist_egg`` command so that it always runs the ``rotate`` +command afterwards to delete all but the newest egg file. It doesn't loop +indefinitely on ``bdist_egg`` because the alias is only expanded once when +used. + +You can remove a defined alias with the ``--remove`` (or ``-r``) option, e.g.:: + + setup.py alias --global-config --remove daily + +would delete the "daily" alias we defined above. + +Aliases can be defined on a project-specific, per-user, or sitewide basis. The +default is to define or remove a project-specific alias, but you can use any of +the `configuration file options`_ (listed under the `saveopts`_ command, below) +to determine which distutils configuration file an aliases will be added to +(or removed from). + +Note that if you omit the "expansion" argument to the ``alias`` command, +you'll get output showing that alias' current definition (and what +configuration file it's defined in). If you omit the alias name as well, +you'll get a listing of all current aliases along with their configuration +file locations. + + +``bdist_egg`` - Create a Python Egg for the project +=================================================== + +This command generates a Python Egg (``.egg`` file) for the project. Python +Eggs are the preferred binary distribution format for EasyInstall, because they +are cross-platform (for "pure" packages), directly importable, and contain +project metadata including scripts and information about the project's +dependencies. They can be simply downloaded and added to ``sys.path`` +directly, or they can be placed in a directory on ``sys.path`` and then +automatically discovered by the egg runtime system. + +This command runs the `egg_info`_ command (if it hasn't already run) to update +the project's metadata (``.egg-info``) directory. If you have added any extra +metadata files to the ``.egg-info`` directory, those files will be included in +the new egg file's metadata directory, for use by the egg runtime system or by +any applications or frameworks that use that metadata. + +You won't usually need to specify any special options for this command; just +use ``bdist_egg`` and you're done. But there are a few options that may +be occasionally useful: + +``--dist-dir=DIR, -d DIR`` + Set the directory where the ``.egg`` file will be placed. If you don't + supply this, then the ``--dist-dir`` setting of the ``bdist`` command + will be used, which is usually a directory named ``dist`` in the project + directory. + +``--plat-name=PLATFORM, -p PLATFORM`` + Set the platform name string that will be embedded in the egg's filename + (assuming the egg contains C extensions). This can be used to override + the distutils default platform name with something more meaningful. Keep + in mind, however, that the egg runtime system expects to see eggs with + distutils platform names, so it may ignore or reject eggs with non-standard + platform names. Similarly, the EasyInstall program may ignore them when + searching web pages for download links. However, if you are + cross-compiling or doing some other unusual things, you might find a use + for this option. + +``--exclude-source-files`` + Don't include any modules' ``.py`` files in the egg, just compiled Python, + C, and data files. (Note that this doesn't affect any ``.py`` files in the + EGG-INFO directory or its subdirectories, since for example there may be + scripts with a ``.py`` extension which must still be retained.) We don't + recommend that you use this option except for packages that are being + bundled for proprietary end-user applications, or for "embedded" scenarios + where space is at an absolute premium. On the other hand, if your package + is going to be installed and used in compressed form, you might as well + exclude the source because Python's ``traceback`` module doesn't currently + understand how to display zipped source code anyway, or how to deal with + files that are in a different place from where their code was compiled. + +There are also some options you will probably never need, but which are there +because they were copied from similar ``bdist`` commands used as an example for +creating this one. They may be useful for testing and debugging, however, +which is why we kept them: + +``--keep-temp, -k`` + Keep the contents of the ``--bdist-dir`` tree around after creating the + ``.egg`` file. + +``--bdist-dir=DIR, -b DIR`` + Set the temporary directory for creating the distribution. The entire + contents of this directory are zipped to create the ``.egg`` file, after + running various installation commands to copy the package's modules, data, + and extensions here. + +``--skip-build`` + Skip doing any "build" commands; just go straight to the + install-and-compress phases. + + +.. _develop: + +``develop`` - Deploy the project source in "Development Mode" +============================================================= + +This command allows you to deploy your project's source for use in one or more +"staging areas" where it will be available for importing. This deployment is +done in such a way that changes to the project source are immediately available +in the staging area(s), without needing to run a build or install step after +each change. + +The ``develop`` command works by creating an ``.egg-link`` file (named for the +project) in the given staging area. If the staging area is Python's +``site-packages`` directory, it also updates an ``easy-install.pth`` file so +that the project is on ``sys.path`` by default for all programs run using that +Python installation. + +The ``develop`` command also installs wrapper scripts in the staging area (or +a separate directory, as specified) that will ensure the project's dependencies +are available on ``sys.path`` before running the project's source scripts. +And, it ensures that any missing project dependencies are available in the +staging area, by downloading and installing them if necessary. + +Last, but not least, the ``develop`` command invokes the ``build_ext -i`` +command to ensure any C extensions in the project have been built and are +up-to-date, and the ``egg_info`` command to ensure the project's metadata is +updated (so that the runtime and wrappers know what the project's dependencies +are). If you make any changes to the project's setup script or C extensions, +you should rerun the ``develop`` command against all relevant staging areas to +keep the project's scripts, metadata and extensions up-to-date. Most other +kinds of changes to your project should not require any build operations or +rerunning ``develop``, but keep in mind that even minor changes to the setup +script (e.g. changing an entry point definition) require you to re-run the +``develop`` or ``test`` commands to keep the distribution updated. + +Here are some of the options that the ``develop`` command accepts. Note that +they affect the project's dependencies as well as the project itself, so if you +have dependencies that need to be installed and you use ``--exclude-scripts`` +(for example), the dependencies' scripts will not be installed either! For +this reason, you may want to use EasyInstall to install the project's +dependencies before using the ``develop`` command, if you need finer control +over the installation options for dependencies. + +``--uninstall, -u`` + Un-deploy the current project. You may use the ``--install-dir`` or ``-d`` + option to designate the staging area. The created ``.egg-link`` file will + be removed, if present and it is still pointing to the project directory. + The project directory will be removed from ``easy-install.pth`` if the + staging area is Python's ``site-packages`` directory. + + Note that this option currently does *not* uninstall script wrappers! You + must uninstall them yourself, or overwrite them by using EasyInstall to + activate a different version of the package. You can also avoid installing + script wrappers in the first place, if you use the ``--exclude-scripts`` + (aka ``-x``) option when you run ``develop`` to deploy the project. + +``--multi-version, -m`` + "Multi-version" mode. Specifying this option prevents ``develop`` from + adding an ``easy-install.pth`` entry for the project(s) being deployed, and + if an entry for any version of a project already exists, the entry will be + removed upon successful deployment. In multi-version mode, no specific + version of the package is available for importing, unless you use + ``pkg_resources.require()`` to put it on ``sys.path``, or you are running + a wrapper script generated by ``setuptools`` or EasyInstall. (In which + case the wrapper script calls ``require()`` for you.) + + Note that if you install to a directory other than ``site-packages``, + this option is automatically in effect, because ``.pth`` files can only be + used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use + the ``--install-dir`` or ``-d`` option (or they are set via configuration + file(s)) your project and its dependencies will be deployed in multi- + version mode. + +``--install-dir=DIR, -d DIR`` + Set the installation directory (staging area). If this option is not + directly specified on the command line or in a distutils configuration + file, the distutils default installation location is used. Normally, this + will be the ``site-packages`` directory, but if you are using distutils + configuration files, setting things like ``prefix`` or ``install_lib``, + then those settings are taken into account when computing the default + staging area. + +``--script-dir=DIR, -s DIR`` + Set the script installation directory. If you don't supply this option + (via the command line or a configuration file), but you *have* supplied + an ``--install-dir`` (via command line or config file), then this option + defaults to the same directory, so that the scripts will be able to find + their associated package installation. Otherwise, this setting defaults + to the location where the distutils would normally install scripts, taking + any distutils configuration file settings into account. + +``--exclude-scripts, -x`` + Don't deploy script wrappers. This is useful if you don't want to disturb + existing versions of the scripts in the staging area. + +``--always-copy, -a`` + Copy all needed distributions to the staging area, even if they + are already present in another directory on ``sys.path``. By default, if + a requirement can be met using a distribution that is already available in + a directory on ``sys.path``, it will not be copied to the staging area. + +``--egg-path=DIR`` + Force the generated ``.egg-link`` file to use a specified relative path + to the source directory. This can be useful in circumstances where your + installation directory is being shared by code running under multiple + platforms (e.g. Mac and Windows) which have different absolute locations + for the code under development, but the same *relative* locations with + respect to the installation directory. If you use this option when + installing, you must supply the same relative path when uninstalling. + +In addition to the above options, the ``develop`` command also accepts all of +the same options accepted by ``easy_install``. If you've configured any +``easy_install`` settings in your ``setup.cfg`` (or other distutils config +files), the ``develop`` command will use them as defaults, unless you override +them in a ``[develop]`` section or on the command line. + + +``easy_install`` - Find and install packages +============================================ + +This command runs the `EasyInstall tool +<http://peak.telecommunity.com/DevCenter/EasyInstall>`_ for you. It is exactly +equivalent to running the ``easy_install`` command. All command line arguments +following this command are consumed and not processed further by the distutils, +so this must be the last command listed on the command line. Please see +the EasyInstall documentation for the options reference and usage examples. +Normally, there is no reason to use this command via the command line, as you +can just use ``easy_install`` directly. It's only listed here so that you know +it's a distutils command, which means that you can: + +* create command aliases that use it, +* create distutils extensions that invoke it as a subcommand, and +* configure options for it in your ``setup.cfg`` or other distutils config + files. + + +.. _egg_info: + +``egg_info`` - Create egg metadata and set build tags +===================================================== + +This command performs two operations: it updates a project's ``.egg-info`` +metadata directory (used by the ``bdist_egg``, ``develop``, and ``test`` +commands), and it allows you to temporarily change a project's version string, +to support "daily builds" or "snapshot" releases. It is run automatically by +the ``sdist``, ``bdist_egg``, ``develop``, ``register``, and ``test`` commands +in order to update the project's metadata, but you can also specify it +explicitly in order to temporarily change the project's version string while +executing other commands. (It also generates the``.egg-info/SOURCES.txt`` +manifest file, which is used when you are building source distributions.) + +In addition to writing the core egg metadata defined by ``setuptools`` and +required by ``pkg_resources``, this command can be extended to write other +metadata files as well, by defining entry points in the ``egg_info.writers`` +group. See the section on `Adding new EGG-INFO Files`_ below for more details. +Note that using additional metadata writers may require you to include a +``setup_requires`` argument to ``setup()`` in order to ensure that the desired +writers are available on ``sys.path``. + + +Release Tagging Options +----------------------- + +The following options can be used to modify the project's version string for +all remaining commands on the setup command line. The options are processed +in the order shown, so if you use more than one, the requested tags will be +added in the following order: + +``--tag-build=NAME, -b NAME`` + Append NAME to the project's version string. Due to the way setuptools + processes "pre-release" version suffixes beginning with the letters "a" + through "e" (like "alpha", "beta", and "candidate"), you will usually want + to use a tag like ".build" or ".dev", as this will cause the version number + to be considered *lower* than the project's default version. (If you + want to make the version number *higher* than the default version, you can + always leave off --tag-build and then use one or both of the following + options.) + + If you have a default build tag set in your ``setup.cfg``, you can suppress + it on the command line using ``-b ""`` or ``--tag-build=""`` as an argument + to the ``egg_info`` command. + +``--tag-svn-revision, -r`` + If the current directory is a Subversion checkout (i.e. has a ``.svn`` + subdirectory, this appends a string of the form "-rNNNN" to the project's + version string, where NNNN is the revision number of the most recent + modification to the current directory, as obtained from the ``svn info`` + command. + + If the current directory is not a Subversion checkout, the command will + look for a ``PKG-INFO`` file instead, and try to find the revision number + from that, by looking for a "-rNNNN" string at the end of the version + number. (This is so that building a package from a source distribution of + a Subversion snapshot will produce a binary with the correct version + number.) + + If there is no ``PKG-INFO`` file, or the version number contained therein + does not end with ``-r`` and a number, then ``-r0`` is used. + +``--no-svn-revision, -R`` + Don't include the Subversion revision in the version number. This option + is included so you can override a default setting put in ``setup.cfg``. + +``--tag-date, -d`` + Add a date stamp of the form "-YYYYMMDD" (e.g. "-20050528") to the + project's version number. + +``--no-date, -D`` + Don't include a date stamp in the version number. This option is included + so you can override a default setting in ``setup.cfg``. + + +(Note: Because these options modify the version number used for source and +binary distributions of your project, you should first make sure that you know +how the resulting version numbers will be interpreted by automated tools +like EasyInstall. See the section above on `Specifying Your Project's +Version`_ for an explanation of pre- and post-release tags, as well as tips on +how to choose and verify a versioning scheme for your your project.) + +For advanced uses, there is one other option that can be set, to change the +location of the project's ``.egg-info`` directory. Commands that need to find +the project's source directory or metadata should get it from this setting: + + +Other ``egg_info`` Options +-------------------------- + +``--egg-base=SOURCEDIR, -e SOURCEDIR`` + Specify the directory that should contain the .egg-info directory. This + should normally be the root of your project's source tree (which is not + necessarily the same as your project directory; some projects use a ``src`` + or ``lib`` subdirectory as the source root). You should not normally need + to specify this directory, as it is normally determined from the + ``package_dir`` argument to the ``setup()`` function, if any. If there is + no ``package_dir`` set, this option defaults to the current directory. + + +``egg_info`` Examples +--------------------- + +Creating a dated "nightly build" snapshot egg:: + + python setup.py egg_info --tag-date --tag-build=DEV bdist_egg + +Creating and uploading a release with no version tags, even if some default +tags are specified in ``setup.cfg``:: + + python setup.py egg_info -RDb "" sdist bdist_egg register upload + +(Notice that ``egg_info`` must always appear on the command line *before* any +commands that you want the version changes to apply to.) + + +.. _install command: + +``install`` - Run ``easy_install`` or old-style installation +============================================================ + +The setuptools ``install`` command is basically a shortcut to run the +``easy_install`` command on the current project. However, for convenience +in creating "system packages" of setuptools-based projects, you can also +use this option: + +``--single-version-externally-managed`` + This boolean option tells the ``install`` command to perform an "old style" + installation, with the addition of an ``.egg-info`` directory so that the + installed project will still have its metadata available and operate + normally. If you use this option, you *must* also specify the ``--root`` + or ``--record`` options (or both), because otherwise you will have no way + to identify and remove the installed files. + +This option is automatically in effect when ``install`` is invoked by another +distutils command, so that commands like ``bdist_wininst`` and ``bdist_rpm`` +will create system packages of eggs. It is also automatically in effect if +you specify the ``--root`` option. + + +``install_egg_info`` - Install an ``.egg-info`` directory in ``site-packages`` +============================================================================== + +Setuptools runs this command as part of ``install`` operations that use the +``--single-version-externally-managed`` options. You should not invoke it +directly; it is documented here for completeness and so that distutils +extensions such as system package builders can make use of it. This command +has only one option: + +``--install-dir=DIR, -d DIR`` + The parent directory where the ``.egg-info`` directory will be placed. + Defaults to the same as the ``--install-dir`` option specified for the + ``install_lib`` command, which is usually the system ``site-packages`` + directory. + +This command assumes that the ``egg_info`` command has been given valid options +via the command line or ``setup.cfg``, as it will invoke the ``egg_info`` +command and use its options to locate the project's source ``.egg-info`` +directory. + + +.. _rotate: + +``rotate`` - Delete outdated distribution files +=============================================== + +As you develop new versions of your project, your distribution (``dist``) +directory will gradually fill up with older source and/or binary distribution +files. The ``rotate`` command lets you automatically clean these up, keeping +only the N most-recently modified files matching a given pattern. + +``--match=PATTERNLIST, -m PATTERNLIST`` + Comma-separated list of glob patterns to match. This option is *required*. + The project name and ``-*`` is prepended to the supplied patterns, in order + to match only distributions belonging to the current project (in case you + have a shared distribution directory for multiple projects). Typically, + you will use a glob pattern like ``.zip`` or ``.egg`` to match files of + the specified type. Note that each supplied pattern is treated as a + distinct group of files for purposes of selecting files to delete. + +``--keep=COUNT, -k COUNT`` + Number of matching distributions to keep. For each group of files + identified by a pattern specified with the ``--match`` option, delete all + but the COUNT most-recently-modified files in that group. This option is + *required*. + +``--dist-dir=DIR, -d DIR`` + Directory where the distributions are. This defaults to the value of the + ``bdist`` command's ``--dist-dir`` option, which will usually be the + project's ``dist`` subdirectory. + +**Example 1**: Delete all .tar.gz files from the distribution directory, except +for the 3 most recently modified ones:: + + setup.py rotate --match=.tar.gz --keep=3 + +**Example 2**: Delete all Python 2.3 or Python 2.4 eggs from the distribution +directory, except the most recently modified one for each Python version:: + + setup.py rotate --match=-py2.3*.egg,-py2.4*.egg --keep=1 + + +.. _saveopts: + +``saveopts`` - Save used options to a configuration file +======================================================== + +Finding and editing ``distutils`` configuration files can be a pain, especially +since you also have to translate the configuration options from command-line +form to the proper configuration file format. You can avoid these hassles by +using the ``saveopts`` command. Just add it to the command line to save the +options you used. For example, this command builds the project using +the ``mingw32`` C compiler, then saves the --compiler setting as the default +for future builds (even those run implicitly by the ``install`` command):: + + setup.py build --compiler=mingw32 saveopts + +The ``saveopts`` command saves all options for every commmand specified on the +command line to the project's local ``setup.cfg`` file, unless you use one of +the `configuration file options`_ to change where the options are saved. For +example, this command does the same as above, but saves the compiler setting +to the site-wide (global) distutils configuration:: + + setup.py build --compiler=mingw32 saveopts -g + +Note that it doesn't matter where you place the ``saveopts`` command on the +command line; it will still save all the options specified for all commands. +For example, this is another valid way to spell the last example:: + + setup.py saveopts -g build --compiler=mingw32 + +Note, however, that all of the commands specified are always run, regardless of +where ``saveopts`` is placed on the command line. + + +Configuration File Options +-------------------------- + +Normally, settings such as options and aliases are saved to the project's +local ``setup.cfg`` file. But you can override this and save them to the +global or per-user configuration files, or to a manually-specified filename. + +``--global-config, -g`` + Save settings to the global ``distutils.cfg`` file inside the ``distutils`` + package directory. You must have write access to that directory to use + this option. You also can't combine this option with ``-u`` or ``-f``. + +``--user-config, -u`` + Save settings to the current user's ``~/.pydistutils.cfg`` (POSIX) or + ``$HOME/pydistutils.cfg`` (Windows) file. You can't combine this option + with ``-g`` or ``-f``. + +``--filename=FILENAME, -f FILENAME`` + Save settings to the specified configuration file to use. You can't + combine this option with ``-g`` or ``-u``. Note that if you specify a + non-standard filename, the ``distutils`` and ``setuptools`` will not + use the file's contents. This option is mainly included for use in + testing. + +These options are used by other ``setuptools`` commands that modify +configuration files, such as the `alias`_ and `setopt`_ commands. + + +.. _setopt: + +``setopt`` - Set a distutils or setuptools option in a config file +================================================================== + +This command is mainly for use by scripts, but it can also be used as a quick +and dirty way to change a distutils configuration option without having to +remember what file the options are in and then open an editor. + +**Example 1**. Set the default C compiler to ``mingw32`` (using long option +names):: + + setup.py setopt --command=build --option=compiler --set-value=mingw32 + +**Example 2**. Remove any setting for the distutils default package +installation directory (short option names):: + + setup.py setopt -c install -o install_lib -r + + +Options for the ``setopt`` command: + +``--command=COMMAND, -c COMMAND`` + Command to set the option for. This option is required. + +``--option=OPTION, -o OPTION`` + The name of the option to set. This option is required. + +``--set-value=VALUE, -s VALUE`` + The value to set the option to. Not needed if ``-r`` or ``--remove`` is + set. + +``--remove, -r`` + Remove (unset) the option, instead of setting it. + +In addition to the above options, you may use any of the `configuration file +options`_ (listed under the `saveopts`_ command, above) to determine which +distutils configuration file the option will be added to (or removed from). + + +.. _test: + +``test`` - Build package and run a unittest suite +================================================= + +When doing test-driven development, or running automated builds that need +testing before they are deployed for downloading or use, it's often useful +to be able to run a project's unit tests without actually deploying the project +anywhere, even using the ``develop`` command. The ``test`` command runs a +project's unit tests without actually deploying it, by temporarily putting the +project's source on ``sys.path``, after first running ``build_ext -i`` and +``egg_info`` to ensure that any C extensions and project metadata are +up-to-date. + +To use this command, your project's tests must be wrapped in a ``unittest`` +test suite by either a function, a ``TestCase`` class or method, or a module +or package containing ``TestCase`` classes. If the named suite is a module, +and the module has an ``additional_tests()`` function, it is called and the +result (which must be a ``unittest.TestSuite``) is added to the tests to be +run. If the named suite is a package, any submodules and subpackages are +recursively added to the overall test suite. (Note: if your project specifies +a ``test_loader``, the rules for processing the chosen ``test_suite`` may +differ; see the `test_loader`_ documentation for more details.) + +Note that many test systems including ``doctest`` support wrapping their +non-``unittest`` tests in ``TestSuite`` objects. So, if you are using a test +package that does not support this, we suggest you encourage its developers to +implement test suite support, as this is a convenient and standard way to +aggregate a collection of tests to be run under a common test harness. + +By default, tests will be run in the "verbose" mode of the ``unittest`` +package's text test runner, but you can get the "quiet" mode (just dots) if +you supply the ``-q`` or ``--quiet`` option, either as a global option to +the setup script (e.g. ``setup.py -q test``) or as an option for the ``test`` +command itself (e.g. ``setup.py test -q``). There is one other option +available: + +``--test-suite=NAME, -s NAME`` + Specify the test suite (or module, class, or method) to be run + (e.g. ``some_module.test_suite``). The default for this option can be + set by giving a ``test_suite`` argument to the ``setup()`` function, e.g.:: + + setup( + # ... + test_suite = "my_package.tests.test_all" + ) + + If you did not set a ``test_suite`` in your ``setup()`` call, and do not + provide a ``--test-suite`` option, an error will occur. + + +.. _upload: + +``upload`` - Upload source and/or egg distributions to PyPI +=========================================================== + +PyPI now supports uploading project files for redistribution; uploaded files +are easily found by EasyInstall, even if you don't have download links on your +project's home page. + +Although Python 2.5 will support uploading all types of distributions to PyPI, +setuptools only supports source distributions and eggs. (This is partly +because PyPI's upload support is currently broken for various other file +types.) To upload files, you must include the ``upload`` command *after* the +``sdist`` or ``bdist_egg`` commands on the setup command line. For example:: + + setup.py bdist_egg upload # create an egg and upload it + setup.py sdist upload # create a source distro and upload it + setup.py sdist bdist_egg upload # create and upload both + +Note that to upload files for a project, the corresponding version must already +be registered with PyPI, using the distutils ``register`` command. It's +usually a good idea to include the ``register`` command at the start of the +command line, so that any registration problems can be found and fixed before +building and uploading the distributions, e.g.:: + + setup.py register sdist bdist_egg upload + +This will update PyPI's listing for your project's current version. + +Note, by the way, that the metadata in your ``setup()`` call determines what +will be listed in PyPI for your package. Try to fill out as much of it as +possible, as it will save you a lot of trouble manually adding and updating +your PyPI listings. Just put it in ``setup.py`` and use the ``register`` +comamnd to keep PyPI up to date. + +The ``upload`` command has a few options worth noting: + +``--sign, -s`` + Sign each uploaded file using GPG (GNU Privacy Guard). The ``gpg`` program + must be available for execution on the system ``PATH``. + +``--identity=NAME, -i NAME`` + Specify the identity or key name for GPG to use when signing. The value of + this option will be passed through the ``--local-user`` option of the + ``gpg`` program. + +``--show-response`` + Display the full response text from server; this is useful for debugging + PyPI problems. + +``--repository=URL, -r URL`` + The URL of the repository to upload to. Defaults to + http://pypi.python.org/pypi (i.e., the main PyPI installation). + +.. _upload_docs: + +``upload_docs`` - Upload package documentation to PyPI +====================================================== + +PyPI now supports uploading project documentation to the dedicated URL +http://packages.python.org/<project>/. + +The ``upload_docs`` command will create the necessary zip file out of a +documentation directory and will post to the repository. + +Note that to upload the documentation of a project, the corresponding version +must already be registered with PyPI, using the distutils ``register`` +command -- just like the ``upload`` command. + +Assuming there is an ``Example`` project with documentation in the +subdirectory ``docs``, e.g.:: + + Example/ + |-- example.py + |-- setup.cfg + |-- setup.py + |-- docs + | |-- build + | | `-- html + | | | |-- index.html + | | | `-- tips_tricks.html + | |-- conf.py + | |-- index.txt + | `-- tips_tricks.txt + +You can simply pass the documentation directory path to the ``upload_docs`` +command:: + + python setup.py upload_docs --upload-dir=docs/build/html + +If no ``--upload-dir`` is given, ``upload_docs`` will attempt to run the +``build_sphinx`` command to generate uploadable documentation. +For the command to become available, `Sphinx <http://sphinx.pocoo.org/>`_ +must be installed in the same environment as distribute. + +As with other ``setuptools``-based commands, you can define useful +defaults in the ``setup.cfg`` of your Python project, e.g.: + +.. code-block:: ini + + [upload_docs] + upload-dir = docs/build/html + +The ``upload_docs`` command has the following options: + +``--upload-dir`` + The directory to be uploaded to the repository. + +``--show-response`` + Display the full response text from server; this is useful for debugging + PyPI problems. + +``--repository=URL, -r URL`` + The URL of the repository to upload to. Defaults to + http://pypi.python.org/pypi (i.e., the main PyPI installation). + + +-------------------------------- +Extending and Reusing Distribute +-------------------------------- + +Creating ``distutils`` Extensions +================================= + +It can be hard to add new commands or setup arguments to the distutils. But +the ``setuptools`` package makes it a bit easier, by allowing you to distribute +a distutils extension as a separate project, and then have projects that need +the extension just refer to it in their ``setup_requires`` argument. + +With ``setuptools``, your distutils extension projects can hook in new +commands and ``setup()`` arguments just by defining "entry points". These +are mappings from command or argument names to a specification of where to +import a handler from. (See the section on `Dynamic Discovery of Services and +Plugins`_ above for some more background on entry points.) + + +Adding Commands +--------------- + +You can add new ``setup`` commands by defining entry points in the +``distutils.commands`` group. For example, if you wanted to add a ``foo`` +command, you might add something like this to your distutils extension +project's setup script:: + + setup( + # ... + entry_points = { + "distutils.commands": [ + "foo = mypackage.some_module:foo", + ], + }, + ) + +(Assuming, of course, that the ``foo`` class in ``mypackage.some_module`` is +a ``setuptools.Command`` subclass.) + +Once a project containing such entry points has been activated on ``sys.path``, +(e.g. by running "install" or "develop" with a site-packages installation +directory) the command(s) will be available to any ``setuptools``-based setup +scripts. It is not necessary to use the ``--command-packages`` option or +to monkeypatch the ``distutils.command`` package to install your commands; +``setuptools`` automatically adds a wrapper to the distutils to search for +entry points in the active distributions on ``sys.path``. In fact, this is +how setuptools' own commands are installed: the setuptools project's setup +script defines entry points for them! + + +Adding ``setup()`` Arguments +---------------------------- + +Sometimes, your commands may need additional arguments to the ``setup()`` +call. You can enable this by defining entry points in the +``distutils.setup_keywords`` group. For example, if you wanted a ``setup()`` +argument called ``bar_baz``, you might add something like this to your +distutils extension project's setup script:: + + setup( + # ... + entry_points = { + "distutils.commands": [ + "foo = mypackage.some_module:foo", + ], + "distutils.setup_keywords": [ + "bar_baz = mypackage.some_module:validate_bar_baz", + ], + }, + ) + +The idea here is that the entry point defines a function that will be called +to validate the ``setup()`` argument, if it's supplied. The ``Distribution`` +object will have the initial value of the attribute set to ``None``, and the +validation function will only be called if the ``setup()`` call sets it to +a non-None value. Here's an example validation function:: + + def assert_bool(dist, attr, value): + """Verify that value is True, False, 0, or 1""" + if bool(value) != value: + raise DistutilsSetupError( + "%r must be a boolean value (got %r)" % (attr,value) + ) + +Your function should accept three arguments: the ``Distribution`` object, +the attribute name, and the attribute value. It should raise a +``DistutilsSetupError`` (from the ``distutils.errors`` module) if the argument +is invalid. Remember, your function will only be called with non-None values, +and the default value of arguments defined this way is always None. So, your +commands should always be prepared for the possibility that the attribute will +be ``None`` when they access it later. + +If more than one active distribution defines an entry point for the same +``setup()`` argument, *all* of them will be called. This allows multiple +distutils extensions to define a common argument, as long as they agree on +what values of that argument are valid. + +Also note that as with commands, it is not necessary to subclass or monkeypatch +the distutils ``Distribution`` class in order to add your arguments; it is +sufficient to define the entry points in your extension, as long as any setup +script using your extension lists your project in its ``setup_requires`` +argument. + + +Adding new EGG-INFO Files +------------------------- + +Some extensible applications or frameworks may want to allow third parties to +develop plugins with application or framework-specific metadata included in +the plugins' EGG-INFO directory, for easy access via the ``pkg_resources`` +metadata API. The easiest way to allow this is to create a distutils extension +to be used from the plugin projects' setup scripts (via ``setup_requires``) +that defines a new setup keyword, and then uses that data to write an EGG-INFO +file when the ``egg_info`` command is run. + +The ``egg_info`` command looks for extension points in an ``egg_info.writers`` +group, and calls them to write the files. Here's a simple example of a +distutils extension defining a setup argument ``foo_bar``, which is a list of +lines that will be written to ``foo_bar.txt`` in the EGG-INFO directory of any +project that uses the argument:: + + setup( + # ... + entry_points = { + "distutils.setup_keywords": [ + "foo_bar = setuptools.dist:assert_string_list", + ], + "egg_info.writers": [ + "foo_bar.txt = setuptools.command.egg_info:write_arg", + ], + }, + ) + +This simple example makes use of two utility functions defined by setuptools +for its own use: a routine to validate that a setup keyword is a sequence of +strings, and another one that looks up a setup argument and writes it to +a file. Here's what the writer utility looks like:: + + def write_arg(cmd, basename, filename): + argname = os.path.splitext(basename)[0] + value = getattr(cmd.distribution, argname, None) + if value is not None: + value = '\n'.join(value)+'\n' + cmd.write_or_delete_file(argname, filename, value) + +As you can see, ``egg_info.writers`` entry points must be a function taking +three arguments: a ``egg_info`` command instance, the basename of the file to +write (e.g. ``foo_bar.txt``), and the actual full filename that should be +written to. + +In general, writer functions should honor the command object's ``dry_run`` +setting when writing files, and use the ``distutils.log`` object to do any +console output. The easiest way to conform to this requirement is to use +the ``cmd`` object's ``write_file()``, ``delete_file()``, and +``write_or_delete_file()`` methods exclusively for your file operations. See +those methods' docstrings for more details. + + +Adding Support for Other Revision Control Systems +------------------------------------------------- + +If you would like to create a plugin for ``setuptools`` to find files in other +source control systems besides CVS and Subversion, you can do so by adding an +entry point to the ``setuptools.file_finders`` group. The entry point should +be a function accepting a single directory name, and should yield +all the filenames within that directory (and any subdirectories thereof) that +are under revision control. + +For example, if you were going to create a plugin for a revision control system +called "foobar", you would write a function something like this: + +.. code-block:: python + + def find_files_for_foobar(dirname): + # loop to yield paths that start with `dirname` + +And you would register it in a setup script using something like this:: + + entry_points = { + "setuptools.file_finders": [ + "foobar = my_foobar_module:find_files_for_foobar" + ] + } + +Then, anyone who wants to use your plugin can simply install it, and their +local setuptools installation will be able to find the necessary files. + +It is not necessary to distribute source control plugins with projects that +simply use the other source control system, or to specify the plugins in +``setup_requires``. When you create a source distribution with the ``sdist`` +command, setuptools automatically records what files were found in the +``SOURCES.txt`` file. That way, recipients of source distributions don't need +to have revision control at all. However, if someone is working on a package +by checking out with that system, they will need the same plugin(s) that the +original author is using. + +A few important points for writing revision control file finders: + +* Your finder function MUST return relative paths, created by appending to the + passed-in directory name. Absolute paths are NOT allowed, nor are relative + paths that reference a parent directory of the passed-in directory. + +* Your finder function MUST accept an empty string as the directory name, + meaning the current directory. You MUST NOT convert this to a dot; just + yield relative paths. So, yielding a subdirectory named ``some/dir`` under + the current directory should NOT be rendered as ``./some/dir`` or + ``/somewhere/some/dir``, but *always* as simply ``some/dir`` + +* Your finder function SHOULD NOT raise any errors, and SHOULD deal gracefully + with the absence of needed programs (i.e., ones belonging to the revision + control system itself. It *may*, however, use ``distutils.log.warn()`` to + inform the user of the missing program(s). + + +Subclassing ``Command`` +----------------------- + +Sorry, this section isn't written yet, and neither is a lot of what's below +this point, except for the change log. You might want to `subscribe to changes +in this page <setuptools?action=subscribe>`_ to see when new documentation is +added or updated. + +XXX + + +Reusing ``setuptools`` Code +=========================== + +``distribute_setup`` +-------------------- + +XXX + + +``setuptools.archive_util`` +--------------------------- + +XXX + + +``setuptools.sandbox`` +---------------------- + +XXX + + +``setuptools.package_index`` +---------------------------- + +XXX + +History +======= + +0.6c9 + * Fixed a missing files problem when using Windows source distributions on + non-Windows platforms, due to distutils not handling manifest file line + endings correctly. + + * Updated Pyrex support to work with Pyrex 0.9.6 and higher. + + * Minor changes for Jython compatibility, including skipping tests that can't + work on Jython. + + * Fixed not installing eggs in ``install_requires`` if they were also used for + ``setup_requires`` or ``tests_require``. + + * Fixed not fetching eggs in ``install_requires`` when running tests. + + * Allow ``ez_setup.use_setuptools()`` to upgrade existing setuptools + installations when called from a standalone ``setup.py``. + + * Added a warning if a namespace package is declared, but its parent package + is not also declared as a namespace. + + * Support Subversion 1.5 + + * Removed use of deprecated ``md5`` module if ``hashlib`` is available + + * Fixed ``bdist_wininst upload`` trying to upload the ``.exe`` twice + + * Fixed ``bdist_egg`` putting a ``native_libs.txt`` in the source package's + ``.egg-info``, when it should only be in the built egg's ``EGG-INFO``. + + * Ensure that _full_name is set on all shared libs before extensions are + checked for shared lib usage. (Fixes a bug in the experimental shared + library build support.) + + * Fix to allow unpacked eggs containing native libraries to fail more + gracefully under Google App Engine (with an ``ImportError`` loading the + C-based module, instead of getting a ``NameError``). + +0.6c7 + * Fixed ``distutils.filelist.findall()`` crashing on broken symlinks, and + ``egg_info`` command failing on new, uncommitted SVN directories. + + * Fix import problems with nested namespace packages installed via + ``--root`` or ``--single-version-externally-managed``, due to the + parent package not having the child package as an attribute. + +0.6c6 + * Added ``--egg-path`` option to ``develop`` command, allowing you to force + ``.egg-link`` files to use relative paths (allowing them to be shared across + platforms on a networked drive). + + * Fix not building binary RPMs correctly. + + * Fix "eggsecutables" (such as setuptools' own egg) only being runnable with + bash-compatible shells. + + * Fix ``#!`` parsing problems in Windows ``.exe`` script wrappers, when there + was whitespace inside a quoted argument or at the end of the ``#!`` line + (a regression introduced in 0.6c4). + + * Fix ``test`` command possibly failing if an older version of the project + being tested was installed on ``sys.path`` ahead of the test source + directory. + + * Fix ``find_packages()`` treating ``ez_setup`` and directories with ``.`` in + their names as packages. + +0.6c5 + * Fix uploaded ``bdist_rpm`` packages being described as ``bdist_egg`` + packages under Python versions less than 2.5. + + * Fix uploaded ``bdist_wininst`` packages being described as suitable for + "any" version by Python 2.5, even if a ``--target-version`` was specified. + +0.6c4 + * Overhauled Windows script wrapping to support ``bdist_wininst`` better. + Scripts installed with ``bdist_wininst`` will always use ``#!python.exe`` or + ``#!pythonw.exe`` as the executable name (even when built on non-Windows + platforms!), and the wrappers will look for the executable in the script's + parent directory (which should find the right version of Python). + + * Fix ``upload`` command not uploading files built by ``bdist_rpm`` or + ``bdist_wininst`` under Python 2.3 and 2.4. + + * Add support for "eggsecutable" headers: a ``#!/bin/sh`` script that is + prepended to an ``.egg`` file to allow it to be run as a script on Unix-ish + platforms. (This is mainly so that setuptools itself can have a single-file + installer on Unix, without doing multiple downloads, dealing with firewalls, + etc.) + + * Fix problem with empty revision numbers in Subversion 1.4 ``entries`` files + + * Use cross-platform relative paths in ``easy-install.pth`` when doing + ``develop`` and the source directory is a subdirectory of the installation + target directory. + + * Fix a problem installing eggs with a system packaging tool if the project + contained an implicit namespace package; for example if the ``setup()`` + listed a namespace package ``foo.bar`` without explicitly listing ``foo`` + as a namespace package. + +0.6c3 + * Fixed breakages caused by Subversion 1.4's new "working copy" format + +0.6c2 + * The ``ez_setup`` module displays the conflicting version of setuptools (and + its installation location) when a script requests a version that's not + available. + + * Running ``setup.py develop`` on a setuptools-using project will now install + setuptools if needed, instead of only downloading the egg. + +0.6c1 + * Fixed ``AttributeError`` when trying to download a ``setup_requires`` + dependency when a distribution lacks a ``dependency_links`` setting. + + * Made ``zip-safe`` and ``not-zip-safe`` flag files contain a single byte, so + as to play better with packaging tools that complain about zero-length + files. + + * Made ``setup.py develop`` respect the ``--no-deps`` option, which it + previously was ignoring. + + * Support ``extra_path`` option to ``setup()`` when ``install`` is run in + backward-compatibility mode. + + * Source distributions now always include a ``setup.cfg`` file that explicitly + sets ``egg_info`` options such that they produce an identical version number + to the source distribution's version number. (Previously, the default + version number could be different due to the use of ``--tag-date``, or if + the version was overridden on the command line that built the source + distribution.) + +0.6b4 + * Fix ``register`` not obeying name/version set by ``egg_info`` command, if + ``egg_info`` wasn't explicitly run first on the same command line. + + * Added ``--no-date`` and ``--no-svn-revision`` options to ``egg_info`` + command, to allow suppressing tags configured in ``setup.cfg``. + + * Fixed redundant warnings about missing ``README`` file(s); it should now + appear only if you are actually a source distribution. + +0.6b3 + * Fix ``bdist_egg`` not including files in subdirectories of ``.egg-info``. + + * Allow ``.py`` files found by the ``include_package_data`` option to be + automatically included. Remove duplicate data file matches if both + ``include_package_data`` and ``package_data`` are used to refer to the same + files. + +0.6b1 + * Strip ``module`` from the end of compiled extension modules when computing + the name of a ``.py`` loader/wrapper. (Python's import machinery ignores + this suffix when searching for an extension module.) + +0.6a11 + * Added ``test_loader`` keyword to support custom test loaders + + * Added ``setuptools.file_finders`` entry point group to allow implementing + revision control plugins. + + * Added ``--identity`` option to ``upload`` command. + + * Added ``dependency_links`` to allow specifying URLs for ``--find-links``. + + * Enhanced test loader to scan packages as well as modules, and call + ``additional_tests()`` if present to get non-unittest tests. + + * Support namespace packages in conjunction with system packagers, by omitting + the installation of any ``__init__.py`` files for namespace packages, and + adding a special ``.pth`` file to create a working package in + ``sys.modules``. + + * Made ``--single-version-externally-managed`` automatic when ``--root`` is + used, so that most system packagers won't require special support for + setuptools. + + * Fixed ``setup_requires``, ``tests_require``, etc. not using ``setup.cfg`` or + other configuration files for their option defaults when installing, and + also made the install use ``--multi-version`` mode so that the project + directory doesn't need to support .pth files. + + * ``MANIFEST.in`` is now forcibly closed when any errors occur while reading + it. Previously, the file could be left open and the actual error would be + masked by problems trying to remove the open file on Windows systems. + +0.6a10 + * Fixed the ``develop`` command ignoring ``--find-links``. + +0.6a9 + * The ``sdist`` command no longer uses the traditional ``MANIFEST`` file to + create source distributions. ``MANIFEST.in`` is still read and processed, + as are the standard defaults and pruning. But the manifest is built inside + the project's ``.egg-info`` directory as ``SOURCES.txt``, and it is rebuilt + every time the ``egg_info`` command is run. + + * Added the ``include_package_data`` keyword to ``setup()``, allowing you to + automatically include any package data listed in revision control or + ``MANIFEST.in`` + + * Added the ``exclude_package_data`` keyword to ``setup()``, allowing you to + trim back files included via the ``package_data`` and + ``include_package_data`` options. + + * Fixed ``--tag-svn-revision`` not working when run from a source + distribution. + + * Added warning for namespace packages with missing ``declare_namespace()`` + + * Added ``tests_require`` keyword to ``setup()``, so that e.g. packages + requiring ``nose`` to run unit tests can make this dependency optional + unless the ``test`` command is run. + + * Made all commands that use ``easy_install`` respect its configuration + options, as this was causing some problems with ``setup.py install``. + + * Added an ``unpack_directory()`` driver to ``setuptools.archive_util``, so + that you can process a directory tree through a processing filter as if it + were a zipfile or tarfile. + + * Added an internal ``install_egg_info`` command to use as part of old-style + ``install`` operations, that installs an ``.egg-info`` directory with the + package. + + * Added a ``--single-version-externally-managed`` option to the ``install`` + command so that you can more easily wrap a "flat" egg in a system package. + + * Enhanced ``bdist_rpm`` so that it installs single-version eggs that + don't rely on a ``.pth`` file. The ``--no-egg`` option has been removed, + since all RPMs are now built in a more backwards-compatible format. + + * Support full roundtrip translation of eggs to and from ``bdist_wininst`` + format. Running ``bdist_wininst`` on a setuptools-based package wraps the + egg in an .exe that will safely install it as an egg (i.e., with metadata + and entry-point wrapper scripts), and ``easy_install`` can turn the .exe + back into an ``.egg`` file or directory and install it as such. + + +0.6a8 + * Fixed some problems building extensions when Pyrex was installed, especially + with Python 2.4 and/or packages using SWIG. + + * Made ``develop`` command accept all the same options as ``easy_install``, + and use the ``easy_install`` command's configuration settings as defaults. + + * Made ``egg_info --tag-svn-revision`` fall back to extracting the revision + number from ``PKG-INFO`` in case it is being run on a source distribution of + a snapshot taken from a Subversion-based project. + + * Automatically detect ``.dll``, ``.so`` and ``.dylib`` files that are being + installed as data, adding them to ``native_libs.txt`` automatically. + + * Fixed some problems with fresh checkouts of projects that don't include + ``.egg-info/PKG-INFO`` under revision control and put the project's source + code directly in the project directory. If such a package had any + requirements that get processed before the ``egg_info`` command can be run, + the setup scripts would fail with a "Missing 'Version:' header and/or + PKG-INFO file" error, because the egg runtime interpreted the unbuilt + metadata in a directory on ``sys.path`` (i.e. the current directory) as + being a corrupted egg. Setuptools now monkeypatches the distribution + metadata cache to pretend that the egg has valid version information, until + it has a chance to make it actually be so (via the ``egg_info`` command). + +0.6a5 + * Fixed missing gui/cli .exe files in distribution. Fixed bugs in tests. + +0.6a3 + * Added ``gui_scripts`` entry point group to allow installing GUI scripts + on Windows and other platforms. (The special handling is only for Windows; + other platforms are treated the same as for ``console_scripts``.) + +0.6a2 + * Added ``console_scripts`` entry point group to allow installing scripts + without the need to create separate script files. On Windows, console + scripts get an ``.exe`` wrapper so you can just type their name. On other + platforms, the scripts are written without a file extension. + +0.6a1 + * Added support for building "old-style" RPMs that don't install an egg for + the target package, using a ``--no-egg`` option. + + * The ``build_ext`` command now works better when using the ``--inplace`` + option and multiple Python versions. It now makes sure that all extensions + match the current Python version, even if newer copies were built for a + different Python version. + + * The ``upload`` command no longer attaches an extra ``.zip`` when uploading + eggs, as PyPI now supports egg uploads without trickery. + + * The ``ez_setup`` script/module now displays a warning before downloading + the setuptools egg, and attempts to check the downloaded egg against an + internal MD5 checksum table. + + * Fixed the ``--tag-svn-revision`` option of ``egg_info`` not finding the + latest revision number; it was using the revision number of the directory + containing ``setup.py``, not the highest revision number in the project. + + * Added ``eager_resources`` setup argument + + * The ``sdist`` command now recognizes Subversion "deleted file" entries and + does not include them in source distributions. + + * ``setuptools`` now embeds itself more thoroughly into the distutils, so that + other distutils extensions (e.g. py2exe, py2app) will subclass setuptools' + versions of things, rather than the native distutils ones. + + * Added ``entry_points`` and ``setup_requires`` arguments to ``setup()``; + ``setup_requires`` allows you to automatically find and download packages + that are needed in order to *build* your project (as opposed to running it). + + * ``setuptools`` now finds its commands, ``setup()`` argument validators, and + metadata writers using entry points, so that they can be extended by + third-party packages. See `Creating distutils Extensions`_ above for more + details. + + * The vestigial ``depends`` command has been removed. It was never finished + or documented, and never would have worked without EasyInstall - which it + pre-dated and was never compatible with. + +0.5a12 + * The zip-safety scanner now checks for modules that might be used with + ``python -m``, and marks them as unsafe for zipping, since Python 2.4 can't + handle ``-m`` on zipped modules. + +0.5a11 + * Fix breakage of the "develop" command that was caused by the addition of + ``--always-unzip`` to the ``easy_install`` command. + +0.5a9 + * Include ``svn:externals`` directories in source distributions as well as + normal subversion-controlled files and directories. + + * Added ``exclude=patternlist`` option to ``setuptools.find_packages()`` + + * Changed --tag-svn-revision to include an "r" in front of the revision number + for better readability. + + * Added ability to build eggs without including source files (except for any + scripts, of course), using the ``--exclude-source-files`` option to + ``bdist_egg``. + + * ``setup.py install`` now automatically detects when an "unmanaged" package + or module is going to be on ``sys.path`` ahead of a package being installed, + thereby preventing the newer version from being imported. If this occurs, + a warning message is output to ``sys.stderr``, but installation proceeds + anyway. The warning message informs the user what files or directories + need deleting, and advises them they can also use EasyInstall (with the + ``--delete-conflicting`` option) to do it automatically. + + * The ``egg_info`` command now adds a ``top_level.txt`` file to the metadata + directory that lists all top-level modules and packages in the distribution. + This is used by the ``easy_install`` command to find possibly-conflicting + "unmanaged" packages when installing the distribution. + + * Added ``zip_safe`` and ``namespace_packages`` arguments to ``setup()``. + Added package analysis to determine zip-safety if the ``zip_safe`` flag + is not given, and advise the author regarding what code might need changing. + + * Fixed the swapped ``-d`` and ``-b`` options of ``bdist_egg``. + +0.5a8 + * The "egg_info" command now always sets the distribution metadata to "safe" + forms of the distribution name and version, so that distribution files will + be generated with parseable names (i.e., ones that don't include '-' in the + name or version). Also, this means that if you use the various ``--tag`` + options of "egg_info", any distributions generated will use the tags in the + version, not just egg distributions. + + * Added support for defining command aliases in distutils configuration files, + under the "[aliases]" section. To prevent recursion and to allow aliases to + call the command of the same name, a given alias can be expanded only once + per command-line invocation. You can define new aliases with the "alias" + command, either for the local, global, or per-user configuration. + + * Added "rotate" command to delete old distribution files, given a set of + patterns to match and the number of files to keep. (Keeps the most + recently-modified distribution files matching each pattern.) + + * Added "saveopts" command that saves all command-line options for the current + invocation to the local, global, or per-user configuration file. Useful for + setting defaults without having to hand-edit a configuration file. + + * Added a "setopt" command that sets a single option in a specified distutils + configuration file. + +0.5a7 + * Added "upload" support for egg and source distributions, including a bug + fix for "upload" and a temporary workaround for lack of .egg support in + PyPI. + +0.5a6 + * Beefed up the "sdist" command so that if you don't have a MANIFEST.in, it + will include all files under revision control (CVS or Subversion) in the + current directory, and it will regenerate the list every time you create a + source distribution, not just when you tell it to. This should make the + default "do what you mean" more often than the distutils' default behavior + did, while still retaining the old behavior in the presence of MANIFEST.in. + + * Fixed the "develop" command always updating .pth files, even if you + specified ``-n`` or ``--dry-run``. + + * Slightly changed the format of the generated version when you use + ``--tag-build`` on the "egg_info" command, so that you can make tagged + revisions compare *lower* than the version specified in setup.py (e.g. by + using ``--tag-build=dev``). + +0.5a5 + * Added ``develop`` command to ``setuptools``-based packages. This command + installs an ``.egg-link`` pointing to the package's source directory, and + script wrappers that ``execfile()`` the source versions of the package's + scripts. This lets you put your development checkout(s) on sys.path without + having to actually install them. (To uninstall the link, use + use ``setup.py develop --uninstall``.) + + * Added ``egg_info`` command to ``setuptools``-based packages. This command + just creates or updates the "projectname.egg-info" directory, without + building an egg. (It's used by the ``bdist_egg``, ``test``, and ``develop`` + commands.) + + * Enhanced the ``test`` command so that it doesn't install the package, but + instead builds any C extensions in-place, updates the ``.egg-info`` + metadata, adds the source directory to ``sys.path``, and runs the tests + directly on the source. This avoids an "unmanaged" installation of the + package to ``site-packages`` or elsewhere. + + * Made ``easy_install`` a standard ``setuptools`` command, moving it from + the ``easy_install`` module to ``setuptools.command.easy_install``. Note + that if you were importing or extending it, you must now change your imports + accordingly. ``easy_install.py`` is still installed as a script, but not as + a module. + +0.5a4 + * Setup scripts using setuptools can now list their dependencies directly in + the setup.py file, without having to manually create a ``depends.txt`` file. + The ``install_requires`` and ``extras_require`` arguments to ``setup()`` + are used to create a dependencies file automatically. If you are manually + creating ``depends.txt`` right now, please switch to using these setup + arguments as soon as practical, because ``depends.txt`` support will be + removed in the 0.6 release cycle. For documentation on the new arguments, + see the ``setuptools.dist.Distribution`` class. + + * Setup scripts using setuptools now always install using ``easy_install`` + internally, for ease of uninstallation and upgrading. + +0.5a1 + * Added support for "self-installation" bootstrapping. Packages can now + include ``ez_setup.py`` in their source distribution, and add the following + to their ``setup.py``, in order to automatically bootstrap installation of + setuptools as part of their setup process:: + + from ez_setup import use_setuptools + use_setuptools() + + from setuptools import setup + # etc... + +0.4a2 + * Added ``ez_setup.py`` installer/bootstrap script to make initial setuptools + installation easier, and to allow distributions using setuptools to avoid + having to include setuptools in their source distribution. + + * All downloads are now managed by the ``PackageIndex`` class (which is now + subclassable and replaceable), so that embedders can more easily override + download logic, give download progress reports, etc. The class has also + been moved to the new ``setuptools.package_index`` module. + + * The ``Installer`` class no longer handles downloading, manages a temporary + directory, or tracks the ``zip_ok`` option. Downloading is now handled + by ``PackageIndex``, and ``Installer`` has become an ``easy_install`` + command class based on ``setuptools.Command``. + + * There is a new ``setuptools.sandbox.run_setup()`` API to invoke a setup + script in a directory sandbox, and a new ``setuptools.archive_util`` module + with an ``unpack_archive()`` API. These were split out of EasyInstall to + allow reuse by other tools and applications. + + * ``setuptools.Command`` now supports reinitializing commands using keyword + arguments to set/reset options. Also, ``Command`` subclasses can now set + their ``command_consumes_arguments`` attribute to ``True`` in order to + receive an ``args`` option containing the rest of the command line. + +0.3a2 + * Added new options to ``bdist_egg`` to allow tagging the egg's version number + with a subversion revision number, the current date, or an explicit tag + value. Run ``setup.py bdist_egg --help`` to get more information. + + * Misc. bug fixes + +0.3a1 + * Initial release. + +Mailing List and Bug Tracker +============================ + +Please use the `distutils-sig mailing list`_ for questions and discussion about +setuptools, and the `setuptools bug tracker`_ ONLY for issues you have +confirmed via the list are actual bugs, and which you have reduced to a minimal +set of steps to reproduce. + +.. _distutils-sig mailing list: http://mail.python.org/pipermail/distutils-sig/ +.. _setuptools bug tracker: http://bugs.python.org/setuptools/ + |