diff options
36 files changed, 1009 insertions, 1354 deletions
diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 487e0a44..d35d9e11 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -20,6 +20,7 @@ Contributors * Martin von Löwis * Noufal Ibrahim * Pete Hollobon +* Philip J. Eby * Philip Jenvey * Reinout van Rees * Robert Myers diff --git a/DEVGUIDE.txt b/DEVGUIDE.txt index 8dcabfd1..db556480 100644 --- a/DEVGUIDE.txt +++ b/DEVGUIDE.txt @@ -2,11 +2,11 @@ Quick notes for contributors ============================ -Distribute is using Mercurial. +Setuptools is developed using the DVCS Mercurial. Grab the code at bitbucket:: - $ hg clone https://bitbucket.org/tarek/distribute + $ hg clone https://bitbucket.org/jaraco/setuptools If you want to contribute changes, we recommend you fork the repository on bitbucket, commit the changes to your repository, and then make a pull request @@ -14,8 +14,8 @@ on bitbucket. If you make some changes, don't forget to: - add a note in CHANGES.txt -And remember that 0.6 (the only development line) is only bug fixes, and the -APIs should be fully backward compatible with Setuptools. +Please commit bug-fixes against the current maintenance branch and new +features to the default branch. You can run the tests via:: diff --git a/MERGE.txt b/MERGE.txt new file mode 100644 index 00000000..f836f354 --- /dev/null +++ b/MERGE.txt @@ -0,0 +1,110 @@ +Process +======= + +In order to try to accurately reflect the fork and then re-merge of the +projects, the merge process brought both code trees together into one +repository and grafted the Distribute fork onto the Setuptools development +line (as if it had been created as a branch in the first place). + +The rebase to get distribute onto setuptools went something like this:: + + hg phase -d -f -r 26b4c29b62db + hg rebase -s 26b4c29b62db -d 7a5cf59c78d7 + +The technique required a late version of mercurial (2.5) to work correctly. + +The only code that was included was the code that was ancestral to the public +releases of Distribute 0.6. Additionally, because Setuptools was not hosted +on Mercurial at the time of the fork and because the Distribute fork did not +include a complete conversion of the Setuptools history, the Distribute +changesets had to be re-applied to a new, different conversion of the +Setuptools SVN repository. As a result, all of the hashes have changed. + +Distribute was grafted in a 'distribute' branch and the 'setuptools-0.6' +branch was targeted for the merge. The 'setuptools' branch remains with +unreleased code and may be incorporated in the future. + +Reconciling Differences +======================= + +There were both technical and philosophical differences between Setuptools +and Distribute. To reconcile these differences in a manageable way, the +following technique was undertaken: + +Create a 'Setuptools-Distribute merge' branch, based on a late release of +Distribute (0.6.35). This was done with a00b441856c4. + +In that branch, first remove code that is no longer relevant to +Setuptools (such as the setuptools patching code). + +Next, in the the merge branch, create another base from at the point where the +fork occurred (such that the code is still essentially an older but pristine +setuptools). This base can be found as 955792b069d0. This creates two heads +in the merge branch, each with a basis in the fork. + +Then, repeatedly copy changes for a +single file or small group of files from a late revision of that file in the +'setuptools-0.6' branch (1aae1efe5733 was used) and commit those changes on +the setuptools-only head. That head is then merged with the head with +Distribute changes. It is in this Mercurial +merge operation that the fundamental differences between Distribute and +Setuptools are reconciled, but since only a single file or small set of files +are used, the scope is limited. + +Finally, once all the challenging files have been reconciled and merged, the +remaining changes from the setuptools-0.6 branch are merged, deferring to the +reconciled changes (a1fa855a5a62 and 160ccaa46be0). + +Originally, jaraco attempted all of this using anonymous heads in the +Distribute branch, but later realized this technique made for a somewhat +unclear merge process, so the changes were re-committed as described above +for clarity. In this way, the "distribute" and "setuptools" branches can +continue to track the official Distribute changesets. + +Concessions +=========== + +With the merge of Setuptools and Distribute, the following concessions were +made: + +Differences from setuptools 0.6c12: + +Major Changes +------------- + +* Python 3 support. +* Improved support for GAE. +* Support `PEP-370 <http://www.python.org/dev/peps/pep-0370/>`_ per-user site + packages. +* Sort order of Distributions in pkg_resources now prefers PyPI to external + links (Distribute issue 163). + +Minor Changes +------------- + +* Wording of some output has changed to replace contractions with their + canonical form (i.e. prefer "could not" to "couldn't"). +* Manifest files are only written for 32-bit .exe launchers. + +Differences from Distribute 0.6.36: + +Major Changes +------------- + +* The _distribute property of the setuptools module has been removed. +* Distributions are once again installed as zipped eggs by default, per the + rationale given in `the seminal bug report + <http://bugs.python.org/setuptools/issue33>`_ indicates that the feature + should remain and no substantial justification was given in the `Distribute + report <https://bitbucket.org/tarek/distribute/issue/19/>`_. + +Minor Changes +------------- + +* The patch for `#174 <https://bitbucket.org/tarek/distribute/issue/174>`_ + has been rolled-back, as the comment on the ticket indicates that the patch + addressed a symptom and not the fundamental issue. +* ``easy_install`` (the command) once again honors setup.cfg if found in the + current directory. The "mis-behavior" characterized in `#99 + <https://bitbucket.org/tarek/distribute/issue/99>`_ is actually intended + behavior, and no substantial rationale was given for the deviation. @@ -1,228 +1,202 @@ =============================== -Installing and Using Distribute +Installing and Using Setuptools =============================== .. contents:: **Table of Contents** ------------ -Disclaimers ------------ - -About the fork -============== - -`Distribute` is a fork of the `Setuptools` project. - -Distribute is intended to replace Setuptools as the standard method -for working with Python module distributions. - -The fork has two goals: - -- Providing a backward compatible version to replace Setuptools - and make all distributions that depend on Setuptools work as - before, but with less bugs and behaviorial issues. - This work is done in the 0.6.x series. +---------------------------------- +Security Issues - Read this First! +---------------------------------- - Starting with version 0.6.2, Distribute supports Python 3. - Installing and using distribute for Python 3 code works exactly - the same as for Python 2 code, but Distribute also helps you to support - Python 2 and Python 3 from the same source code by letting you run 2to3 - on the code as a part of the build process, by setting the keyword parameter - ``use_2to3`` to True. See http://packages.python.org/distribute for more - information. +Setuptools and ``easy_install`` currently default to allowing automated +download and execution of code from anywhere on the internet, without actually +verifying the owners of the websites or the authors of the code. If you want +your installation to be more secure, you will need to: -- Refactoring the code, and releasing it in several distributions. - This work is being done in the 0.7.x series but not yet released. + 1. Manually install the `requests <https://pypi.python.org/pypi/requests>`_ + library **after** installing setuptools, using an SSL-enabled browser or + other tool. (This will enable SSL certificate verification.) -The roadmap is still evolving, and the page that is up-to-date is -located at : `http://packages.python.org/distribute/roadmap`. + 2. Configure your default ``--allow-hosts`` setting so that ``easy_install`` + will only download from sites you trust. (E.g., to only download from + ``pypi.python.org`` or some other trusted package index.) -If you install `Distribute` and want to switch back for any reason to -`Setuptools`, get to the `Uninstallation instructions`_ section. + 3. If you are using a Python version less than 2.6, you will also need to + install the `SSL backport module <https://pypi.python.org/pypi/requests>`_ + to enable SSL downloads from PyPI. (Unfortunately, the ``requests`` + package does not support older versions of Python at this time, so SSL + certificate verification will not be enabled. But at least you'll still be + able to use PyPI, which is in the process of switching to an all-SSL policy + for downloads. + +For more information on how to do all of the above, and for other security- +related information, please see the full `setuptools security documentation +<http://peak.telecommunity.com/DevCenter/SetuptoolsSecurity>`_. -More documentation -================== - -You can get more information in the Sphinx-based documentation, located -at http://packages.python.org/distribute. This documentation includes the old -Setuptools documentation that is slowly replaced, and brand new content. - -About the installation process -============================== - -The `Distribute` installer modifies your installation by de-activating an -existing installation of `Setuptools` in a bootstrap process. This process -has been tested in various installation schemes and contexts but in case of a -bug during this process your Python installation might be left in a broken -state. Since all modified files and directories are copied before the -installation starts, you will be able to get back to a normal state by reading -the instructions in the `Uninstallation instructions`_ section. - -In any case, it is recommended to save you `site-packages` directory before -you start the installation of `Distribute`. ------------------------- Installation Instructions ------------------------- -Distribute is only released as a source distribution. - -It can be installed using pip, and can be done so with the source tarball, -or by using the ``distribute_setup.py`` script provided online. - -``distribute_setup.py`` is the simplest and preferred way on all systems. +Windows +======= -distribute_setup.py -=================== +32-bit version of Python + Install setuptools using the provided ``.exe`` installer. -Download -`distribute_setup.py <http://python-distribute.org/distribute_setup.py>`_ -and execute it, using the Python interpreter of your choice. +64-bit versions of Python + Download `ez_setup.py`_ and run it; it will download the appropriate .egg file and install it for you. (Currently, the provided ``.exe`` installer does not support 64-bit versions of Python for Windows, due to a `distutils installer compatibility issue`_ -If your shell has the ``curl`` program you can do:: +.. _ez_setup.py: http://peak.telecommunity.com/dist/ez_setup.py +.. _distutils installer compatibility issue: http://bugs.python.org/issue6792 - $ curl -O http://python-distribute.org/distribute_setup.py - $ python distribute_setup.py -Notice this file is also provided in the source release. +NOTE: Regardless of what sort of Python you're using, if you've previously +installed older versions of setuptools, please delete all ``setuptools*.egg`` +and ``setuptools.pth`` files from your system's ``site-packages`` directory +(and any other ``sys.path`` directories) FIRST. -pip -=== +If you are upgrading a previous version of setuptools that was installed using +an ``.exe`` installer, please be sure to also *uninstall that older version* +via your system's "Add/Remove Programs" feature, BEFORE installing the newer +version. -Run easy_install or pip:: +Once installation is complete, you will find an ``easy_install.exe`` program in +your Python ``Scripts`` subdirectory. Be sure to add this directory to your +``PATH`` environment variable, if you haven't already done so. - $ pip install distribute -Source installation -=================== +RPM-Based Systems +================= -Download the source tarball, uncompress it, then run the install command:: +Install setuptools using the provided source RPM. The included ``.spec`` file +assumes you are installing using the default ``python`` executable, and is not +specific to a particular Python version. The ``easy_install`` executable will +be installed to a system ``bin`` directory such as ``/usr/bin``. - $ curl -O http://pypi.python.org/packages/source/d/distribute/distribute-0.6.40.tar.gz - $ tar -xzvf distribute-0.6.40.tar.gz - $ cd distribute-0.6.40 - $ python setup.py install +If you wish to install to a location other than the default Python +installation's default ``site-packages`` directory (and ``$prefix/bin`` for +scripts), please use the ``.egg``-based installation approach described in the +following section. ---------------------------- -Uninstallation Instructions ---------------------------- -Like other distutils-based distributions, Distribute doesn't provide an -uninstaller yet. It's all done manually! We are all waiting for PEP 376 -support in Python. - -Distribute is installed in three steps: - -1. it gets out of the way an existing installation of Setuptools -2. it installs a `fake` setuptools installation -3. it installs distribute - -Distribute can be removed like this: - -- remove the ``distribute*.egg`` file located in your site-packages directory -- remove the ``setuptools.pth`` file located in you site-packages directory -- remove the easy_install script located in you ``sys.prefix/bin`` directory -- remove the ``setuptools*.egg`` directory located in your site-packages directory, - if any. - -If you want to get back to setuptools: - -- reinstall setuptools using its instruction. - -Lastly: - -- remove the *.OLD.* directory located in your site-packages directory if any, - **once you have checked everything was working correctly again**. +Cygwin, Mac OS X, Linux, Other +============================== -------------------------- -Quick help for developers -------------------------- +1. Download the appropriate egg for your version of Python (e.g. + ``setuptools-0.6c9-py2.4.egg``). Do NOT rename it. -To create an egg which is compatible with Distribute, use the same -practice as with Setuptools, e.g.:: +2. Run it as if it were a shell script, e.g. ``sh setuptools-0.6c9-py2.4.egg``. + Setuptools will install itself using the matching version of Python (e.g. + ``python2.4``), and will place the ``easy_install`` executable in the + default location for installing Python scripts (as determined by the + standard distutils configuration files, or by the Python installation). - from setuptools import setup +If you want to install setuptools to somewhere other than ``site-packages`` or +your default distutils installation locations for libraries and scripts, you +may include EasyInstall command-line options such as ``--prefix``, +``--install-dir``, and so on, following the ``.egg`` filename on the same +command line. For example:: - setup(... - ) + sh setuptools-0.6c9-py2.4.egg --prefix=~ -To use `pkg_resources` to access data files in the egg, you should -require the Setuptools distribution explicitly:: +You can use ``--help`` to get a full options list, but we recommend consulting +the `EasyInstall manual`_ for detailed instructions, especially `the section +on custom installation locations`_. - from setuptools import setup +.. _EasyInstall manual: http://peak.telecommunity.com/DevCenter/EasyInstall +.. _the section on custom installation locations: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations - setup(... - install_requires=['setuptools'] - ) -Only if you need Distribute-specific functionality should you depend -on it explicitly. In this case, replace the Setuptools dependency:: +Cygwin Note +----------- - from setuptools import setup +If you are trying to install setuptools for the **Windows** version of Python +(as opposed to the Cygwin version that lives in ``/usr/bin``), you must make +sure that an appropriate executable (``python2.3``, ``python2.4``, or +``python2.5``) is on your **Cygwin** ``PATH`` when invoking the egg. For +example, doing the following at a Cygwin bash prompt will install setuptools +for the **Windows** Python found at ``C:\\Python24``:: - setup(... - install_requires=['distribute'] - ) + ln -s /cygdrive/c/Python24/python.exe python2.4 + PATH=.:$PATH sh setuptools-0.6c9-py2.4.egg + rm python2.4 ------------ -Install FAQ ------------ -- **Why is Distribute wrapping my Setuptools installation?** +Downloads +========= - Since Distribute is a fork, and since it provides the same package - and modules, it renames the existing Setuptools egg and inserts a - new one which merely wraps the Distribute code. This way, full - backwards compatibility is kept for packages which rely on the - Setuptools modules. +All setuptools downloads can be found at `the project's home page in the Python +Package Index`_. Scroll to the very bottom of the page to find the links. - At the same time, packages can meet their dependency on Setuptools - without actually installing it (which would disable Distribute). +.. _the project's home page in the Python Package Index: http://pypi.python.org/pypi/setuptools#files -- **How does Distribute interact with virtualenv?** +In addition to the PyPI downloads, the development version of ``setuptools`` +is available from the `Python SVN sandbox`_, and in-development versions of the +`0.6 branch`_ are available as well. - Everytime you create a virtualenv it will install setuptools by default. - You either need to re-install Distribute in it right after or pass the - ``--distribute`` option when creating it. +.. _0.6 branch: http://svn.python.org/projects/sandbox/branches/setuptools-0.6/#egg=setuptools-dev06 - Once installed, your virtualenv will use Distribute transparently. +.. _Python SVN sandbox: http://svn.python.org/projects/sandbox/trunk/setuptools/#egg=setuptools-dev - Although, if you have Setuptools installed in your system-wide Python, - and if the virtualenv you are in was generated without the `--no-site-packages` - option, the Distribute installation will stop. +-------------------------------- +Using Setuptools and EasyInstall +-------------------------------- - You need in this case to build a virtualenv with the `--no-site-packages` - option or to install `Distribute` globally. +Here are some of the available manuals, tutorials, and other resources for +learning about Setuptools, Python Eggs, and EasyInstall: -- **How does Distribute interacts with zc.buildout?** +* `The EasyInstall user's guide and reference manual`_ +* `The setuptools Developer's Guide`_ +* `The pkg_resources API reference`_ +* `Package Compatibility Notes`_ (user-maintained) +* `The Internal Structure of Python Eggs`_ - You can use Distribute in your zc.buildout, with the --distribute option, - starting at zc.buildout 1.4.2:: +Questions, comments, and bug reports should be directed to the `distutils-sig +mailing list`_. If you have written (or know of) any tutorials, documentation, +plug-ins, or other resources for setuptools users, please let us know about +them there, so this reference list can be updated. If you have working, +*tested* patches to correct problems or add features, you may submit them to +the `setuptools bug tracker`_. - $ python bootstrap.py --distribute +.. _setuptools bug tracker: http://bugs.python.org/setuptools/ +.. _Package Compatibility Notes: http://peak.telecommunity.com/DevCenter/PackageNotes +.. _The Internal Structure of Python Eggs: http://peak.telecommunity.com/DevCenter/EggFormats +.. _The setuptools Developer's Guide: http://peak.telecommunity.com/DevCenter/setuptools +.. _The pkg_resources API reference: http://peak.telecommunity.com/DevCenter/PkgResources +.. _The EasyInstall user's guide and reference manual: http://peak.telecommunity.com/DevCenter/EasyInstall +.. _distutils-sig mailing list: http://mail.python.org/pipermail/distutils-sig/ - For previous zc.buildout versions, *the only thing* you need to do - is use the bootstrap at `http://python-distribute.org/bootstrap.py`. Run - that bootstrap and ``bin/buildout`` (and all other buildout-generated - scripts) will transparently use distribute instead of setuptools. You do - not need a specific buildout release. - A shared eggs directory is no problem (since 0.6.6): the setuptools egg is - left in place unmodified. So other buildouts that do not yet use the new - bootstrap continue to work just fine. And there is no need to list - ``distribute`` somewhere in your eggs: using the bootstrap is enough. +------- +Credits +------- - The source code for the bootstrap script is located at - `http://bitbucket.org/tarek/buildout-distribute`. +* The original design for the ``.egg`` format and the ``pkg_resources`` API was + co-created by Phillip Eby and Bob Ippolito. Bob also implemented the first + version of ``pkg_resources``, and supplied the OS X operating system version + compatibility algorithm. +* Ian Bicking implemented many early "creature comfort" features of + easy_install, including support for downloading via Sourceforge and + Subversion repositories. Ian's comments on the Web-SIG about WSGI + application deployment also inspired the concept of "entry points" in eggs, + and he has given talks at PyCon and elsewhere to inform and educate the + community about eggs and setuptools. +* Jim Fulton contributed time and effort to build automated tests of various + aspects of ``easy_install``, and supplied the doctests for the command-line + ``.exe`` wrappers on Windows. ------------------------------ -Feedback and getting involved ------------------------------ +* Phillip J. Eby is the principal author and maintainer of setuptools, and + first proposed the idea of an importable binary distribution format for + Python application plug-ins. -- Mailing list: http://mail.python.org/mailman/listinfo/distutils-sig -- Issue tracker: http://bitbucket.org/tarek/distribute/issues/ -- Code Repository: http://bitbucket.org/tarek/distribute +* Significant parts of the implementation of setuptools were funded by the Open + Source Applications Foundation, to provide a plug-in infrastructure for the + Chandler PIM application. In addition, many OSAF staffers (such as Mike + "Code Bear" Taylor) contributed their time and stress as guinea pigs for the + use of eggs and setuptools, even before eggs were "cool". (Thanks, guys!) +.. _files: diff --git a/distribute_setup.py b/distribute_setup.py deleted file mode 100644 index c098b9d2..00000000 --- a/distribute_setup.py +++ /dev/null @@ -1,546 +0,0 @@ -#!python -"""Bootstrap distribute installation - -If you want to use setuptools in your package's setup.py, just include this -file in the same directory with it, and add this to the top of your setup.py:: - - from distribute_setup import use_setuptools - use_setuptools() - -If you want to require a specific version of setuptools, set a download -mirror, or use an alternate download directory, you can do so by supplying -the appropriate options to ``use_setuptools()``. - -This file can also be run as a script to install or upgrade setuptools. -""" -import os -import shutil -import sys -import time -import fnmatch -import tempfile -import tarfile -import optparse - -from distutils import log - -try: - from site import USER_SITE -except ImportError: - USER_SITE = None - -try: - import subprocess - - def _python_cmd(*args): - args = (sys.executable,) + args - return subprocess.call(args) == 0 - -except ImportError: - # will be used for python 2.3 - def _python_cmd(*args): - args = (sys.executable,) + args - # quoting arguments if windows - if sys.platform == 'win32': - def quote(arg): - if ' ' in arg: - return '"%s"' % arg - return arg - args = [quote(arg) for arg in args] - return os.spawnl(os.P_WAIT, sys.executable, *args) == 0 - -DEFAULT_VERSION = "0.6.40" -DEFAULT_URL = "http://pypi.python.org/packages/source/d/distribute/" -SETUPTOOLS_FAKED_VERSION = "0.6c11" - -SETUPTOOLS_PKG_INFO = """\ -Metadata-Version: 1.0 -Name: setuptools -Version: %s -Summary: xxxx -Home-page: xxx -Author: xxx -Author-email: xxx -License: xxx -Description: xxx -""" % SETUPTOOLS_FAKED_VERSION - - -def _install(tarball, install_args=()): - # extracting the tarball - tmpdir = tempfile.mkdtemp() - log.warn('Extracting in %s', tmpdir) - old_wd = os.getcwd() - try: - os.chdir(tmpdir) - tar = tarfile.open(tarball) - _extractall(tar) - tar.close() - - # going in the directory - subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0]) - os.chdir(subdir) - log.warn('Now working in %s', subdir) - - # installing - log.warn('Installing Distribute') - if not _python_cmd('setup.py', 'install', *install_args): - log.warn('Something went wrong during the installation.') - log.warn('See the error message above.') - # exitcode will be 2 - return 2 - finally: - os.chdir(old_wd) - shutil.rmtree(tmpdir) - - -def _build_egg(egg, tarball, to_dir): - # extracting the tarball - tmpdir = tempfile.mkdtemp() - log.warn('Extracting in %s', tmpdir) - old_wd = os.getcwd() - try: - os.chdir(tmpdir) - tar = tarfile.open(tarball) - _extractall(tar) - tar.close() - - # going in the directory - subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0]) - os.chdir(subdir) - log.warn('Now working in %s', subdir) - - # building an egg - log.warn('Building a Distribute egg in %s', to_dir) - _python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir) - - finally: - os.chdir(old_wd) - shutil.rmtree(tmpdir) - # returning the result - log.warn(egg) - if not os.path.exists(egg): - raise IOError('Could not build the egg.') - - -def _do_download(version, download_base, to_dir, download_delay): - egg = os.path.join(to_dir, 'distribute-%s-py%d.%d.egg' - % (version, sys.version_info[0], sys.version_info[1])) - if not os.path.exists(egg): - tarball = download_setuptools(version, download_base, - to_dir, download_delay) - _build_egg(egg, tarball, to_dir) - sys.path.insert(0, egg) - import setuptools - setuptools.bootstrap_install_from = egg - - -def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL, - to_dir=os.curdir, download_delay=15, no_fake=True): - # making sure we use the absolute path - to_dir = os.path.abspath(to_dir) - was_imported = 'pkg_resources' in sys.modules or \ - 'setuptools' in sys.modules - try: - try: - import pkg_resources - if not hasattr(pkg_resources, '_distribute'): - if not no_fake: - _fake_setuptools() - raise ImportError - except ImportError: - return _do_download(version, download_base, to_dir, download_delay) - try: - pkg_resources.require("distribute>=" + version) - return - except pkg_resources.VersionConflict: - e = sys.exc_info()[1] - if was_imported: - sys.stderr.write( - "The required version of distribute (>=%s) is not available,\n" - "and can't be installed while this script is running. Please\n" - "install a more recent version first, using\n" - "'easy_install -U distribute'." - "\n\n(Currently using %r)\n" % (version, e.args[0])) - sys.exit(2) - else: - del pkg_resources, sys.modules['pkg_resources'] # reload ok - return _do_download(version, download_base, to_dir, - download_delay) - except pkg_resources.DistributionNotFound: - return _do_download(version, download_base, to_dir, - download_delay) - finally: - if not no_fake: - _create_fake_setuptools_pkg_info(to_dir) - - -def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL, - to_dir=os.curdir, delay=15): - """Download distribute from a specified location and return its filename - - `version` should be a valid distribute version number that is available - as an egg for download under the `download_base` URL (which should end - with a '/'). `to_dir` is the directory where the egg will be downloaded. - `delay` is the number of seconds to pause before an actual download - attempt. - """ - # making sure we use the absolute path - to_dir = os.path.abspath(to_dir) - try: - from urllib.request import urlopen - except ImportError: - from urllib2 import urlopen - tgz_name = "distribute-%s.tar.gz" % version - url = download_base + tgz_name - saveto = os.path.join(to_dir, tgz_name) - src = dst = None - if not os.path.exists(saveto): # Avoid repeated downloads - try: - log.warn("Downloading %s", url) - src = urlopen(url) - # Read/write all in one block, so we don't create a corrupt file - # if the download is interrupted. - data = src.read() - dst = open(saveto, "wb") - dst.write(data) - finally: - if src: - src.close() - if dst: - dst.close() - return os.path.realpath(saveto) - - -def _no_sandbox(function): - def __no_sandbox(*args, **kw): - try: - from setuptools.sandbox import DirectorySandbox - if not hasattr(DirectorySandbox, '_old'): - def violation(*args): - pass - DirectorySandbox._old = DirectorySandbox._violation - DirectorySandbox._violation = violation - patched = True - else: - patched = False - except ImportError: - patched = False - - try: - return function(*args, **kw) - finally: - if patched: - DirectorySandbox._violation = DirectorySandbox._old - del DirectorySandbox._old - - return __no_sandbox - - -def _patch_file(path, content): - """Will backup the file then patch it""" - f = open(path) - existing_content = f.read() - f.close() - if existing_content == content: - # already patched - log.warn('Already patched.') - return False - log.warn('Patching...') - _rename_path(path) - f = open(path, 'w') - try: - f.write(content) - finally: - f.close() - return True - -_patch_file = _no_sandbox(_patch_file) - - -def _same_content(path, content): - f = open(path) - existing_content = f.read() - f.close() - return existing_content == content - - -def _rename_path(path): - new_name = path + '.OLD.%s' % time.time() - log.warn('Renaming %s to %s', path, new_name) - os.rename(path, new_name) - return new_name - - -def _remove_flat_installation(placeholder): - if not os.path.isdir(placeholder): - log.warn('Unkown installation at %s', placeholder) - return False - found = False - for file in os.listdir(placeholder): - if fnmatch.fnmatch(file, 'setuptools*.egg-info'): - found = True - break - if not found: - log.warn('Could not locate setuptools*.egg-info') - return - - log.warn('Moving elements out of the way...') - pkg_info = os.path.join(placeholder, file) - if os.path.isdir(pkg_info): - patched = _patch_egg_dir(pkg_info) - else: - patched = _patch_file(pkg_info, SETUPTOOLS_PKG_INFO) - - if not patched: - log.warn('%s already patched.', pkg_info) - return False - # now let's move the files out of the way - for element in ('setuptools', 'pkg_resources.py', 'site.py'): - element = os.path.join(placeholder, element) - if os.path.exists(element): - _rename_path(element) - else: - log.warn('Could not find the %s element of the ' - 'Setuptools distribution', element) - return True - -_remove_flat_installation = _no_sandbox(_remove_flat_installation) - - -def _after_install(dist): - log.warn('After install bootstrap.') - placeholder = dist.get_command_obj('install').install_purelib - _create_fake_setuptools_pkg_info(placeholder) - - -def _create_fake_setuptools_pkg_info(placeholder): - if not placeholder or not os.path.exists(placeholder): - log.warn('Could not find the install location') - return - pyver = '%s.%s' % (sys.version_info[0], sys.version_info[1]) - setuptools_file = 'setuptools-%s-py%s.egg-info' % \ - (SETUPTOOLS_FAKED_VERSION, pyver) - pkg_info = os.path.join(placeholder, setuptools_file) - if os.path.exists(pkg_info): - log.warn('%s already exists', pkg_info) - return - - log.warn('Creating %s', pkg_info) - try: - f = open(pkg_info, 'w') - except EnvironmentError: - log.warn("Don't have permissions to write %s, skipping", pkg_info) - return - try: - f.write(SETUPTOOLS_PKG_INFO) - finally: - f.close() - - pth_file = os.path.join(placeholder, 'setuptools.pth') - log.warn('Creating %s', pth_file) - f = open(pth_file, 'w') - try: - f.write(os.path.join(os.curdir, setuptools_file)) - finally: - f.close() - -_create_fake_setuptools_pkg_info = _no_sandbox( - _create_fake_setuptools_pkg_info -) - - -def _patch_egg_dir(path): - # let's check if it's already patched - pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO') - if os.path.exists(pkg_info): - if _same_content(pkg_info, SETUPTOOLS_PKG_INFO): - log.warn('%s already patched.', pkg_info) - return False - _rename_path(path) - os.mkdir(path) - os.mkdir(os.path.join(path, 'EGG-INFO')) - pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO') - f = open(pkg_info, 'w') - try: - f.write(SETUPTOOLS_PKG_INFO) - finally: - f.close() - return True - -_patch_egg_dir = _no_sandbox(_patch_egg_dir) - - -def _before_install(): - log.warn('Before install bootstrap.') - _fake_setuptools() - - -def _under_prefix(location): - if 'install' not in sys.argv: - return True - args = sys.argv[sys.argv.index('install') + 1:] - for index, arg in enumerate(args): - for option in ('--root', '--prefix'): - if arg.startswith('%s=' % option): - top_dir = arg.split('root=')[-1] - return location.startswith(top_dir) - elif arg == option: - if len(args) > index: - top_dir = args[index + 1] - return location.startswith(top_dir) - if arg == '--user' and USER_SITE is not None: - return location.startswith(USER_SITE) - return True - - -def _fake_setuptools(): - log.warn('Scanning installed packages') - try: - import pkg_resources - except ImportError: - # we're cool - log.warn('Setuptools or Distribute does not seem to be installed.') - return - ws = pkg_resources.working_set - try: - setuptools_dist = ws.find( - pkg_resources.Requirement.parse('setuptools', replacement=False) - ) - except TypeError: - # old distribute API - setuptools_dist = ws.find( - pkg_resources.Requirement.parse('setuptools') - ) - - if setuptools_dist is None: - log.warn('No setuptools distribution found') - return - # detecting if it was already faked - setuptools_location = setuptools_dist.location - log.warn('Setuptools installation detected at %s', setuptools_location) - - # if --root or --preix was provided, and if - # setuptools is not located in them, we don't patch it - if not _under_prefix(setuptools_location): - log.warn('Not patching, --root or --prefix is installing Distribute' - ' in another location') - return - - # let's see if its an egg - if not setuptools_location.endswith('.egg'): - log.warn('Non-egg installation') - res = _remove_flat_installation(setuptools_location) - if not res: - return - else: - log.warn('Egg installation') - pkg_info = os.path.join(setuptools_location, 'EGG-INFO', 'PKG-INFO') - if (os.path.exists(pkg_info) and - _same_content(pkg_info, SETUPTOOLS_PKG_INFO)): - log.warn('Already patched.') - return - log.warn('Patching...') - # let's create a fake egg replacing setuptools one - res = _patch_egg_dir(setuptools_location) - if not res: - return - log.warn('Patching complete.') - _relaunch() - - -def _relaunch(): - log.warn('Relaunching...') - # we have to relaunch the process - # pip marker to avoid a relaunch bug - _cmd1 = ['-c', 'install', '--single-version-externally-managed'] - _cmd2 = ['-c', 'install', '--record'] - if sys.argv[:3] == _cmd1 or sys.argv[:3] == _cmd2: - sys.argv[0] = 'setup.py' - args = [sys.executable] + sys.argv - sys.exit(subprocess.call(args)) - - -def _extractall(self, path=".", members=None): - """Extract all members from the archive to the current working - directory and set owner, modification time and permissions on - directories afterwards. `path' specifies a different directory - to extract to. `members' is optional and must be a subset of the - list returned by getmembers(). - """ - import copy - import operator - from tarfile import ExtractError - directories = [] - - if members is None: - members = self - - for tarinfo in members: - if tarinfo.isdir(): - # Extract directories with a safe mode. - directories.append(tarinfo) - tarinfo = copy.copy(tarinfo) - tarinfo.mode = 448 # decimal for oct 0700 - self.extract(tarinfo, path) - - # Reverse sort directories. - if sys.version_info < (2, 4): - def sorter(dir1, dir2): - return cmp(dir1.name, dir2.name) - directories.sort(sorter) - directories.reverse() - else: - directories.sort(key=operator.attrgetter('name'), reverse=True) - - # Set correct owner, mtime and filemode on directories. - for tarinfo in directories: - dirpath = os.path.join(path, tarinfo.name) - try: - self.chown(tarinfo, dirpath) - self.utime(tarinfo, dirpath) - self.chmod(tarinfo, dirpath) - except ExtractError: - e = sys.exc_info()[1] - if self.errorlevel > 1: - raise - else: - self._dbg(1, "tarfile: %s" % e) - - -def _build_install_args(options): - """ - Build the arguments to 'python setup.py install' on the distribute package - """ - install_args = [] - if options.user_install: - if sys.version_info < (2, 6): - log.warn("--user requires Python 2.6 or later") - raise SystemExit(1) - install_args.append('--user') - return install_args - -def _parse_args(): - """ - Parse the command line for options - """ - parser = optparse.OptionParser() - parser.add_option( - '--user', dest='user_install', action='store_true', default=False, - help='install in user site package (requires Python 2.6 or later)') - parser.add_option( - '--download-base', dest='download_base', metavar="URL", - default=DEFAULT_URL, - help='alternative URL from where to download the distribute package') - options, args = parser.parse_args() - # positional arguments are ignored - return options - -def main(version=DEFAULT_VERSION): - """Install or upgrade setuptools and EasyInstall""" - options = _parse_args() - tarball = download_setuptools(download_base=options.download_base) - return _install(tarball, _build_install_args(options)) - -if __name__ == '__main__': - sys.exit(main()) diff --git a/docs/_templates/indexsidebar.html b/docs/_templates/indexsidebar.html index 932909f3..9d49ae5d 100644 --- a/docs/_templates/indexsidebar.html +++ b/docs/_templates/indexsidebar.html @@ -1,8 +1,8 @@ -<h3>Download</h3> +<h3>Download</h3> -<p>Current version: <b>{{ version }}</b></p> -<p>Get Distribute from the <a href="http://pypi.python.org/pypi/distribute"> Python Package Index</a> +<p>Current version: <b>{{ version }}</b></p> +<p>Get Setuptools from the <a href="http://pypi.python.org/pypi/setuptools"> Python Package Index</a> <h3>Questions? Suggestions? Contributions?</h3> -<p>Visit the <a href="http://bitbucket.org/tarek/distribute">Distribute project page</a> </p> +<p>Visit the <a href="http://bitbucket.org/jaraco/setuptools">Setuptools project page</a> </p> diff --git a/docs/conf.py b/docs/conf.py index ac332deb..9a60cfe1 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Distribute documentation build configuration file, created by +# Setuptools documentation build configuration file, created by # sphinx-quickstart on Fri Jul 17 14:22:37 2009. # # This file is execfile()d with the current directory set to its containing dir. @@ -40,8 +40,8 @@ source_suffix = '.txt' master_doc = 'index' # General information about the project. -project = u'Distribute' -copyright = u'2009-2011, The fellowship of the packaging' +project = u'Setuptools' +copyright = u'2009-2013, The fellowship of the packaging' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -106,10 +106,10 @@ html_theme_path = ['_theme'] # The name for this set of Sphinx documents. If None, it defaults to # "<project> v<release> documentation". -html_title = "Distribute documentation" +html_title = "Setuptools documentation" # A shorter title for the navigation bar. Default is the same as html_title. -html_short_title = "Distribute" +html_short_title = "Setuptools" # The name of an image file (relative to this directory) to place at the top # of the sidebar. @@ -161,7 +161,7 @@ html_use_index = False #html_file_suffix = '' # Output file base name for HTML help builder. -htmlhelp_basename = 'Distributedoc' +htmlhelp_basename = 'Setuptoolsdoc' # -- Options for LaTeX output -------------------------------------------------- @@ -175,7 +175,7 @@ htmlhelp_basename = 'Distributedoc' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ - ('index', 'Distribute.tex', ur'Distribute Documentation', + ('index', 'Setuptools.tex', ur'Setuptools Documentation', ur'The fellowship of the packaging', 'manual'), ] diff --git a/docs/easy_install.txt b/docs/easy_install.txt index 9b4fcfbb..dac0fa8f 100644 --- a/docs/easy_install.txt +++ b/docs/easy_install.txt @@ -94,7 +94,7 @@ directory, you can also retarget the installation location for scripts so they go on a directory that's already on the ``PATH``. For more information see the sections below on `Command-Line Options`_ and `Configuration Files`_. You can pass command line options (such as ``--script-dir``) to -``distribute_setup.py`` to control where ``easy_install.exe`` will be installed. +``ez_setup.py`` to control where ``easy_install.exe`` will be installed. @@ -776,10 +776,12 @@ Command-Line Options package not being available locally, or due to the use of the ``--update`` or ``-U`` option. -``--no-find-links`` Blocks the addition of any link. (New in Distribute 0.6.11) - This is useful if you want to avoid adding links defined in a project - easy_install is installing (wether it's a requested project or a - dependency.). When used, ``--find-links`` is ignored. +``--no-find-links`` Blocks the addition of any link. + This parameter is useful if you want to avoid adding links defined in a + project easy_install is installing (whether it's a requested project or a + dependency). When used, ``--find-links`` is ignored. + + Added in Distribute 0.6.11 and Setuptools 0.7. ``--delete-conflicting, -D`` (Removed in 0.6a11) (As of 0.6a11, this option is no longer necessary; please do not use it!) diff --git a/docs/index.txt b/docs/index.txt index 5f3b945b..21442b93 100644 --- a/docs/index.txt +++ b/docs/index.txt @@ -1,16 +1,15 @@ -Welcome to Distribute's documentation! -====================================== +Welcome to Setuptools' documentation! +===================================== -`Distribute` is a fork of the `Setuptools` project. +Setuptools is a fully-featured, actively-maintained, and stable library +designed to facilitate packaging Python projects, where packaging includes: -Distribute is intended to replace Setuptools as the standard method for -working with Python module distributions. - -For those who may wonder why they should switch to Distribute over Setuptools, it’s quite simple: - -- Distribute is a drop-in replacement for Setuptools -- The code is actively maintained, and has over 10 commiters -- Distribute offers Python 3 support ! + - Python package and module definitions + - Distribution package metadata + - Test hooks + - Project installation + - Platform-specific details + - Python 3 support Documentation content: @@ -23,14 +22,3 @@ Documentation content: setuptools easy_install pkg_resources - - -.. image:: http://python-distribute.org/pip_distribute.png - -Design done by Idan Gazit (http://pixane.com) - License: cc-by-3.0 - -Copy & paste:: - - curl -O http://python-distribute.org/distribute_setup.py - python distribute_setup.py - easy_install pip
\ No newline at end of file diff --git a/docs/python3.txt b/docs/python3.txt index 2f6cde4a..1e019951 100644 --- a/docs/python3.txt +++ b/docs/python3.txt @@ -1,18 +1,19 @@ ===================================================== -Supporting both Python 2 and Python 3 with Distribute +Supporting both Python 2 and Python 3 with Setuptools ===================================================== -Starting with version 0.6.2, Distribute supports Python 3. Installing and -using distribute for Python 3 code works exactly the same as for Python 2 -code, but Distribute also helps you to support Python 2 and Python 3 from +Starting with Distribute version 0.6.2 and Setuptools 0.7, the Setuptools +project supported Python 3. Installing and +using setuptools for Python 3 code works exactly the same as for Python 2 +code, but Setuptools also helps you to support Python 2 and Python 3 from the same source code by letting you run 2to3 on the code as a part of the build process, by setting the keyword parameter ``use_2to3`` to True. -Distribute as help during porting +Setuptools as help during porting ================================= -Distribute can make the porting process much easier by automatically running +Setuptools can make the porting process much easier by automatically running 2to3 as a part of the test running. To do this you need to configure the setup.py so that you can run the unit tests with ``python setup.py test``. @@ -24,10 +25,10 @@ The test command will now first run the build command during which the code will be converted with 2to3, and the tests will then be run from the build directory, as opposed from the source directory as is normally done. -Distribute will convert all Python files, and also all doctests in Python +Setuptools will convert all Python files, and also all doctests in Python files. However, if you have doctests located in separate text files, these will not automatically be converted. By adding them to the -``convert_2to3_doctests`` keyword parameter Distrubute will convert them as +``convert_2to3_doctests`` keyword parameter Setuptools will convert them as well. By default, the conversion uses all fixers in the ``lib2to3.fixers`` package. @@ -86,12 +87,14 @@ Advanced features If you don't want to run the 2to3 conversion on the doctests in Python files, you can turn that off by setting ``setuptools.use_2to3_on_doctests = False``. -Note on compatibility with setuptools -===================================== +Note on compatibility with older versions of setuptools +======================================================= -Setuptools do not know about the new keyword parameters to support Python 3. +Setuptools earlier than 0.7 does not know about the new keyword parameters to +support Python 3. As a result it will warn about the unknown keyword parameters if you use -setuptools instead of Distribute under Python 2. This is not an error, and +those versions of setuptools instead of Distribute under Python 2. This output +is not an error, and install process will continue as normal, but if you want to get rid of that error this is easy. Simply conditionally add the new parameters into an extra dict and pass that dict into setup():: @@ -117,5 +120,5 @@ dict and pass that dict into setup():: **extra ) -This way the parameters will only be used under Python 3, where you have to -use Distribute. +This way the parameters will only be used under Python 3, where Distribute or +Setuptools 0.7 or later is required. diff --git a/docs/roadmap.txt b/docs/roadmap.txt index ea5070ea..44bcdb0f 100644 --- a/docs/roadmap.txt +++ b/docs/roadmap.txt @@ -2,85 +2,13 @@ Roadmap ======= -Distribute has two branches: +Setuptools has merged with Distribute and to provide a unified codebase for +ongoing development. -- 0.6.x : provides a Setuptools-0.6cX compatible version -- 0.7.x : will provide a refactoring - -0.6.x -===== - -Not "much" is going to happen here, we want this branch to be helpful -to the community *today* by addressing the 40-or-so bugs -that were found in Setuptools and never fixed. This is eventually -happen soon because its development is -fast : there are up to 5 commiters that are working on it very often -(and the number grows weekly.) - -The biggest issue with this branch is that it is providing the same -packages and modules setuptools does, and this -requires some bootstrapping work where we make sure once Distribute is -installed, all Distribution that requires Setuptools -will continue to work. This is done by faking the metadata of -Setuptools 0.6c9. That's the only way we found to do this. - -There's one major thing though: thanks to the work of Lennart, Alex, -Martin, this branch supports Python 3, -which is great to have to speed up Py3 adoption. - -The goal of the 0.6.x is to remove as much bugs as we can, and try if -possible to remove the patches done -on Distutils. We will support 0.6.x maintenance for years and we will -promote its usage everywhere instead of -Setuptools. - -Some new commands are added there, when they are helpful and don't -interact with the rest. I am thinking -about "upload_docs" that let you upload documentation to PyPI. The -goal is to move it to Distutils -at some point, if the documentation feature of PyPI stays and starts to be used. - -0.7.x -===== - -We've started to refactor Distribute with this roadmap in mind (and -no, as someone said, it's not vaporware, -we've done a lot already) - -- 0.7.x can be installed and used with 0.6.x - -- easy_install is going to be deprecated ! use Pip ! - -- the version system will be deprecated, in favor of the one in Distutils - -- no more Distutils monkey-patch that happens once you use the code - (things like 'from distutils import cmd; cmd.Command = CustomCommand') - -- no more custom site.py (that is: if something misses in Python's - site.py we'll add it there instead of patching it) - -- no more namespaced packages system, if PEP 382 (namespaces package - support) makes it to 2.7 - -- The code is splitted in many packages and might be distributed under - several distributions. - - - distribute.resources: that's the old pkg_resources, but - reorganized in clean, pep-8 modules. This package will - only contain the query APIs and will focus on being PEP 376 - compatible. We will promote its usage and see if Pip wants - to use it as a basis. - It will probably shrink a lot though, once the stdlib provides PEP 376 support. - - - distribute.entrypoints: that's the old pkg_resources entry points - system, but on its own. it uses distribute.resources - - - distribute.index: that's package_index and a few other things. - everything required to interact with PyPI. We will promote - its usage and see if Pip wants to use it as a basis. - - - distribute.core (might be renamed to main): that's everything - else, and uses the other packages. - -Goal: A first release before (or when) Python 2.7 / 3.2 is out. +This new effort will draw from the resources of both projects to leverage +community contribution for ongoing advancement but also maintain stability +for the user base. +An initial release of Setuptools 0.7 will attempt to be compatible both with +Setuptools 0.6c11 and Distribute 0.6.36. Where compatibility cannot be +achieved, the changes should be well-documented. diff --git a/docs/setuptools.txt b/docs/setuptools.txt index fe8bb3f6..2504dcda 100644 --- a/docs/setuptools.txt +++ b/docs/setuptools.txt @@ -1,8 +1,8 @@ ================================================== -Building and Distributing Packages with Distribute +Building and Distributing Packages with Setuptools ================================================== -``Distribute`` is a collection of enhancements to the Python ``distutils`` +``Setuptools`` is a collection of enhancements to the Python ``distutils`` (for Python 2.3.5 and up on most platforms; 64-bit platforms require a minimum of Python 2.4) that allow you to more easily build and distribute Python packages, especially ones that have dependencies on other packages. @@ -15,7 +15,7 @@ including just a single `bootstrap module`_ (an 8K .py file), your package will automatically download and install ``setuptools`` if the user is building your package from source and doesn't have a suitable version already installed. -.. _bootstrap module: http://nightly.ziade.org/distribute_setup.py +.. _bootstrap module: http://bitbucket.org/jaraco/setuptools/downloads/ez_setup.py Feature Highlights: diff --git a/docs/using.txt b/docs/using.txt index 192f1dc2..6f93c386 100644 --- a/docs/using.txt +++ b/docs/using.txt @@ -1,21 +1,10 @@ ================================ -Using Distribute in your project +Using Setuptools in your project ================================ -To use Distribute in your project, the recommended way is to ship -`distribute_setup.py` alongside your `setup.py` script and call +To use Setuptools in your project, the recommended way is to ship +`ez_setup.py` alongside your `setup.py` script and call it at the very begining of `setup.py` like this:: - from distribute_setup import use_setuptools + from ez_setup import use_setuptools use_setuptools() - -Another way is to add ``Distribute`` in the ``install_requires`` option:: - - from setuptools import setup - - setup(... - install_requires=['distribute'] - ) - - -XXX to be finished diff --git a/ez_setup.py b/ez_setup.py new file mode 100644 index 00000000..57f019b1 --- /dev/null +++ b/ez_setup.py @@ -0,0 +1,274 @@ +#!python +"""Bootstrap setuptools installation + +If you want to use setuptools in your package's setup.py, just include this +file in the same directory with it, and add this to the top of your setup.py:: + + from ez_setup import use_setuptools + use_setuptools() + +If you want to require a specific version of setuptools, set a download +mirror, or use an alternate download directory, you can do so by supplying +the appropriate options to ``use_setuptools()``. + +This file can also be run as a script to install or upgrade setuptools. +""" +import os +import shutil +import sys +import tempfile +import tarfile +import optparse + +from distutils import log + +try: + from site import USER_SITE +except ImportError: + USER_SITE = None + +try: + import subprocess + + def _python_cmd(*args): + args = (sys.executable,) + args + return subprocess.call(args) == 0 + +except ImportError: + # will be used for python 2.3 + def _python_cmd(*args): + args = (sys.executable,) + args + # quoting arguments if windows + if sys.platform == 'win32': + def quote(arg): + if ' ' in arg: + return '"%s"' % arg + return arg + args = [quote(arg) for arg in args] + return os.spawnl(os.P_WAIT, sys.executable, *args) == 0 + +DEFAULT_VERSION = "0.6.40" +DEFAULT_URL = "http://pypi.python.org/packages/source/s/setuptools/" + + +def _install(tarball, install_args=()): + # extracting the tarball + tmpdir = tempfile.mkdtemp() + log.warn('Extracting in %s', tmpdir) + old_wd = os.getcwd() + try: + os.chdir(tmpdir) + tar = tarfile.open(tarball) + _extractall(tar) + tar.close() + + # going in the directory + subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0]) + os.chdir(subdir) + log.warn('Now working in %s', subdir) + + # installing + log.warn('Installing Setuptools') + if not _python_cmd('setup.py', 'install', *install_args): + log.warn('Something went wrong during the installation.') + log.warn('See the error message above.') + # exitcode will be 2 + return 2 + finally: + os.chdir(old_wd) + shutil.rmtree(tmpdir) + + +def _build_egg(egg, tarball, to_dir): + # extracting the tarball + tmpdir = tempfile.mkdtemp() + log.warn('Extracting in %s', tmpdir) + old_wd = os.getcwd() + try: + os.chdir(tmpdir) + tar = tarfile.open(tarball) + _extractall(tar) + tar.close() + + # going in the directory + subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0]) + os.chdir(subdir) + log.warn('Now working in %s', subdir) + + # building an egg + log.warn('Building a Setuptools egg in %s', to_dir) + _python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir) + + finally: + os.chdir(old_wd) + shutil.rmtree(tmpdir) + # returning the result + log.warn(egg) + if not os.path.exists(egg): + raise IOError('Could not build the egg.') + + +def _do_download(version, download_base, to_dir, download_delay): + egg = os.path.join(to_dir, 'setuptools-%s-py%d.%d.egg' + % (version, sys.version_info[0], sys.version_info[1])) + if not os.path.exists(egg): + tarball = download_setuptools(version, download_base, + to_dir, download_delay) + _build_egg(egg, tarball, to_dir) + sys.path.insert(0, egg) + import setuptools + setuptools.bootstrap_install_from = egg + + +def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL, + to_dir=os.curdir, download_delay=15): + # making sure we use the absolute path + to_dir = os.path.abspath(to_dir) + was_imported = 'pkg_resources' in sys.modules or \ + 'setuptools' in sys.modules + try: + import pkg_resources + except ImportError: + return _do_download(version, download_base, to_dir, download_delay) + try: + pkg_resources.require("setuptools>=" + version) + return + except pkg_resources.VersionConflict: + e = sys.exc_info()[1] + if was_imported: + sys.stderr.write( + "The required version of setuptools (>=%s) is not available,\n" + "and can't be installed while this script is running. Please\n" + "install a more recent version first, using\n" + "'easy_install -U setuptools'." + "\n\n(Currently using %r)\n" % (version, e.args[0])) + sys.exit(2) + else: + del pkg_resources, sys.modules['pkg_resources'] # reload ok + return _do_download(version, download_base, to_dir, + download_delay) + except pkg_resources.DistributionNotFound: + return _do_download(version, download_base, to_dir, + download_delay) + + +def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL, + to_dir=os.curdir, delay=15): + """Download setuptools from a specified location and return its filename + + `version` should be a valid setuptools version number that is available + as an egg for download under the `download_base` URL (which should end + with a '/'). `to_dir` is the directory where the egg will be downloaded. + `delay` is the number of seconds to pause before an actual download + attempt. + """ + # making sure we use the absolute path + to_dir = os.path.abspath(to_dir) + try: + from urllib.request import urlopen + except ImportError: + from urllib2 import urlopen + tgz_name = "setuptools-%s.tar.gz" % version + url = download_base + tgz_name + saveto = os.path.join(to_dir, tgz_name) + src = dst = None + if not os.path.exists(saveto): # Avoid repeated downloads + try: + log.warn("Downloading %s", url) + src = urlopen(url) + # Read/write all in one block, so we don't create a corrupt file + # if the download is interrupted. + data = src.read() + dst = open(saveto, "wb") + dst.write(data) + finally: + if src: + src.close() + if dst: + dst.close() + return os.path.realpath(saveto) + + +def _extractall(self, path=".", members=None): + """Extract all members from the archive to the current working + directory and set owner, modification time and permissions on + directories afterwards. `path' specifies a different directory + to extract to. `members' is optional and must be a subset of the + list returned by getmembers(). + """ + import copy + import operator + from tarfile import ExtractError + directories = [] + + if members is None: + members = self + + for tarinfo in members: + if tarinfo.isdir(): + # Extract directories with a safe mode. + directories.append(tarinfo) + tarinfo = copy.copy(tarinfo) + tarinfo.mode = 448 # decimal for oct 0700 + self.extract(tarinfo, path) + + # Reverse sort directories. + if sys.version_info < (2, 4): + def sorter(dir1, dir2): + return cmp(dir1.name, dir2.name) + directories.sort(sorter) + directories.reverse() + else: + directories.sort(key=operator.attrgetter('name'), reverse=True) + + # Set correct owner, mtime and filemode on directories. + for tarinfo in directories: + dirpath = os.path.join(path, tarinfo.name) + try: + self.chown(tarinfo, dirpath) + self.utime(tarinfo, dirpath) + self.chmod(tarinfo, dirpath) + except ExtractError: + e = sys.exc_info()[1] + if self.errorlevel > 1: + raise + else: + self._dbg(1, "tarfile: %s" % e) + + +def _build_install_args(options): + """ + Build the arguments to 'python setup.py install' on the setuptools package + """ + install_args = [] + if options.user_install: + if sys.version_info < (2, 6): + log.warn("--user requires Python 2.6 or later") + raise SystemExit(1) + install_args.append('--user') + return install_args + +def _parse_args(): + """ + Parse the command line for options + """ + parser = optparse.OptionParser() + parser.add_option( + '--user', dest='user_install', action='store_true', default=False, + help='install in user site package (requires Python 2.6 or later)') + parser.add_option( + '--download-base', dest='download_base', metavar="URL", + default=DEFAULT_URL, + help='alternative URL from where to download the setuptools package') + options, args = parser.parse_args() + # positional arguments are ignored + return options + +def main(version=DEFAULT_VERSION): + """Install or upgrade setuptools and EasyInstall""" + options = _parse_args() + tarball = download_setuptools(download_base=options.download_base) + return _install(tarball, _build_install_args(options)) + +if __name__ == '__main__': + sys.exit(main()) diff --git a/pkg_resources.py b/pkg_resources.py index f8de449e..03f750d7 100644 --- a/pkg_resources.py +++ b/pkg_resources.py @@ -13,7 +13,7 @@ The package resource API is designed to work with normal filesystem packages, method. """ -import sys, os, zipimport, time, re, imp, types +import sys, os, zipimport, time, re, imp from urlparse import urlparse, urlunparse try: @@ -39,15 +39,6 @@ if sys.version_info >= (3, 3) and sys.implementation.name == "cpython": else: importlib_bootstrap = None -# This marker is used to simplify the process that checks is the -# setuptools package was installed by the Setuptools project -# or by the Distribute project, in case Setuptools creates -# a distribution with the same version. -# -# The bootstrapping script for instance, will check if this -# attribute is present to decide wether to reinstall the package -_distribute = True - def _bypass_ensure_directory(name, mode=0777): # Sandbox-bypassing version of ensure_directory() if not WRITE_SUPPORT: @@ -96,6 +87,7 @@ _sget_none = _sset_none = lambda *args: None + def get_supported_platform(): """Return this platform's maximum compatible version. @@ -269,6 +261,12 @@ macosVersionString = re.compile(r"macosx-(\d+)\.(\d+)-(.*)") darwinVersionString = re.compile(r"darwin-(\d+)\.(\d+)\.(\d+)-(.*)") get_platform = get_build_platform # XXX backward compat + + + + + + def compatible_platforms(provided,required): """Can code for the `provided` platform run on the `required` platform? @@ -444,7 +442,7 @@ class WorkingSet(object): def add_entry(self, entry): """Add a path item to ``.entries``, finding any distributions on it - ``find_distributions(entry,True)`` is used to find distributions + ``find_distributions(entry, True)`` is used to find distributions corresponding to the path entry, and they are added. `entry` is always appended to ``.entries``, even if it is already present. (This is because ``sys.path`` can contain the same value more than @@ -551,7 +549,7 @@ class WorkingSet(object): keys2.append(dist.key) self._added_new(dist) - def resolve(self, requirements, env=None, installer=None, replacement=True): + def resolve(self, requirements, env=None, installer=None): """List all distributions needed to (recursively) meet `requirements` `requirements` must be a sequence of ``Requirement`` objects. `env`, @@ -570,9 +568,6 @@ class WorkingSet(object): while requirements: req = requirements.pop(0) # process dependencies breadth-first - if _override_setuptools(req) and replacement: - req = Requirement.parse('distribute') - if req in processed: # Ignore cyclic or redundant dependencies continue @@ -692,7 +687,6 @@ class WorkingSet(object): activated to fulfill the requirements; all relevant distributions are included, even if they were already activated in this working set. """ - needed = self.resolve(parse_requirements(requirements)) for dist in needed: @@ -700,7 +694,6 @@ class WorkingSet(object): return needed - def subscribe(self, callback): """Invoke `callback` for all distributions (including existing ones)""" if callback in self.callbacks: @@ -709,14 +702,15 @@ class WorkingSet(object): for dist in self: callback(dist) - def _added_new(self, dist): for callback in self.callbacks: callback(dist) def __getstate__(self): - return (self.entries[:], self.entry_keys.copy(), self.by_key.copy(), - self.callbacks[:]) + return ( + self.entries[:], self.entry_keys.copy(), self.by_key.copy(), + self.callbacks[:] + ) def __setstate__(self, (entries, keys, by_key, callbacks)): self.entries = entries[:] @@ -725,8 +719,6 @@ class WorkingSet(object): self.callbacks = callbacks[:] - - class Environment(object): """Searchable snapshot of distributions on a search path""" @@ -1823,7 +1815,7 @@ def _handle_ns(packageName, path_item): return None module = sys.modules.get(packageName) if module is None: - module = sys.modules[packageName] = types.ModuleType(packageName) + module = sys.modules[packageName] = imp.new_module(packageName) module.__path__ = []; _set_parent_ns(packageName) elif not hasattr(module,'__path__'): raise TypeError("Not a package:", packageName) @@ -2362,26 +2354,12 @@ class Distribution(object): """Insert self.location in path before its nearest parent directory""" loc = loc or self.location - - if self.project_name == 'setuptools': - try: - version = self.version - except ValueError: - version = '' - if '0.7' in version: - raise ValueError( - "A 0.7-series setuptools cannot be installed " - "with distribute. Found one at %s" % str(self.location)) - if not loc: return - if path is sys.path: - self.check_version_conflict() - nloc = _normalize_cached(loc) bdir = os.path.dirname(nloc) - npath= map(_normalize_cached, path) + npath= [(p and _normalize_cached(p) or p) for p in path] bp = None for p, item in enumerate(npath): @@ -2389,10 +2367,14 @@ class Distribution(object): break elif item==bdir and self.precedence==EGG_DIST: # if it's an .egg, give it precedence over its directory + if path is sys.path: + self.check_version_conflict() path.insert(p, loc) npath.insert(p, nloc) break else: + if path is sys.path: + self.check_version_conflict() path.append(loc) return @@ -2409,9 +2391,8 @@ class Distribution(object): return - def check_version_conflict(self): - if self.key=='distribute': + if self.key=='setuptools': return # ignore the inevitable setuptools self-conflicts :( nsp = dict.fromkeys(self._get_metadata('namespace_packages.txt')) @@ -2670,7 +2651,7 @@ class Requirement: def __contains__(self,item): if isinstance(item,Distribution): - if item.key <> self.key: return False + if item.key != self.key: return False if self.index: item = item.parsed_version # only get if we need it elif isinstance(item,basestring): item = parse_version(item) @@ -2692,22 +2673,11 @@ class Requirement: def __repr__(self): return "Requirement.parse(%r)" % str(self) #@staticmethod - def parse(s, replacement=True): + def parse(s): reqs = list(parse_requirements(s)) if reqs: - if len(reqs) == 1: - founded_req = reqs[0] - # if asked for setuptools distribution - # and if distribute is installed, we want to give - # distribute instead - if _override_setuptools(founded_req) and replacement: - distribute = list(parse_requirements('distribute')) - if len(distribute) == 1: - return distribute[0] - return founded_req - else: - return founded_req - + if len(reqs)==1: + return reqs[0] raise ValueError("Expected only one requirement", s) raise ValueError("No requirements found", s) @@ -2724,26 +2694,6 @@ state_machine = { } -def _override_setuptools(req): - """Return True when distribute wants to override a setuptools dependency. - - We want to override when the requirement is setuptools and the version is - a variant of 0.6. - - """ - if req.project_name == 'setuptools': - if not len(req.specs): - # Just setuptools: ok - return True - for comparator, version in req.specs: - if comparator in ['==', '>=', '>']: - if '0.7' in version: - # We want some setuptools not from the 0.6 series. - return False - return True - return False - - def _get_mro(cls): """Get an mro for a type or classic class""" if not isinstance(cls,type): @@ -2809,7 +2759,6 @@ _initialize(globals()) # Prepare the master working set and make the ``require()`` API available _declare_state('object', working_set = WorkingSet()) - try: # Does the main program list any requirements? from __main__ import __requires__ @@ -21,7 +21,7 @@ if sys.version_info >= (3,): manifest_file.close() dir_util.create_tree(tmp_src, fl.files) outfiles_2to3 = [] - dist_script = os.path.join("build", "src", "distribute_setup.py") + dist_script = os.path.join("build", "src", "ez_setup.py") for f in fl.files: outf, copied = file_util.copy_file(f, os.path.join(tmp_src, f), update=1) if copied and outf.endswith(".py") and outf != dist_script: @@ -82,7 +82,7 @@ class build_py(_build_py): class test(_test): """Specific test class to avoid rewriting the entry_points.txt""" def run(self): - entry_points = os.path.join('distribute.egg-info', 'entry_points.txt') + entry_points = os.path.join('setuptools.egg-info', 'entry_points.txt') if not os.path.exists(entry_points): _test.run(self) @@ -107,32 +107,6 @@ class test(_test): f.close() -# if we are installing Distribute using "python setup.py install" -# we need to get setuptools out of the way -def _easy_install_marker(): - return (len(sys.argv) == 5 and sys.argv[2] == 'bdist_egg' and - sys.argv[3] == '--dist-dir' and 'egg-dist-tmp-' in sys.argv[-1]) - -def _buildout_marker(): - command = os.environ.get('_') - if command: - return 'buildout' in os.path.basename(command) - -def _being_installed(): - if os.environ.get('DONT_PATCH_SETUPTOOLS') is not None: - return False - if _buildout_marker(): - # Installed by buildout, don't mess with a global setuptools. - return False - # easy_install marker - if "--help" in sys.argv[1:] or "-h" in sys.argv[1:]: # Don't bother doing anything if they're just asking for help - return False - return 'install' in sys.argv[1:] or _easy_install_marker() - -if _being_installed(): - from distribute_setup import _before_install - _before_install() - readme_file = open('README.txt') # the release script adds hyperlinks to issues if os.path.exists('CHANGES (links).txt'): @@ -145,7 +119,7 @@ readme_file.close() changes_file.close() dist = setup( - name="distribute", + name="setuptools", version=VERSION, description="Easily download, build, install, upgrade, and uninstall " "Python packages", @@ -154,7 +128,7 @@ dist = setup( license="PSF or ZPL", long_description = long_description, keywords = "CPAN PyPI distutils eggs package management", - url = "http://packages.python.org/distribute", + url = "http://pypi.python.org/pypi/setuptools", test_suite = 'setuptools.tests', src_root = src_root, packages = find_packages(), @@ -235,7 +209,3 @@ dist = setup( """).strip().splitlines(), scripts = scripts, ) - -if _being_installed(): - from distribute_setup import _after_install - _after_install(dist) diff --git a/setuptools.egg-info/entry_points.txt b/setuptools.egg-info/entry_points.txt new file mode 100644 index 00000000..663882d6 --- /dev/null +++ b/setuptools.egg-info/entry_points.txt @@ -0,0 +1,62 @@ +[distutils.commands] +bdist_rpm = setuptools.command.bdist_rpm:bdist_rpm +rotate = setuptools.command.rotate:rotate +develop = setuptools.command.develop:develop +setopt = setuptools.command.setopt:setopt +build_py = setuptools.command.build_py:build_py +saveopts = setuptools.command.saveopts:saveopts +egg_info = setuptools.command.egg_info:egg_info +register = setuptools.command.register:register +upload_docs = setuptools.command.upload_docs:upload_docs +install_egg_info = setuptools.command.install_egg_info:install_egg_info +alias = setuptools.command.alias:alias +easy_install = setuptools.command.easy_install:easy_install +install_scripts = setuptools.command.install_scripts:install_scripts +bdist_wininst = setuptools.command.bdist_wininst:bdist_wininst +bdist_egg = setuptools.command.bdist_egg:bdist_egg +install = setuptools.command.install:install +test = setuptools.command.test:test +install_lib = setuptools.command.install_lib:install_lib +build_ext = setuptools.command.build_ext:build_ext +sdist = setuptools.command.sdist:sdist + +[egg_info.writers] +dependency_links.txt = setuptools.command.egg_info:overwrite_arg +requires.txt = setuptools.command.egg_info:write_requirements +PKG-INFO = setuptools.command.egg_info:write_pkg_info +eager_resources.txt = setuptools.command.egg_info:overwrite_arg +top_level.txt = setuptools.command.egg_info:write_toplevel_names +namespace_packages.txt = setuptools.command.egg_info:overwrite_arg +entry_points.txt = setuptools.command.egg_info:write_entries +depends.txt = setuptools.command.egg_info:warn_depends_obsolete + +[console_scripts] +easy_install = setuptools.command.easy_install:main +easy_install-2.7 = setuptools.command.easy_install:main + +[setuptools.file_finders] +svn_cvs = setuptools.command.sdist:_default_revctrl + +[distutils.setup_keywords] +dependency_links = setuptools.dist:assert_string_list +entry_points = setuptools.dist:check_entry_points +extras_require = setuptools.dist:check_extras +use_2to3_exclude_fixers = setuptools.dist:assert_string_list +package_data = setuptools.dist:check_package_data +install_requires = setuptools.dist:check_requirements +use_2to3 = setuptools.dist:assert_bool +use_2to3_fixers = setuptools.dist:assert_string_list +include_package_data = setuptools.dist:assert_bool +exclude_package_data = setuptools.dist:check_package_data +namespace_packages = setuptools.dist:check_nsp +test_suite = setuptools.dist:check_test_suite +eager_resources = setuptools.dist:assert_string_list +zip_safe = setuptools.dist:assert_bool +test_loader = setuptools.dist:check_importable +packages = setuptools.dist:check_packages +convert_2to3_doctests = setuptools.dist:assert_string_list +tests_require = setuptools.dist:check_requirements + +[setuptools.installation] +eggsecutable = setuptools.command.easy_install:bootstrap + diff --git a/setuptools/__init__.py b/setuptools/__init__.py index 9de373f9..414b4b5d 100644 --- a/setuptools/__init__.py +++ b/setuptools/__init__.py @@ -8,22 +8,12 @@ from distutils.util import convert_path import os import sys -__version__ = '0.6' +__version__ = '0.7' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', 'find_packages' ] -# This marker is used to simplify the process that checks is the -# setuptools package was installed by the Setuptools project -# or by the Distribute project, in case Setuptools creates -# a distribution with the same version. -# -# The distribute_setup script for instance, will check if this -# attribute is present to decide whether to reinstall the package -# or not. -_distribute = True - bootstrap_install_from = None # If we run 2to3 on .py files, should we also convert docstrings? @@ -51,7 +41,7 @@ def find_packages(where='.', exclude=()): os.path.isfile(os.path.join(fn,'__init__.py')) ): out.append(prefix+name); stack.append((fn,prefix+name+'.')) - for pat in list(exclude)+['ez_setup', 'distribute_setup']: + for pat in list(exclude)+['ez_setup']: from fnmatch import fnmatchcase out = [item for item in out if not fnmatchcase(item,pat)] return out diff --git a/setuptools/archive_util.py b/setuptools/archive_util.py index e22b25c0..1109f346 100755 --- a/setuptools/archive_util.py +++ b/setuptools/archive_util.py @@ -6,7 +6,7 @@ __all__ = [ "UnrecognizedFormat", "extraction_drivers", "unpack_directory", ] -import zipfile, tarfile, os, shutil +import zipfile, tarfile, os, shutil, posixpath from pkg_resources import ensure_directory from distutils.errors import DistutilsError @@ -138,7 +138,7 @@ def unpack_zipfile(filename, extract_dir, progress_filter=default_filter): name = info.filename # don't extract absolute paths or ones with .. in them - if name.startswith('/') or '..' in name: + if name.startswith('/') or '..' in name.split('/'): continue target = os.path.join(extract_dir, *name.split('/')) @@ -172,43 +172,39 @@ def unpack_tarfile(filename, extract_dir, progress_filter=default_filter): by ``tarfile.open()``). See ``unpack_archive()`` for an explanation of the `progress_filter` argument. """ - try: tarobj = tarfile.open(filename) except tarfile.TarError: raise UnrecognizedFormat( "%s is not a compressed or uncompressed tar file" % (filename,) ) - try: tarobj.chown = lambda *args: None # don't do any chowning! for member in tarobj: name = member.name # don't extract absolute paths or ones with .. in them - if not name.startswith('/') and '..' not in name: + if not name.startswith('/') and '..' not in name.split('/'): prelim_dst = os.path.join(extract_dir, *name.split('/')) - final_dst = progress_filter(name, prelim_dst) - # If progress_filter returns None, then we do not extract - # this file - # TODO: Do we really need to limit to just these file types? - # tarobj.extract() will handle all files on all platforms, - # turning file types that aren't allowed on that platform into - # regular files. - if final_dst and (member.isfile() or member.isdir() or - member.islnk() or member.issym()): - tarobj.extract(member, extract_dir) - if final_dst != prelim_dst: - shutil.move(prelim_dst, final_dst) + + # resolve any links and to extract the link targets as normal files + while member is not None and (member.islnk() or member.issym()): + linkpath = member.linkname + if member.issym(): + linkpath = posixpath.join(posixpath.dirname(member.name), linkpath) + linkpath = posixpath.normpath(linkpath) + member = tarobj._getmember(linkpath) + + if member is not None and (member.isfile() or member.isdir()): + final_dst = progress_filter(name, prelim_dst) + if final_dst: + if final_dst.endswith(os.sep): + final_dst = final_dst[:-1] + try: + tarobj._extract_member(member, final_dst) # XXX Ugh + except tarfile.ExtractError: + pass # chown/chmod/mkfifo/mknode/makedev failed return True finally: tarobj.close() - - - extraction_drivers = unpack_directory, unpack_zipfile, unpack_tarfile - - - - - diff --git a/setuptools/command/alias.py b/setuptools/command/alias.py index f5368b29..40c00b55 100755 --- a/setuptools/command/alias.py +++ b/setuptools/command/alias.py @@ -9,7 +9,7 @@ def shquote(arg): """Quote an argument for later parsing by shlex.split()""" for c in '"', "'", "\\", "#": if c in arg: return repr(arg) - if arg.split()<>[arg]: + if arg.split()!=[arg]: return repr(arg) return arg @@ -33,7 +33,7 @@ class alias(option_base): def finalize_options(self): option_base.finalize_options(self) - if self.remove and len(self.args)<>1: + if self.remove and len(self.args)!=1: raise DistutilsOptionError( "Must specify exactly one argument (the alias name) when " "using --remove" diff --git a/setuptools/command/bdist_egg.py b/setuptools/command/bdist_egg.py index 17fae984..c3356bb7 100644 --- a/setuptools/command/bdist_egg.py +++ b/setuptools/command/bdist_egg.py @@ -170,12 +170,13 @@ class bdist_egg(Command): def run(self): # Generate metadata first self.run_command("egg_info") - # We run install_lib before install_data, because some data hacks # pull their data path from the install_lib command. log.info("installing library code to %s" % self.bdist_dir) instcmd = self.get_finalized_command('install') old_root = instcmd.root; instcmd.root = None + if self.distribution.has_c_libraries() and not self.skip_build: + self.run_command('build_clib') cmd = self.call_command('install_lib', warn_dir=0) instcmd.root = old_root @@ -195,7 +196,6 @@ class bdist_egg(Command): to_compile.extend(self.make_init_files()) if to_compile: cmd.byte_compile(to_compile) - if self.distribution.data_files: self.do_install_data() @@ -407,7 +407,7 @@ def write_safety_flag(egg_dir, safe): for flag,fn in safety_flags.items(): fn = os.path.join(egg_dir, fn) if os.path.exists(fn): - if safe is None or bool(safe)<>flag: + if safe is None or bool(safe)!=flag: os.unlink(fn) elif safe is not None and bool(safe)==flag: f=open(fn,'wt'); f.write('\n'); f.close() diff --git a/setuptools/command/bdist_wininst.py b/setuptools/command/bdist_wininst.py index 93e6846d..e8521f83 100755 --- a/setuptools/command/bdist_wininst.py +++ b/setuptools/command/bdist_wininst.py @@ -2,26 +2,24 @@ from distutils.command.bdist_wininst import bdist_wininst as _bdist_wininst import os, sys class bdist_wininst(_bdist_wininst): + _good_upload = _bad_upload = None def create_exe(self, arcname, fullname, bitmap=None): _bdist_wininst.create_exe(self, arcname, fullname, bitmap) - dist_files = getattr(self.distribution, 'dist_files', []) - + installer_name = self.get_installer_filename(fullname) if self.target_version: - installer_name = os.path.join(self.dist_dir, - "%s.win32-py%s.exe" % - (fullname, self.target_version)) pyversion = self.target_version - - # fix 2.5 bdist_wininst ignoring --target-version spec - bad = ('bdist_wininst','any',installer_name) - if bad in dist_files: - dist_files.remove(bad) + # fix 2.5+ bdist_wininst ignoring --target-version spec + self._bad_upload = ('bdist_wininst', 'any', installer_name) else: - installer_name = os.path.join(self.dist_dir, - "%s.win32.exe" % fullname) pyversion = 'any' - good = ('bdist_wininst', pyversion, installer_name) + self._good_upload = ('bdist_wininst', pyversion, installer_name) + + def _fix_upload_names(self): + good, bad = self._good_upload, self._bad_upload + dist_files = getattr(self.distribution, 'dist_files', []) + if bad in dist_files: + dist_files.remove(bad) if good not in dist_files: dist_files.append(good) @@ -36,6 +34,49 @@ class bdist_wininst(_bdist_wininst): self._is_running = True try: _bdist_wininst.run(self) + self._fix_upload_names() finally: self._is_running = False + + if not hasattr(_bdist_wininst, 'get_installer_filename'): + def get_installer_filename(self, fullname): + # Factored out to allow overriding in subclasses + if self.target_version: + # if we create an installer for a specific python version, + # it's better to include this in the name + installer_name = os.path.join(self.dist_dir, + "%s.win32-py%s.exe" % + (fullname, self.target_version)) + else: + installer_name = os.path.join(self.dist_dir, + "%s.win32.exe" % fullname) + return installer_name + # get_installer_filename() + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/setuptools/command/build_ext.py b/setuptools/command/build_ext.py index 4a94572c..f2a53258 100644 --- a/setuptools/command/build_ext.py +++ b/setuptools/command/build_ext.py @@ -82,17 +82,15 @@ class build_ext(_build_ext): def get_ext_filename(self, fullname): filename = _build_ext.get_ext_filename(self,fullname) - if fullname not in self.ext_map: - return filename - ext = self.ext_map[fullname] - if isinstance(ext,Library): - fn, ext = os.path.splitext(filename) - return self.shlib_compiler.library_filename(fn,libtype) - elif use_stubs and ext._links_to_dynamic: - d,fn = os.path.split(filename) - return os.path.join(d,'dl-'+fn) - else: - return filename + if fullname in self.ext_map: + ext = self.ext_map[fullname] + if isinstance(ext,Library): + fn, ext = os.path.splitext(filename) + return self.shlib_compiler.library_filename(fn,libtype) + elif use_stubs and ext._links_to_dynamic: + d,fn = os.path.split(filename) + return os.path.join(d,'dl-'+fn) + return filename def initialize_options(self): _build_ext.initialize_options(self) diff --git a/setuptools/command/easy_install.py b/setuptools/command/easy_install.py index 10a8f272..4cd058bb 100755 --- a/setuptools/command/easy_install.py +++ b/setuptools/command/easy_install.py @@ -7,7 +7,7 @@ A tool for doing automatic download/extract/build of distutils-based Python packages. For detailed documentation, see the accompanying EasyInstall.txt file, or visit the `EasyInstall home page`__. -__ http://packages.python.org/distribute/easy_install.html +__ http://packages.python.org/setuptools/easy_install.html """ import sys @@ -202,7 +202,7 @@ class easy_install(Command): def finalize_options(self): if self.version: - print 'distribute %s' % get_distribution('distribute').version + print 'setuptools %s' % get_distribution('setuptools').version sys.exit() py_version = sys.version.split()[0] @@ -402,7 +402,7 @@ class easy_install(Command): # Is it a configured, PYTHONPATH, implicit, or explicit site dir? is_site_dir = instdir in self.all_site_dirs - if not is_site_dir: + if not is_site_dir and not self.multi_version: # No? Then directly test whether it does .pth file processing is_site_dir = self.check_pth_processing() else: @@ -467,7 +467,7 @@ variable. For information on other options, you may wish to consult the documentation at: - http://packages.python.org/distribute/easy_install.html + http://packages.python.org/setuptools/easy_install.html Please make the appropriate changes for your system and try again. """ @@ -590,7 +590,6 @@ Please make the appropriate changes for your system and try again. spec, tmpdir, self.upgrade, self.editable, not self.always_copy, self.local_index ) - if dist is None: msg = "Could not find suitable distribution for %r" % spec if self.always_copy: @@ -661,8 +660,7 @@ Please make the appropriate changes for your system and try again. self.update_pth(dist) self.package_index.add(dist) self.local_index.add(dist) - if not self.editable: - self.install_egg_scripts(dist) + self.install_egg_scripts(dist) self.installed_projects[dist.key] = dist log.info(self.installation_report(requirement, dist, *info)) if (dist.has_metadata('dependency_links.txt') and @@ -710,7 +708,7 @@ Please make the appropriate changes for your system and try again. return True if not dist.has_metadata('zip-safe'): return True - return True + return False def maybe_move(self, spec, dist_filename, setup_base): dst = os.path.join(self.build_directory, spec.key) @@ -896,7 +894,7 @@ Please make the appropriate changes for your system and try again. f = open(pkg_inf,'w') f.write('Metadata-Version: 1.0\n') for k,v in cfg.items('metadata'): - if k<>'target_version': + if k!='target_version': f.write('%s: %s\n' % (k.replace('_','-').title(), v)) f.close() script_dir = os.path.join(egg_info,'scripts') @@ -1170,7 +1168,8 @@ See the setuptools documentation for the "develop" command for more info. if not self.dry_run: self.pth_file.save() - if dist.key=='distribute': + + if dist.key=='setuptools': # Ensure that setuptools itself never becomes unavailable! # XXX should this check for latest version? filename = os.path.join(self.install_dir,'setuptools.pth') @@ -1190,7 +1189,6 @@ See the setuptools documentation for the "develop" command for more info. def pf(src,dst): if dst.endswith('.py') and not src.startswith('EGG-INFO/'): to_compile.append(dst) - to_chmod.append(dst) elif dst.endswith('.dll') or dst.endswith('.so'): to_chmod.append(dst) self.unpack_progress(src,dst) @@ -1229,6 +1227,7 @@ See the setuptools documentation for the "develop" command for more info. + def no_default_version_msg(self): return """bad install directory or PYTHONPATH @@ -1255,7 +1254,7 @@ Here are some of your options for correcting the problem: * You can set up the installation directory to support ".pth" files by using one of the approaches described here: - http://packages.python.org/distribute/easy_install.html#custom-installation-locations + http://packages.python.org/setuptools/easy_install.html#custom-installation-locations Please make the appropriate changes for your system and try again.""" % ( self.install_dir, os.environ.get('PYTHONPATH','') @@ -1277,7 +1276,7 @@ Please make the appropriate changes for your system and try again.""" % ( return # already did it, or don't need to sitepy = os.path.join(self.install_dir, "site.py") - source = resource_string(Requirement.parse("distribute"), "site.py") + source = resource_string(Requirement.parse("setuptools"), "site.py") current = "" if os.path.exists(sitepy): @@ -1512,7 +1511,7 @@ def get_exe_prefixes(exe_filename): ('PURELIB/', ''), ('PLATLIB/pywin32_system32', ''), ('PLATLIB/', ''), ('SCRIPTS/', 'EGG-INFO/scripts/'), - ('DATA/LIB/site-packages', ''), + ('DATA/lib/site-packages', ''), ] z = zipfile.ZipFile(exe_filename) try: @@ -1523,7 +1522,7 @@ def get_exe_prefixes(exe_filename): if parts[1].endswith('.egg-info'): prefixes.insert(0,('/'.join(parts[:2]), 'EGG-INFO/')) break - if len(parts)<>2 or not name.endswith('.pth'): + if len(parts)!=2 or not name.endswith('.pth'): continue if name.endswith('-nspkg.pth'): continue @@ -1846,7 +1845,7 @@ def get_script_args(dist, executable=sys_executable, wininst=False): launcher = launcher.replace(".", "-64.") else: launcher = launcher.replace(".", "-32.") - if os.path.exists(new_header[2:-1]) or sys.platform!='win32': + if os.path.exists(new_header[2:-1].strip('"')) or sys.platform!='win32': hdr = new_header else: hdr = header @@ -1948,12 +1947,6 @@ usage: %(script)s [options] requirement_or_url ... def _show_help(self,*args,**kw): with_ei_usage(lambda: Distribution._show_help(self,*args,**kw)) - def find_config_files(self): - files = Distribution.find_config_files(self) - if 'setup.cfg' in files: - files.remove('setup.cfg') - return files - if argv is None: argv = sys.argv[1:] diff --git a/setuptools/command/egg_info.py b/setuptools/command/egg_info.py index 0c2ea0cc..cd3ea198 100755 --- a/setuptools/command/egg_info.py +++ b/setuptools/command/egg_info.py @@ -224,18 +224,21 @@ class egg_info(Command): data = f.read() f.close() - if data.startswith('10') or data.startswith('9') or data.startswith('8'): + if data.startswith('<?xml'): + dirurl = urlre.search(data).group(1) # get repository URL + localrev = max([int(m.group(1)) for m in revre.finditer(data)]+[0]) + else: + try: svnver = int(data.splitlines()[0]) + except: svnver=-1 + if data<8: + log.warn("unrecognized .svn/entries format; skipping %s", base) + dirs[:] = [] + continue + data = map(str.splitlines,data.split('\n\x0c\n')) del data[0][0] # get rid of the '8' or '9' or '10' dirurl = data[0][3] localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]+[0]) - elif data.startswith('<?xml'): - dirurl = urlre.search(data).group(1) # get repository URL - localrev = max([int(m.group(1)) for m in revre.finditer(data)]+[0]) - else: - log.warn("unrecognized .svn/entries format; skipping %s", base) - dirs[:] = [] - continue if base==os.curdir: base_url = dirurl+'/' # save the root url elif not dirurl.startswith(base_url): @@ -248,9 +251,6 @@ class egg_info(Command): - - - def find_sources(self): """Generate SOURCES.txt manifest file""" manifest_filename = os.path.join(self.egg_info,"SOURCES.txt") diff --git a/setuptools/command/sdist.py b/setuptools/command/sdist.py index 2fa3771a..f8f964b3 100755 --- a/setuptools/command/sdist.py +++ b/setuptools/command/sdist.py @@ -1,6 +1,7 @@ from distutils.command.sdist import sdist as _sdist from distutils.util import convert_path from distutils import log +from glob import glob import os, re, sys, pkg_resources from glob import glob @@ -41,7 +42,6 @@ def joinpath(prefix,suffix): - def walk_revctrl(dirname=''): """Find all files under revision control""" for ep in pkg_resources.iter_entry_points('setuptools.file_finders'): @@ -89,18 +89,22 @@ def entries_finder(dirname, filename): f = open(filename,'rU') data = f.read() f.close() - if data.startswith('10') or data.startswith('9') or data.startswith('8'): + if data.startswith('<?xml'): + for match in entries_pattern.finditer(data): + yield joinpath(dirname,unescape(match.group(1))) + else: + svnver=-1 + try: svnver = int(data.splitlines()[0]) + except: pass + if svnver<8: + log.warn("unrecognized .svn/entries format in %s", os.path.abspath(dirname)) + return for record in map(str.splitlines, data.split('\n\x0c\n')[1:]): # subversion 1.6/1.5/1.4 if not record or len(record)>=6 and record[5]=="delete": continue # skip deleted yield joinpath(dirname, record[0]) - elif data.startswith('<?xml'): - for match in entries_pattern.finditer(data): - yield joinpath(dirname,unescape(match.group(1))) - else: - log.warn("unrecognized .svn/entries format in %s", os.path.abspath(dirname)) - + finders = [ (convert_path('CVS/Entries'), @@ -121,10 +125,6 @@ finders = [ - - - - class sdist(_sdist): """Smart sdist that finds anything supported by revision control""" @@ -157,7 +157,7 @@ class sdist(_sdist): import distutils.command if 'check' not in distutils.command.__all__: self.check_metadata() - + self.make_distribution() dist_files = getattr(self.distribution,'dist_files',[]) @@ -166,6 +166,26 @@ class sdist(_sdist): if data not in dist_files: dist_files.append(data) + def __read_template_hack(self): + # This grody hack closes the template file (MANIFEST.in) if an + # exception occurs during read_template. + # Doing so prevents an error when easy_install attempts to delete the + # file. + try: + _sdist.read_template(self) + except: + sys.exc_info()[2].tb_next.tb_frame.f_locals['template'].close() + raise + # Beginning with Python 2.7.2, 3.1.4, and 3.2.1, this leaky file handle + # has been fixed, so only override the method if we're using an earlier + # Python. + if ( + sys.version_info < (2,7,2) + or (3,0) <= sys.version_info < (3,1,4) + or (3,2) <= sys.version_info < (3,2,1) + ): + read_template = __read_template_hack + def add_defaults(self): standards = [READMES, self.distribution.script_name] @@ -219,26 +239,6 @@ class sdist(_sdist): build_scripts = self.get_finalized_command('build_scripts') self.filelist.extend(build_scripts.get_source_files()) - def __read_template_hack(self): - # This grody hack closes the template file (MANIFEST.in) if an - # exception occurs during read_template. - # Doing so prevents an error when easy_install attempts to delete the - # file. - try: - _sdist.read_template(self) - except: - sys.exc_info()[2].tb_next.tb_frame.f_locals['template'].close() - raise - # Beginning with Python 2.7.2, 3.1.4, and 3.2.1, this leaky file handle - # has been fixed, so only override the method if we're using an earlier - # Python. - if ( - sys.version_info < (2,7,2) - or (3,0) <= sys.version_info < (3,1,4) - or (3,2) <= sys.version_info < (3,2,1) - ): - read_template = __read_template_hack - def check_readme(self): for f in READMES: if os.path.exists(f): @@ -301,13 +301,4 @@ class sdist(_sdist): - - - - - - - - - # diff --git a/setuptools/depends.py b/setuptools/depends.py index 4b7b3437..5fdf2d7e 100644 --- a/setuptools/depends.py +++ b/setuptools/depends.py @@ -36,7 +36,7 @@ class Require: def version_ok(self,version): """Is 'version' sufficiently up-to-date?""" return self.attribute is None or self.format is None or \ - str(version)<>"unknown" and version >= self.requested_version + str(version)!="unknown" and version >= self.requested_version def get_version(self, paths=None, default="unknown"): diff --git a/setuptools/dist.py b/setuptools/dist.py index 998a4dbe..89208da8 100644 --- a/setuptools/dist.py +++ b/setuptools/dist.py @@ -62,8 +62,8 @@ def check_nsp(dist, attr, value): parent = '.'.join(nsp.split('.')[:-1]) if parent not in value: distutils.log.warn( - "%r is declared as a package namespace, but %r is not:" - " please correct this in setup.py", nsp, parent + "WARNING: %r is declared as a package namespace, but %r" + " is not: please correct this in setup.py", nsp, parent ) def check_extras(dist, attr, value): @@ -122,6 +122,47 @@ def check_package_data(dist, attr, value): "wildcard patterns" ) +def check_packages(dist, attr, value): + for pkgname in value: + if not re.match(r'\w+(\.\w+)*', pkgname): + distutils.log.warn( + "WARNING: %r not a valid package name; please use only" + ".-separated package names in setup.py", pkgname + ) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + class Distribution(_Distribution): """Distribution with support for features, tests, and package data @@ -845,11 +886,4 @@ class Feature: -def check_packages(dist, attr, value): - for pkgname in value: - if not re.match(r'\w+(\.\w+)*', pkgname): - distutils.log.warn( - "WARNING: %r not a valid package name; please use only" - ".-separated package names in setup.py", pkgname - ) diff --git a/setuptools/package_index.py b/setuptools/package_index.py index 0ee21e3b..3a6b6fac 100755 --- a/setuptools/package_index.py +++ b/setuptools/package_index.py @@ -1,7 +1,7 @@ """PyPI and direct package downloading""" -import sys, os.path, re, urlparse, urllib, urllib2, shutil, random, socket, cStringIO +import sys, os.path, re, urlparse, urllib2, shutil, random, socket, cStringIO import base64 -import httplib +import httplib, urllib from pkg_resources import * from distutils import log from distutils.errors import DistutilsError @@ -10,7 +10,6 @@ try: except ImportError: from md5 import md5 from fnmatch import translate - EGG_FRAGMENT = re.compile(r'^egg=([-A-Za-z0-9_.]+)$') HREF = re.compile("""href\\s*=\\s*['"]?([^'"> ]+)""", re.I) # this is here to fix emacs' cruddy broken syntax highlighting @@ -55,6 +54,8 @@ def parse_bdist_wininst(name): def egg_info_for_url(url): scheme, server, path, parameters, query, fragment = urlparse.urlparse(url) base = urllib2.unquote(path.split('/')[-1]) + if server=='sourceforge.net' and base=='download': # XXX Yuck + base = urllib2.unquote(path.split('/')[-2]) if '#' in base: base, fragment = base.split('#',1) return base,fragment @@ -77,14 +78,12 @@ def distros_for_location(location, basename, metadata=None): if basename.endswith('.egg') and '-' in basename: # only one, unambiguous interpretation return [Distribution.from_location(location, basename, metadata)] - if basename.endswith('.exe'): win_base, py_ver, platform = parse_bdist_wininst(basename) if win_base is not None: return interpret_distro_name( location, win_base, metadata, py_ver, BINARY_DIST, platform ) - # Try source distro extensions (.zip, .tgz, etc.) # for ext in EXTENSIONS: @@ -154,8 +153,8 @@ def find_external_links(url, page): if match: yield urlparse.urljoin(url, htmldecode(match.group(1))) -user_agent = "Python-urllib/%s distribute/%s" % ( - sys.version[:3], require('distribute')[0].version +user_agent = "Python-urllib/%s setuptools/%s" % ( + sys.version[:3], require('setuptools')[0].version ) @@ -199,10 +198,10 @@ class PackageIndex(Environment): return self.info("Reading %s", url) + self.fetched_urls[url] = True # prevent multiple fetch attempts f = self.open_url(url, "Download error on %s: %%s -- Some packages may not be found!" % url) if f is None: return - self.fetched_urls[url] = self.fetched_urls[f.url] = True - + self.fetched_urls[f.url] = True if 'html' not in f.headers.get('content-type', '').lower(): f.close() # not html, we can't process it return @@ -352,7 +351,7 @@ class PackageIndex(Environment): def check_md5(self, cs, info, filename, tfp): if re.match('md5=[0-9a-f]{32}$', info): self.debug("Validating md5 checksum for %s", filename) - if cs.hexdigest()<>info[4:]: + if cs.hexdigest()!=info[4:]: tfp.close() os.unlink(filename) raise DistutilsError( @@ -451,7 +450,6 @@ class PackageIndex(Environment): set, development and system eggs (i.e., those using the ``.egg-info`` format) will be ignored. """ - # process a Requirement self.info("Searching for %s", requirement) skipped = {} @@ -471,10 +469,9 @@ class PackageIndex(Environment): continue if dist in req and (dist.precedence<=SOURCE_DIST or not source): - self.info("Best match: %s", dist) - return dist.clone( - location=self.download(dist.location, tmpdir) - ) + return dist + + if force_scan: self.prescan() @@ -498,7 +495,10 @@ class PackageIndex(Environment): (source and "a source distribution of " or ""), requirement, ) - return dist + else: + self.info("Best match: %s", dist) + return dist.clone(location=self.download(dist.location, tmpdir)) + def fetch(self, requirement, tmpdir, force_scan=False, source=False): """Obtain a file suitable for fulfilling `requirement` @@ -514,12 +514,6 @@ class PackageIndex(Environment): return None - - - - - - def gen_setup(self, filename, fragment, tmpdir): match = EGG_FRAGMENT.match(fragment) dists = match and [d for d in @@ -581,8 +575,7 @@ class PackageIndex(Environment): size = -1 if "content-length" in headers: # Some servers return multiple Content-Length headers :( - content_length = headers.get("Content-Length") - size = int(content_length) + size = max(map(int,headers.getheaders("Content-Length"))) self.reporthook(url, filename, blocknum, bs, size) tfp = open(filename,'wb') while True: @@ -640,9 +633,8 @@ class PackageIndex(Environment): def _download_url(self, scheme, url, tmpdir): # Determine download filename # - name = filter(None,urlparse.urlparse(url)[2].split('/')) + name, fragment = egg_info_for_url(url) if name: - name = name[-1] while '..' in name: name = name.replace('..','.').replace('\\','_') else: @@ -668,7 +660,6 @@ class PackageIndex(Environment): return self._attempt_download(url, filename) - def scan_url(self, url): self.process_url(url, True) @@ -695,10 +686,39 @@ class PackageIndex(Environment): os.unlink(filename) raise DistutilsError("Unexpected HTML page found at "+url) + + + + + + + + + + + + + + + def _download_svn(self, url, filename): url = url.split('#',1)[0] # remove any fragment for svn's sake + creds = '' + if url.lower().startswith('svn:') and '@' in url: + scheme, netloc, path, p, q, f = urlparse.urlparse(url) + if not netloc and path.startswith('//') and '/' in path[2:]: + netloc, path = path[2:].split('/',1) + auth, host = urllib.splituser(netloc) + if auth: + if ':' in auth: + user, pw = auth.split(':',1) + creds = " --username=%s --password=%s" % (user, pw) + else: + creds = " --username="+auth + netloc = host + url = urlparse.urlunparse((scheme, netloc, url, p, q, f)) self.info("Doing subversion checkout from %s to %s", url, filename) - os.system("svn checkout -q %s %s" % (url, filename)) + os.system("svn checkout%s -q %s %s" % (creds, url, filename)) return filename def _vcs_split_rev_from_url(self, url, pop_prefix=False): @@ -759,6 +779,18 @@ class PackageIndex(Environment): def warn(self, msg, *args): log.warn(msg, *args) + + + + + + + + + + + + # This pattern matches a character entity reference (a decimal numeric # references, a hexadecimal numeric reference, or a named reference). entity_sub = re.compile(r'&(#(\d+|x[\da-fA-F]+)|[\w.:-]+);?').sub @@ -838,7 +870,7 @@ def open_with_auth(url): raise httplib.InvalidURL("nonnumeric port: ''") if scheme in ('http', 'https'): - auth, host = urllib2.splituser(netloc) + auth, host = urllib.splituser(netloc) else: auth = None diff --git a/setuptools/sandbox.py b/setuptools/sandbox.py index 1583b81f..e026ff13 100755 --- a/setuptools/sandbox.py +++ b/setuptools/sandbox.py @@ -9,9 +9,40 @@ except NameError: _file = None _open = open from distutils.errors import DistutilsError +from pkg_resources import working_set + __all__ = [ "AbstractSandbox", "DirectorySandbox", "SandboxViolation", "run_setup", ] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + def run_setup(setup_script, args): """Run a distutils setup script, sandboxed in its directory""" old_dir = os.getcwd() @@ -29,6 +60,9 @@ def run_setup(setup_script, args): try: sys.argv[:] = [setup_script]+list(args) sys.path.insert(0, setup_dir) + # reset to include setup dir, w/clean callback list + working_set.__init__() + working_set.callbacks.append(lambda dist:dist.activate()) DirectorySandbox(setup_dir).run( lambda: execfile( "setup.py", @@ -55,6 +89,8 @@ def run_setup(setup_script, args): sys.argv[:] = save_argv tempfile.tempdir = save_tmp + + class AbstractSandbox: """Wrap 'os' module and 'open()' builtin for virtualizing setup scripts""" @@ -86,7 +122,6 @@ class AbstractSandbox: __builtin__.open = _open self._copy(_os) - def _mk_dual_path_wrapper(name): original = getattr(_os,name) def wrap(self,src,dst,*args,**kw): @@ -95,7 +130,6 @@ class AbstractSandbox: return original(src,dst,*args,**kw) return wrap - for name in ["rename", "link", "symlink"]: if hasattr(_os,name): locals()[name] = _mk_dual_path_wrapper(name) @@ -118,7 +152,6 @@ class AbstractSandbox: ]: if hasattr(_os,name): locals()[name] = _mk_single_path_wrapper(name) - def _mk_single_with_return(name): original = getattr(_os,name) def wrap(self,path,*args,**kw): @@ -208,6 +241,7 @@ class DirectorySandbox(AbstractSandbox): self._violation("tmpnam") def _ok(self,path): + if hasattr(_os,'devnull') and path==_os.devnull: return True active = self._active try: self._active = False @@ -240,16 +274,11 @@ class DirectorySandbox(AbstractSandbox): self._violation("os.open", file, flags, mode) return _os.open(file,flags,mode) - WRITE_FLAGS = reduce( - operator.or_, - [getattr(_os, a, 0) for a in + operator.or_, [getattr(_os, a, 0) for a in "O_WRONLY O_RDWR O_APPEND O_CREAT O_TRUNC O_TEMPORARY".split()] ) - - - class SandboxViolation(DistutilsError): """A setup script attempted to modify the filesystem outside the sandbox""" diff --git a/setuptools/tests/test_easy_install.py b/setuptools/tests/test_easy_install.py index 582219ce..395056e7 100644 --- a/setuptools/tests/test_easy_install.py +++ b/setuptools/tests/test_easy_install.py @@ -13,7 +13,7 @@ import StringIO import distutils.core from setuptools.sandbox import run_setup, SandboxViolation -from setuptools.command.easy_install import easy_install, fix_jython_executable, get_script_args, main +from setuptools.command.easy_install import easy_install, fix_jython_executable, get_script_args from setuptools.command.easy_install import PthDistributions from setuptools.command import easy_install as easy_install_pkg from setuptools.dist import Distribution @@ -84,41 +84,6 @@ class TestEasyInstallTest(unittest.TestCase): self.assertEqual(script, WANTED) - def test_no_setup_cfg(self): - # makes sure easy_install as a command (main) - # doesn't use a setup.cfg file that is located - # in the current working directory - dir = tempfile.mkdtemp() - setup_cfg = open(os.path.join(dir, 'setup.cfg'), 'w') - setup_cfg.write('[easy_install]\nfind_links = http://example.com') - setup_cfg.close() - setup_py = open(os.path.join(dir, 'setup.py'), 'w') - setup_py.write(SETUP_PY) - setup_py.close() - - from setuptools.dist import Distribution - - def _parse_command_line(self): - msg = 'Error: a local setup.cfg was used' - opts = self.command_options - if 'easy_install' in opts: - assert 'find_links' not in opts['easy_install'], msg - return self._old_parse_command_line() - - Distribution._old_parse_command_line = Distribution.parse_command_line - Distribution.parse_command_line = _parse_command_line - - old_wd = os.getcwd() - try: - os.chdir(dir) - reset_setup_stop_context( - lambda: self.assertRaises(SystemExit, main, []) - ) - finally: - os.chdir(old_wd) - shutil.rmtree(dir) - Distribution.parse_command_line = Distribution._old_parse_command_line - def test_no_find_links(self): # new option '--no-find-links', that blocks find-links added at # the project level @@ -266,10 +231,10 @@ class TestUserInstallTest(unittest.TestCase): del os.environ['PYTHONPATH'] def test_setup_requires(self): - """Regression test for issue #318 + """Regression test for Distribute issue #318 - Ensures that a package with setup_requires can be installed when - distribute is installed in the user site-packages without causing a + Ensure that a package with setup_requires can be installed when + setuptools is installed in the user site-packages without causing a SandboxViolation. """ @@ -373,13 +338,13 @@ class TestSetupRequires(unittest.TestCase): doesn't exist) and invoke installer on it. """ def build_sdist(dir): - dist_path = os.path.join(dir, 'distribute-test-fetcher-1.0.tar.gz') + dist_path = os.path.join(dir, 'setuptools-test-fetcher-1.0.tar.gz') make_trivial_sdist( dist_path, textwrap.dedent(""" import setuptools setuptools.setup( - name="distribute-test-fetcher", + name="setuptools-test-fetcher", version="1.0", setup_requires = ['does-not-exist'], ) @@ -447,7 +412,7 @@ def argv_context(f, repl): def reset_setup_stop_context(f): """ - When the distribute tests are run using setup.py test, and then + When the setuptools tests are run using setup.py test, and then one wants to invoke another setup() command (such as easy_install) within those tests, it's necessary to reset the global variable in distutils.core so that the setup() command will run naturally. diff --git a/setuptools/tests/test_packageindex.py b/setuptools/tests/test_packageindex.py index 3e446b54..9c4bfadb 100644 --- a/setuptools/tests/test_packageindex.py +++ b/setuptools/tests/test_packageindex.py @@ -110,7 +110,7 @@ class TestPackageIndex(unittest.TestCase): - someone reuploads the package (with a different md5) - while easy_installing, an MD5 error occurs because the external link is used - -> Distribute should use the link from pypi, not the external one. + -> Setuptools should use the link from pypi, not the external one. """ if sys.platform.startswith('java'): # Skip this test on jython because binding to :0 fails diff --git a/setuptools/tests/test_resources.py b/setuptools/tests/test_resources.py index 292b78d1..9c2ed9c9 100644 --- a/setuptools/tests/test_resources.py +++ b/setuptools/tests/test_resources.py @@ -197,29 +197,6 @@ class DistroTests(TestCase): ) self.assertRaises(UnknownExtra, d.requires, ["foo"]) - def testSetuptoolsDistributeCombination(self): - # Ensure that installing a 0.7-series setuptools fails. PJE says that - # it will not co-exist. - ws = WorkingSet([]) - d = Distribution( - "/some/path", - project_name="setuptools", - version="0.7a1") - self.assertRaises(ValueError, ws.add, d) - # A 0.6-series is no problem - d2 = Distribution( - "/some/path", - project_name="setuptools", - version="0.6c9") - ws.add(d2) - - # a unexisting version needs to work - ws = WorkingSet([]) - d3 = Distribution( - "/some/path", - project_name="setuptools") - ws.add(d3) - class EntryPointTests(TestCase): @@ -372,21 +349,13 @@ class RequirementsTests(TestCase): self.assertTrue(d("foo-0.3a3.egg") in r2) self.assertTrue(d("foo-0.3a5.egg") in r2) - def testDistributeSetuptoolsOverride(self): - # Plain setuptools or distribute mean we return distribute. - self.assertEqual( - Requirement.parse('setuptools').project_name, 'distribute') - self.assertEqual( - Requirement.parse('distribute').project_name, 'distribute') - # setuptools lower than 0.7 means distribute - self.assertEqual( - Requirement.parse('setuptools==0.6c9').project_name, 'distribute') - self.assertEqual( - Requirement.parse('setuptools==0.6c10').project_name, 'distribute') - self.assertEqual( - Requirement.parse('setuptools>=0.6').project_name, 'distribute') + def testSetuptoolsProjectName(self): + """ + The setuptools project should implement the setuptools package. + """ + self.assertEqual( - Requirement.parse('setuptools < 0.7').project_name, 'distribute') + Requirement.parse('setuptools').project_name, 'setuptools') # setuptools 0.7 and higher means setuptools. self.assertEqual( Requirement.parse('setuptools == 0.7').project_name, 'setuptools') @@ -590,7 +559,7 @@ class NamespaceTests(TestCase): def setUp(self): self._ns_pkgs = pkg_resources._namespace_packages.copy() - self._tmpdir = tempfile.mkdtemp(prefix="tests-distribute-") + self._tmpdir = tempfile.mkdtemp(prefix="tests-setuptools-") os.makedirs(os.path.join(self._tmpdir, "site-pkgs")) self._prev_sys_path = sys.path[:] sys.path.append(os.path.join(self._tmpdir, "site-pkgs")) @@ -634,7 +603,7 @@ class NamespaceTests(TestCase): try: import pkg1.pkg2 except ImportError, e: - self.fail("Distribute tried to import the parent namespace package") + self.fail("Setuptools tried to import the parent namespace package") # check the _namespace_packages dict self._assertIn("pkg1.pkg2", pkg_resources._namespace_packages.keys()) self.assertEqual(pkg_resources._namespace_packages["pkg1"], ["pkg1.pkg2"]) diff --git a/tests/install_test.py b/tests/install_test.py deleted file mode 100644 index 02deb818..00000000 --- a/tests/install_test.py +++ /dev/null @@ -1,75 +0,0 @@ -import urllib2 -import sys -import os - -if os.path.exists('distribute_setup.py'): - print 'distribute_setup.py exists in the current dir, aborting' - sys.exit(2) - -print '**** Starting Test' -print '\n\n' - -is_jython = sys.platform.startswith('java') -if is_jython: - import subprocess - -print 'Downloading bootstrap' -file = urllib2.urlopen('http://nightly.ziade.org/distribute_setup.py') -f = open('distribute_setup.py', 'w') -f.write(file.read()) -f.close() - -# running it -args = [sys.executable] + ['distribute_setup.py'] -if is_jython: - res = subprocess.call(args) -else: - res = os.spawnv(os.P_WAIT, sys.executable, args) - -if res != 0: - print '**** Test failed, please send me the output at tarek@ziade.org' - os.remove('distribute_setup.py') - sys.exit(2) - -# now checking if Distribute is installed -script = """\ -import sys -try: - import setuptools -except ImportError: - sys.exit(0) - -sys.exit(hasattr(setuptools, "_distribute")) -""" - -root = 'script' -seed = 0 -script_name = '%s%d.py' % (root, seed) - -while os.path.exists(script_name): - seed += 1 - script_name = '%s%d.py' % (root, seed) - -f = open(script_name, 'w') -try: - f.write(script) -finally: - f.close() - -try: - args = [sys.executable] + [script_name] - if is_jython: - res = subprocess.call(args) - else: - res = os.spawnv(os.P_WAIT, sys.executable, args) - - print '\n\n' - if res: - print '**** Test is OK' - else: - print '**** Test failed, please send me the output at tarek@ziade.org' -finally: - if os.path.exists(script_name): - os.remove(script_name) - os.remove('distribute_setup.py') - diff --git a/tests/manual_test.py b/tests/manual_test.py index 0d5051f1..44cf2d06 100644 --- a/tests/manual_test.py +++ b/tests/manual_test.py @@ -51,9 +51,8 @@ eggs = extensions """ -BOOTSTRAP = 'http://python-distribute.org/bootstrap.py' +BOOTSTRAP = 'http://downloads.buildout.org/1/bootstrap.py' PYVER = sys.version.split()[0][:3] -DEV_URL = 'http://bitbucket.org/tarek/distribute/get/0.6-maintenance.zip#egg=distribute-dev' _VARS = {'base': '.', 'py_version_short': PYVER} @@ -66,26 +65,25 @@ else: @tempdir def test_virtualenv(): - """virtualenv with distribute""" + """virtualenv with setuptools""" purelib = os.path.abspath(Template(PURELIB).substitute(**_VARS)) - _system_call('virtualenv', '--no-site-packages', '.', '--distribute') - _system_call('bin/easy_install', 'distribute==dev') + _system_call('virtualenv', '--no-site-packages', '.') + _system_call('bin/easy_install', 'setuptools==dev') # linux specific site_pkg = os.listdir(purelib) site_pkg.sort() - assert 'distribute' in site_pkg[0] + assert 'setuptools' in site_pkg[0] easy_install = os.path.join(purelib, 'easy-install.pth') with open(easy_install) as f: res = f.read() - assert 'distribute' in res - assert 'setuptools' not in res + assert 'setuptools' in res @tempdir def test_full(): """virtualenv + pip + buildout""" _system_call('virtualenv', '--no-site-packages', '.') - _system_call('bin/easy_install', '-q', 'distribute==dev') - _system_call('bin/easy_install', '-qU', 'distribute==dev') + _system_call('bin/easy_install', '-q', 'setuptools==dev') + _system_call('bin/easy_install', '-qU', 'setuptools==dev') _system_call('bin/easy_install', '-q', 'pip') _system_call('bin/pip', 'install', '-q', 'zc.buildout') @@ -95,16 +93,16 @@ def test_full(): with open('bootstrap.py', 'w') as f: f.write(urlopen(BOOTSTRAP).read()) - _system_call('bin/python', 'bootstrap.py', '--distribute') + _system_call('bin/python', 'bootstrap.py') _system_call('bin/buildout', '-q') eggs = os.listdir('eggs') eggs.sort() assert len(eggs) == 3 - assert eggs[0].startswith('distribute') - assert eggs[1:] == ['extensions-0.3-py2.6.egg', - 'zc.recipe.egg-1.2.2-py2.6.egg'] + assert eggs[1].startswith('setuptools') + del eggs[1] + assert eggs == ['extensions-0.3-py2.6.egg', + 'zc.recipe.egg-1.2.2-py2.6.egg'] if __name__ == '__main__': test_virtualenv() test_full() - diff --git a/tests/test_distribute_setup.py b/tests/test_ez_setup.py index 1f3da058..922bd884 100644 --- a/tests/test_distribute_setup.py +++ b/tests/test_ez_setup.py @@ -9,10 +9,9 @@ CURDIR = os.path.abspath(os.path.dirname(__file__)) TOPDIR = os.path.split(CURDIR)[0] sys.path.insert(0, TOPDIR) -from distribute_setup import (use_setuptools, _build_egg, _python_cmd, - _do_download, _install, DEFAULT_URL, - DEFAULT_VERSION) -import distribute_setup +from ez_setup import (use_setuptools, _build_egg, _python_cmd, _do_download, + _install, DEFAULT_URL, DEFAULT_VERSION) +import ez_setup class TestSetup(unittest.TestCase): @@ -54,20 +53,11 @@ class TestSetup(unittest.TestCase): def test_install(self): def _faked(*args): return True - distribute_setup.python_cmd = _faked + ez_setup.python_cmd = _faked _install(self.tarball) def test_use_setuptools(self): self.assertEqual(use_setuptools(), None) - # make sure fake_setuptools is not called by default - import pkg_resources - del pkg_resources._distribute - def fake_setuptools(*args): - raise AssertionError - - pkg_resources._fake_setuptools = fake_setuptools - use_setuptools() - if __name__ == '__main__': unittest.main() |