From 4b4b36bdbd02ad0620ec04ac64a4758260cb26f9 Mon Sep 17 00:00:00 2001 From: holger krekel Date: Sat, 22 Dec 2012 21:56:55 +0100 Subject: rename --- doc/config-v2.txt | 315 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ doc/newconfig.txt | 308 ---------------------------------------------------- 2 files changed, 315 insertions(+), 308 deletions(-) create mode 100644 doc/config-v2.txt delete mode 100644 doc/newconfig.txt diff --git a/doc/config-v2.txt b/doc/config-v2.txt new file mode 100644 index 0000000..4c803d2 --- /dev/null +++ b/doc/config-v2.txt @@ -0,0 +1,315 @@ +V2: new tox multi-dimensional, platform-specific configuration +-------------------------------------------------------------------- + +.. note:: + + This is a draft document sketching a to-be-done implementation. + It does not fully specify each change yet but should give a good + idea of where things are heading. + +**Abstract**: Adding multi-dimensional configuration, platform-specification +and multiple installers to tox.ini. + +**Target audience**: Developers using or wanting to use tox for testing +their python projects. + +Issues with current tox (1.4) configuration +------------------------------------------------ + +Tox is used as a tool for creating and managing virtualenv environments +and running tests in them. As of tox-1.4 there are some issues frequenetly +coming up with its configuration language: + +- there is no way to instruct tox to parametrize testenv specifications + other than to list all combinations by specifying a ``[testenv:...]`` + section for each combination. Examples of real life situations + arising from this: + + * http://code.larlet.fr/django-rest-framework/src/eed0f39a7e45/tox.ini + + * https://bitbucket.org/tabo/django-treebeard/src/93b579395a9c/tox.ini + +- there is no way to have platform specific settings other than to + define specific testenvs and invoke tox with a platform-specific + testenv list. + +- there is no way to specify the platforms against which a project + shall successfully run. + +- tox always uses pip for installing packages currently. This has + several issues: + + - no way to check if installing via easy_install works + - no installs of packages with compiled c-extensions (win32 standard) + +Goals, resolving those issues +------------------------------------ + +This document discusses a possible solution for each of these issues, +namely these goals: + +- allow to more easily define and run dependency/interpreter variants + with testenvs, maintaining per-testenv customizability +- allow platform-specific settings +- allow to specify platforms against which tests should run +- allow to run installer-variants (easy_install or pip) + + +Example: Generating and selecting variants +---------------------------------------------- + +Suppose you want to test your package against mypkg-1.3 and mypkg-1.4 +versions, against python2.6, 2.7 interpreters and on ``linux`` and +``win32`` platforms. Today you would have to +write down 2*2*2 = 8 ``[testenv*]`` sections and then instruct +tox to on the respective platform with a respective environment name list. + +With tox-1.X there will be no need to write down such boilerplate stuff. +Without further ado, here is how a ``tox.ini`` would look like:: + + # combination syntax gives 2 * 2 * 2 = 8 testenv names + # + envlist = [py26,py27]-[mypkg13,mypkg14]-[win,linux] + + [testenv] + deps = pytest + # variant specific dependencies + mypkg13: mypkg<1.4 + mypkg14: mypkg>=1.4,<1.5 + platform= + win: windows + linux: linux + basepython= + py26: python2.6 + py27: python2.7 + + commands = py.test + +Let's go through this step by step:: + + envlist = [py26,py27]-[mypkg13,mypkg14]-[windows,linux] + +This creates a list of ``2*2*2=8`` environment names. It is +a short form for writing the environments down explicitely +like this:: + + envlist = py26-mypkg13-windows, py26-mypkg13-linux, + py26-mypkg14-windows, py26-mypkg14-linux, + py27-mypkg13-windows, py27-mypkg13-linux, + py27-mypkg14-windows, py27-mypkg14-linux, + +Let's look at the next config item, the declaration of the generic +testenv. All the eight testenvironments will derive from this one. +Unlike with earlier tox versions, there is no need to write down +eight different ``[testenv:...]`` sections:: + + [testenv] + deps = pytest + # variant specific dependencies + mypkg13: mypkg<1.4 + mypkg14: mypkg>=1.4,<1.5 + +This defines an unconditional dependency ``pytest`` which is going to be +installed in all environments. It also define two conditional deps: + +- if ``mypkg13`` is part of the environment name, the ``mypkg<1.4`` spec + will be used, otherwise the line is empty. +- if ``mypkg14`` is part of the environment name, the ``mypkg>=1.4,<1.5`` spec + will be used, otherwise the line is empty. + +The next configuration item defines the platform, depending on the +environment name for which the ``[testenv]`` is used:: + + platform= + win: windows + linux: linux + +These two conditional settings will lead to either ``windows`` or +``linux`` as the platform string. When the test environment is run, +its platform string needs to be contained match the string returned +from ``platform.platform()``. Otherwise the environment will be skipped. + +The next configuration item in the ``testenv`` section deals with +the python interpreter:: + + basepython = + py26: python2.6 + py27: python2.7 + +This defines the two executables, depending on if ``py26`` or ``py27`` +appears in the environment name. + +The last config item is simply the invocation of the test runner:: + + commands = py.test + +Nothing special here :) + +However, as tox provides good defaults for platform and basepython +settings, we can cut them out from our tox.ini:: + + envlist = [py26,py27]-[mypkg13,mypkg14]-[win,linux] + + [testenv] + deps = pytest + # variant specific dependencies + mypkg13: mypkg<1.4 + mypkg14: mypkg>=1.4,<1.5 + +Voila, this ``tox.ini`` file defines 8 environments. + + +The new "platform" setting +-------------------------------------- + +A testenv can define a new ``platform`` setting. If its value +is not contained in the string obtained from calling ``platform.platform()`` +the environment will be skipped. + +Generator expressions in the envlist setting +---------------------------------------------------------- + +The new ``envlist`` setting allows to use ``[CSV]`` expressions +where ``CSV`` is a list of comma-separated values. The basic +generating algorithm works like this: + +- expand: for each CSV-expression in an environment name in the list + produce an additional environment name for each value in the CSV +- repeat: as long as there are CSV-expressions, continue the process + + +Variant specification with [variant:VARNAME] +---------------------------------------------- + +Apart from using conditional settings, you can also write down +a ``[variant::VARIANTNAME]`` section, allowing to define settings +for the respective variant. Variant settings will be merged from +left to right so an environment name ``abc-def`` will lookup +and merge settings from ``abc``, then from ``def``. + +.. + Note that a direct ``[variant:xyz-abc]`` testenv definition + can override any automatically produced settings. + +Showing all expanded sections +------------------------------- + +To help with understanding how the variants will produce section values, +you can ask tox to show their expansion with a new option:: + + $ tox -l [XXX output ommitted for now] + +Making sure your packages installs with easy_install +------------------------------------------------------ + +The new "installer" testenv setting allows to specify the tool for +installation in a given test environment:: + + [testenv] + installer = + easy: easy_install + pip: pip + +If you want to have your package installed with both easy_install +and pip, you can list them in your envlist likes this:: + + [tox] + envlist = py[26,27,32]-django[13,14]-[easy,pip] + +If no installer is specified, ``pip`` will be used. + +Default settings related to environments names/variants +--------------------------------------------------------------- + +tox comes with predefined settings for certain variants, namely: + +* ``[easy,pip]`` use easy_install or pip respectively +* ``[py24,py25,py26,py27,py31,py32,py33,pypy19]`` use the respective + pythonNN or PyPy interpreter +* ``[win32,linux,darwin]`` defines the according ``platform``. + +You can use those in your “envlist” specification +without the need to define them yourself. + +Transforming the examples: django-rest +------------------------------------------------ + +The original `django-rest-framework tox.ini +`_ +file has 159 lines and a lot of repetition, the new one would +have 20 +lines and almost no repetition:: + + [tox] + envlist = [py25,py26,py27]-[django12,django13]-[,example] + + [testenv] + commands = python setup.py test + deps= + coverage==3.4 + unittest-xml-reporting==1.2 + Pyyaml==3.10 + django12: django==1.2.4 + django13: django==1.3.1 + + [variant:example] + commands = python examples/runtests.py + +deps = + wsgiref==0.1.2 + Pygments==1.4 + httplib2==0.6.0 + Markdown==2.0.3 + +Note that ``[,example]`` in the envlist denotes an empty env and the +"example" variant. The empty variant means that there are no specific +settings and thus no need to define a variant name. + +Note also that ``+deps`` means that we are appending to dependencies, +not substituting them. + +Transforming the examples: django-treebeard +------------------------------------------------ + +Another `tox.ini +`_ +has 233 lines and runs tests against multiple Postgres and Mysql +engines. It also performs backend-specific test commands, passing +different command line options to the test script. With the new tox-1.X +we not only can do the same with 32 non-repetive configuration lines but +we also produce 36 specific testenvs with specific dependencies and test +commands:: + + [tox] + envlist = + [py24,py25,py26,py27]-[django11,django12,django13]-[nodb,pg,mysql] + docs + + [testenv:docs] + changedir = docs + deps = + Sphinx + Django + commands = + make clean + make html + + [testenv] + deps= + coverage + pysqlite + django11: django==1.1.4 + django12: django==1.2.7 + django13: django==1.3.1 + django14: django==1.4 + nodb: pysqlite + pg: psycopg2 + mysql: MySQL-python + + commands = + nodb: {envpython} runtests.py {posargs} + pg: {envpython} runtests.py {posargs} \ + --DATABASE_ENGINE=postgresql_psycopg2 \ + --DATABASE_USER=postgres {posargs} + mysql: {envpython} runtests.py --DATABASE_ENGINE=mysql \ + --DATABASE_USER=root {posargs} + +It's noteworthy here that you can also use conditionals in the commands. diff --git a/doc/newconfig.txt b/doc/newconfig.txt deleted file mode 100644 index 84f1355..0000000 --- a/doc/newconfig.txt +++ /dev/null @@ -1,308 +0,0 @@ -V2: new tox multi-dimensional, platform-specific configuration --------------------------------------------------------------------- - -**Abstract**: Adding multi-dimensional configuration, platform-specification -and multiple installers to tox.ini. - -**Target audience**: Developers using or wanting to use tox for testing -their python projects. - -Issues with current tox (1.4) configuration ------------------------------------------------- - -Tox is used as a tool for creating and managing virtualenv environments -and running tests in them. As of tox-1.4 there are some issues frequenetly -coming up with its configuration language: - -- there is no way to instruct tox to parametrize testenv specifications - other than to list all combinations by specifying a ``[testenv:...]`` - section for each combination. Examples of real life situations - arising from this: - - * http://code.larlet.fr/django-rest-framework/src/eed0f39a7e45/tox.ini - - * https://bitbucket.org/tabo/django-treebeard/src/93b579395a9c/tox.ini - -- there is no way to have platform specific settings other than to - define specific testenvs and invoke tox with a platform-specific - testenv list. - -- there is no way to specify the platforms against which a project - shall successfully run. - -- tox always uses pip for installing packages currently. This has - several issues: - - - no way to check if installing via easy_install works - - no installs of packages with compiled c-extensions (win32 standard) - -This document discusses a possible solution for each of these issues. -It does so by going through examples and by transforming the above -two - -- allow to more easily define and run dependency/interpreter variants - with testenvs, maintaining per-testenv customizability -- allow to define settings in a platform-specific way -- allow to define the platforms against which tests should run -- allow to run variants of installing via easy_install or pip. - - -Example: Generating and selecting variants ----------------------------------------------- - -Suppose you want to test your package against mypkg-1.3 and mypkg-1.4 -versions, against python2.6, 2.7 interpreters and on ``linux`` and -``win32`` platforms. Today you would have to -write down 2*2*2 = 8 ``[testenv*]`` sections and then instruct -tox to on the respective platform with a respective environment name list. - -With tox-1.5 there is no need to write down such boilerplate stuff. -Without further ado, here is how a suitable ``tox.ini`` would look like:: - -Without much further introduction, here is an example ``tox.ini``:: - - # combination syntax gives 2 * 2 * 2 = 8 testenv names - # - envlist = [py26,py27]-[mypkg13,mypkg14]-[win,linux] - - [testenv] - deps = pytest - # variant specific dependencies - mypkg13: mypkg<1.4 - mypkg14: mypkg>=1.4,<1.5 - platform= - win: windows - linux: linux - basepython= - py26: python2.6 - py27: python2.7 - - commands = py.test - -Let's go through this step by step:: - - envlist = [py26,py27]-[mypkg13,mypkg14]-[windows,linux] - -This creates a list of ``2*2*2=8`` environment names. It is -a short form for writing the environments down explicitely -like this:: - - envlist = py26-mypkg13-windows, py26-mypkg13-linux, - py26-mypkg14-windows, py26-mypkg14-linux, - py27-mypkg13-windows, py27-mypkg13-linux, - py27-mypkg14-windows, py27-mypkg14-linux, - -Let's look at the next config item, the declaration of the generic -testenv. All the eight testenvironments will derive from this one. -Unlike with earlier tox versions, there is no need to write down -eight different ``[testenv:...]`` sections:: - - [testenv] - deps = pytest - # variant specific dependencies - mypkg13: mypkg<1.4 - mypkg14: mypkg>=1.4,<1.5 - -This defines an unconditional dependency ``pytest`` which is going to be -installed in all environments. It also define two conditional deps: - -- if ``mypkg13`` is part of the environment name, the ``mypkg<1.4`` spec - will be used, otherwise the line is empty. -- if ``mypkg14`` is part of the environment name, the ``mypkg>=1.4,<1.5`` spec - will be used, otherwise the line is empty. - -The next configuration item defines the platform, depending on the -environment name for which the ``[testenv]`` is used:: - - platform= - win: windows - linux: linux - -These two conditional settings will lead to either ``windows`` or -``linux`` as the platform string. When the test environment is run, -its platform string needs to be contained match the string returned -from ``platform.platform()``. Otherwise the environment will be skipped. - -The next configuration item in the ``testenv`` section deals with -the python interpreter:: - - basepython = - py26: python2.6 - py27: python2.7 - -This defines the two executables, depending on if ``py26`` or ``py27`` -appears in the environment name. - -The last config item is simply the invocation of the test runner:: - - commands = py.test - -Nothing special here :) - -However, as tox provides good defaults for platform and basepython -settings, we can cut them out from our tox.ini:: - - envlist = [py26,py27]-[mypkg13,mypkg14]-[win,linux] - - [testenv] - deps = pytest - # variant specific dependencies - mypkg13: mypkg<1.4 - mypkg14: mypkg>=1.4,<1.5 - -Voila, this ``tox.ini`` file defines 8 environments. - - -The new "platform" setting --------------------------------------- - -A testenv can define a new ``platform`` setting. If its value -is not contained in the string obtained from calling ``platform.platform()`` -the environment will be skipped. - -Generator expressions in the envlist setting ----------------------------------------------------------- - -The new ``envlist`` setting allows to use ``[CSV]`` expressions -where ``CSV`` is a list of comma-separated values. The basic -generating algorithm works like this: - -- expand: for each CSV-expression in an environment name in the list - produce an additional environment name for each value in the CSV -- repeat: as long as there are CSV-expressions, continue the process - - -Variant specification with [variant:VARNAME] ----------------------------------------------- - -Apart from using conditional settings, you can also write down -a ``[variant::VARIANTNAME]`` section, allowing to define settings -for the respective variant. Variant settings will be merged from -left to right so an environment name ``abc-def`` will lookup -and merge settings from ``abc``, then from ``def``. - -.. - Note that a direct ``[variant:xyz-abc]`` testenv definition - can override any automatically produced settings. - -Showing all expanded sections -------------------------------- - -To help with understanding how the variants will produce section values, -you can ask tox to show their expansion with a new option:: - - $ tox -l [XXX output ommitted for now] - -Making sure your packages installs with easy_install ------------------------------------------------------- - -The new "installer" testenv setting allows to specify the tool for -installation:: - - [testenv] - installer = - easy_install ; "easy" in envname - pip ; "pip" in envname or "easy" not in envname - -If you want to have your package installed with both easy_install -and pip, you can list them in your envlist likes this:: - - [tox] - envlist = py[26,27,32]-django[13,14]-[easy,pip] - -If no installer is specified, ``pip`` will be used. - -Default settings for specific names in environments ---------------------------------------------------------------- - -tox comes with predefined settings for certain variants, namely: - -* ``[easy,pip]`` use easy_install or pip respectively -* ``[py24,py25,py26,py27,py31,py32,py33,pypy19]`` use the respective - pythonNN or PyPy interpreter -* ``[win32,linux,darwin]`` defines the according ``platform``. - -You can use those in your “envlist” specification -without the need to define them yourself. - -Transforming the examples: django-rest ------------------------------------------------- - -The original `tox.ini `_ -file has 159 lines and a lot of repetition, the new one would +have 20 -lines and almost no repetition:: - - [tox] - envlist = [py25,py26,py27]-[django12,django13]-[,example] - - [testenv] - commands = python setup.py test - deps= - coverage==3.4 - unittest-xml-reporting==1.2 - Pyyaml==3.10 - django12: django==1.2.4 - django13: django==1.3.1 - - [variant:example] - commands = python examples/runtests.py - +deps = - wsgiref==0.1.2 - Pygments==1.4 - httplib2==0.6.0 - Markdown==2.0.3 - -Note that ``[,example]`` in the envlist denotes an empty env and the - "example" variant. The empty variant means that there are no specific -settings and thus no need to define a variant name. - -Note also that ``+deps`` means that we are appending to dependencies, -not substituting them. - -Transforming the examples: django-treebeard ------------------------------------------------- - -Another `tox.ini -`_ -has 233 lines and runs tests against multiple Postgres and Mysql -engines. It also performs backend-specific test commands, passing -different command line options to the test script. With the new tox-1.X -we not only can do the same with 32 non-repetive configuration lines but -we also produce 36 specific testenvs with specific dependencies and test -commands:: - - [tox] - envlist = - [py24,py25,py26,py27]-[django11,django12,django13]-[nodb,pg,mysql] - docs - - [testenv:docs] - changedir = docs - deps = - Sphinx - Django - commands = - make clean - make html - - [testenv] - deps= - coverage - pysqlite - django11: django==1.1.4 - django12: django==1.2.7 - django13: django==1.3.1 - django14: django==1.4 - nodb: pysqlite - pg: psycopg2 - mysql: MySQL-python - - commands = - nodb: {envpython} runtests.py {posargs} - pg: {envpython} runtests.py {posargs} \ - --DATABASE_ENGINE=postgresql_psycopg2 \ - --DATABASE_USER=postgres {posargs} - mysql: {envpython} runtests.py --DATABASE_ENGINE=mysql \ - --DATABASE_USER=root {posargs} - -It's noteworthy here that you can also use conditionals in the commands. -- cgit v1.2.1