diff options
| -rw-r--r-- | README.unittests.rst | 80 | ||||
| -rw-r--r-- | doc/build/changelog/unreleased_14/4789.rst | 7 | ||||
| -rw-r--r-- | lib/sqlalchemy/dialects/mssql/pyodbc.py | 31 | ||||
| -rw-r--r-- | setup.cfg | 2 | ||||
| -rw-r--r-- | setup.py | 42 | ||||
| -rw-r--r-- | tox.ini | 3 |
6 files changed, 89 insertions, 76 deletions
diff --git a/README.unittests.rst b/README.unittests.rst index c32837f78..e587a89eb 100644 --- a/README.unittests.rst +++ b/README.unittests.rst @@ -2,35 +2,34 @@ SQLALCHEMY UNIT TESTS ===================== -Updated for 1.1, 1.2 - Basic Test Running ================== -A test target exists within the setup.py script. For basic test runs:: +Tox is used to run the test suite fully. For basic test runs against +a single Python interpreter:: - python setup.py test + tox -Running with Tox -================ +Advanced Tox Options +==================== For more elaborate CI-style test running, the tox script provided will run against various Python / database targets. For a basic run against Python 2.7 using an in-memory SQLite database:: - tox -e py27-sqlite + tox -e py38-sqlite The tox runner contains a series of target combinations that can run against various combinations of databases. The test suite can be run against SQLite with "backend" tests also running against a PostgreSQL database:: - tox -e py36-sqlite-postgresql + tox -e py38-sqlite-postgresql -Or to run just "backend" tests against a MySQL database:: +Or to run just "backend" tests against a MySQL databases:: - tox -e py36-mysql-backendonly + tox -e py38-mysql-backendonly Running against backends other than SQLite requires that a database of that vendor be available at a specific URL. See "Setting Up Databases" below @@ -39,12 +38,12 @@ for details. The py.test Engine ================== -Both the tox runner and the setup.py runner are using py.test to invoke -the test suite. Within the realm of py.test, SQLAlchemy itself is adding -a large series of option and customizations to the py.test runner using -plugin points, to allow for SQLAlchemy's multiple database support, -database setup/teardown and connectivity, multi process support, as well as -lots of skip / database selection rules. +The tox runner is using py.test to invoke the test suite. Within the realm of +py.test, SQLAlchemy itself is adding a large series of option and +customizations to the py.test runner using plugin points, to allow for +SQLAlchemy's multiple database support, database setup/teardown and +connectivity, multi process support, as well as lots of skip / database +selection rules. Running tests with py.test directly grants more immediate control over database options and test selection. @@ -66,11 +65,12 @@ Above will run the tests in the test/sql/test_query.py file (a pretty good file for basic "does this database work at all?" to start with) against a running PostgreSQL database at the given URL. -The py.test frontend can also run tests against multiple kinds of databases -at once - a large subset of tests are marked as "backend" tests, which will -be run against each available backend, and additionally lots of tests are targeted -at specific backends only, which only run if a matching backend is made available. -For example, to run the test suite against both PostgreSQL and MySQL at the same time:: +The py.test frontend can also run tests against multiple kinds of databases at +once - a large subset of tests are marked as "backend" tests, which will be run +against each available backend, and additionally lots of tests are targeted at +specific backends only, which only run if a matching backend is made available. +For example, to run the test suite against both PostgreSQL and MySQL at the +same time:: py.test -n4 --db postgresql --db mysql @@ -85,7 +85,7 @@ a pre-set URL. These can be seen using --dbs:: Available --db options (use --dburi to override) default sqlite:///:memory: firebird firebird://sysdba:masterkey@localhost//Users/classic/foo.fdb - mssql mssql+pyodbc://scott:tiger@ms_2008 + mssql mssql+pyodbc://scott:tiger^5HHH@mssql2017:1433/test?driver=ODBC+Driver+13+for+SQL+Server mssql_pymssql mssql+pymssql://scott:tiger@ms_2008 mysql mysql://scott:tiger@127.0.0.1:3306/test?charset=utf8 oracle oracle://scott:tiger@127.0.0.1:1521 @@ -97,6 +97,11 @@ a pre-set URL. These can be seen using --dbs:: sqlite sqlite:///:memory: sqlite_file sqlite:///querytest.db +Note that a pyodbc URL **must be against a hostname / database name +combination, not a DSN name** when using the multiprocessing option; this is +because the test suite needs to generate new URLs to refer to per-process +databases that are created on the fly. + What those mean is that if you have a database running that can be accessed by the above URL, you can run the test suite against it using ``--db <name>``. @@ -124,15 +129,30 @@ of the fixed one in setup.cfg. Database Configuration ====================== +Step one, the **database chosen for tests must be entirely empty**. A lot +of what SQLAlchemy tests is creating and dropping lots of tables +as well as running database introspection to see what is there. If there +are pre-existing tables or other objects in the target database already, +these will get in the way. A failed test run can also be followed by + a run that includes the "--dropfirst" option, which will try to drop +all existing tables in the target database. + +The above paragraph changes somewhat when the multiprocessing option +is used, in that separate databases will be created instead, however +in the case of Postgresql, the starting database is used as a template, +so the starting database must still be empty. + The test runner will by default create and drop tables within the default -database that's in the database URL, *unless* the multiprocessing option -is in use via the py.test "-n" flag, which invokes pytest-xdist. The -multiprocessing option is **enabled by default** for both the tox runner -and the setup.py frontend. When multiprocessing is used, the SQLAlchemy -testing framework will create a new database for each process, and then -tear it down after the test run is complete. So it will be necessary -for the database user to have access to CREATE DATABASE in order for this -to work. +database that's in the database URL, *unless* the multiprocessing option is in +use via the py.test "-n" flag, which invokes pytest-xdist. The +multiprocessing option is **enabled by default** when using the tox runner. +When multiprocessing is used, the SQLAlchemy testing framework will create a +new database for each process, and then tear it down after the test run is +complete. So it will be necessary for the database user to have access to +CREATE DATABASE in order for this to work. Additionally, as mentioned +earlier, the database URL must be formatted such that it can be rewritten on +the fly to refer to these other databases, which means for pyodbc it must refer +to a hostname/database name combination, not a DSN name. Several tests require alternate usernames or schemas to be present, which are used to test dotted-name access scenarios. On some databases such diff --git a/doc/build/changelog/unreleased_14/4789.rst b/doc/build/changelog/unreleased_14/4789.rst new file mode 100644 index 000000000..0d7e1855a --- /dev/null +++ b/doc/build/changelog/unreleased_14/4789.rst @@ -0,0 +1,7 @@ +.. change:: + :tags: change, tests + :tickets: 4789 + + "python setup.py test" is no longer a test runner, as this is deprecated by + Pypa. Please use "tox" with no arguments for a basic test run. + diff --git a/lib/sqlalchemy/dialects/mssql/pyodbc.py b/lib/sqlalchemy/dialects/mssql/pyodbc.py index 2246886a3..f12fb5ead 100644 --- a/lib/sqlalchemy/dialects/mssql/pyodbc.py +++ b/lib/sqlalchemy/dialects/mssql/pyodbc.py @@ -21,8 +21,11 @@ detailed in `ConnectionStrings <https://code.google.com/p/pyodbc/wiki/Connection DSN Connections ^^^^^^^^^^^^^^^ -A DSN-based connection is **preferred** overall when using ODBC. A -basic DSN-based connection looks like:: +A DSN connection in ODBC means that a pre-existing ODBC datasource is +configured on the client machine. The application then specifies the name +of this datasource, which encompasses details such as the specific ODBC driver +in use as well as the network address of the database. Assuming a datasource +is configured on the client, a basic DSN-based connection looks like:: engine = create_engine("mssql+pyodbc://scott:tiger@some_dsn") @@ -36,15 +39,16 @@ the ``Trusted_Connection=yes`` directive to the ODBC string. Hostname Connections ^^^^^^^^^^^^^^^^^^^^ -Hostname-based connections are **not preferred**, however are supported. -The ODBC driver name must be explicitly specified:: +Hostname-based connections are also supported by pyodbc. These are often +easier to use than a DSN and have the additional advantage that the specific +database name to connect towards may be specified locally in the URL, rather +than it being fixed as part of a datasource configuration. - engine = create_engine("mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=SQL+Server+Native+Client+10.0") +When using a hostname connection, the driver name must also be specified in the +query parameters of the URL. As these names usually have spaces in them, the +name must be URL encoded which means using plus signs for spaces:: -.. versionchanged:: 1.0.0 Hostname-based PyODBC connections now require the - SQL Server driver name specified explicitly. SQLAlchemy cannot - choose an optimal default here as it varies based on platform - and installed drivers. + engine = create_engine("mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=SQL+Server+Native+Client+10.0") Other keywords interpreted by the Pyodbc dialect to be passed to ``pyodbc.connect()`` in both the DSN and hostname cases include: @@ -53,10 +57,11 @@ Other keywords interpreted by the Pyodbc dialect to be passed to Pass through exact Pyodbc string ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -A PyODBC connection string can also be sent exactly as specified in -`ConnectionStrings <https://code.google.com/p/pyodbc/wiki/ConnectionStrings>`_ -into the driver using the parameter ``odbc_connect``. The delimeters must be -URL escaped, however, as illustrated below using ``urllib.parse.quote_plus``:: +A PyODBC connection string can also be sent in pyodbc's format directly, as +specified in `ConnectionStrings +<https://code.google.com/p/pyodbc/wiki/ConnectionStrings>`_ into the driver +using the parameter ``odbc_connect``. The delimeters must be URL encoded, as +illustrated below using ``urllib.parse.quote_plus``:: import urllib params = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 10.0};SERVER=dagger;DATABASE=test;UID=user;PWD=password") @@ -59,7 +59,7 @@ postgresql_psycopg2cffi=postgresql+psycopg2cffi://scott:tiger@127.0.0.1:5432/tes mysql=mysql://scott:tiger@127.0.0.1:3306/test?charset=utf8mb4 pymysql=mysql+pymysql://scott:tiger@127.0.0.1:3306/test?charset=utf8mb4 -mssql=mssql+pyodbc://scott:tiger@ms_2008 +mssql=mssql+pyodbc://scott:tiger^5HHH@mssql2017:1433/test?driver=ODBC+Driver+13+for+SQL+Server mssql_pymssql=mssql+pymssql://scott:tiger@ms_2008 oracle=oracle://scott:tiger@127.0.0.1:1521 @@ -81,37 +81,22 @@ class Distribution(_Distribution): return True -class PyTest(TestCommand): - # from http://pytest.org/latest/goodpractices.html\ - # #integrating-with-setuptools-python-setup-py-test-pytest-runner - # TODO: prefer pytest-runner package at some point, however it was - # not working at the time of this comment. - user_options = [("pytest-args=", "a", "Arguments to pass to py.test")] - - default_options = ["-n", "4", "-q", "--nomemory"] - - def initialize_options(self): - TestCommand.initialize_options(self) - self.pytest_args = "" - - def finalize_options(self): - TestCommand.finalize_options(self) - self.test_args = [] - self.test_suite = True +class UseTox(TestCommand): + RED = 31 + RESET_SEQ = "\033[0m" + BOLD_SEQ = "\033[1m" + COLOR_SEQ = "\033[1;%dm" def run_tests(self): - import shlex - - # import here, cause outside the eggs aren't loaded - import pytest - - errno = pytest.main( - self.default_options + shlex.split(self.pytest_args) + sys.stderr.write( + "%s%spython setup.py test is deprecated by pypa. Please invoke " + "'tox' with no arguments for a basic test run.\n%s" + % (self.COLOR_SEQ % self.RED, self.BOLD_SEQ, self.RESET_SEQ) ) - sys.exit(errno) + sys.exit(1) -cmdclass["test"] = PyTest +cmdclass["test"] = UseTox def status_msgs(*msgs): @@ -156,11 +141,6 @@ def run_setup(with_cext): package_dir={"": "lib"}, license="MIT", cmdclass=cmdclass, - tests_require=[ - "pytest>=2.5.2,!=3.9.1,!=3.9.2", - "mock", - "pytest-xdist", - ], long_description=readme, python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*", classifiers=[ @@ -1,6 +1,6 @@ [tox] -envlist = py{27,34,35,36,37,38}-{cext,nocext} +envlist = py [testenv] # note that we have a .coveragerc file that points coverage specifically @@ -75,6 +75,7 @@ passenv=ORACLE_HOME NLS_LANG TOX_POSTGRESQL TOX_MYSQL TOX_ORACLE TOX_MSSQL TOX_S # for nocext, we rm *.so in lib in case we are doing usedevelop=True commands= + cext: /bin/true nocext: sh -c "rm -f lib/sqlalchemy/*.so" {env:BASECOMMAND} {env:WORKERS} {env:SQLITE:} {env:POSTGRESQL:} {env:MYSQL:} {env:ORACLE:} {env:MSSQL:} {env:BACKENDONLY:} {env:IDENTS:} {env:NOMEMORY:} {env:COVERAGE:} {posargs} oracle,oracle6,oracle5,mssql: python reap_dbs.py db_idents.txt |