Cap protobuf < 4.x

tooz doesn't use upper-constraints so we get the newest version of all
dependencies. It seems one of these, etcd3, is incompatible with recent
versions of protobuf.

  Failed to import test module: tooz.tests.drivers.test_etcd3
  Traceback (most recent call last):
    File "/usr/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
      module = self._get_module_from_name(name)
    File "/usr/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
      __import__(name)
    File "/home/zuul/src/opendev.org/openstack/tooz/tooz/tests/drivers/test_etcd3.py", line 22, in <module>
      import tooz.drivers.etcd3 as etcd3_driver
    File "/home/zuul/src/opendev.org/openstack/tooz/tooz/drivers/etcd3.py", line 18, in <module>
      import etcd3
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/__init__.py", line 3, in <module>
      import etcd3.etcdrpc as etcdrpc
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/__init__.py", line 1, in <module>
      from .rpc_pb2 import *
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/rpc_pb2.py", line 16, in <module>
      from etcd3.etcdrpc import kv_pb2 as kv__pb2
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/kv_pb2.py", line 36, in <module>
      type=None),
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 755, in __new__
      _message.Message._CheckCalledFromGeneratedFile()
  TypeError: Descriptors cannot not be created directly.
  If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
  If you cannot immediately regenerate your protos, some other possible workarounds are:
   1. Downgrade the protobuf package to 3.20.x or lower.
   2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

The etcd3 dependency is only used for the etcd3 driver which was
deprecated in change Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac. The
protobuf dependency is only used for etcd3. We can't outright remove the
etcd3 driver yet as it hasn't been long enough since we deprecated it.
That will happen soon enough though and until then we can simply cap
protobuf to an older version that is compatible with the most recent
etcd3 release.

While we're here, we also sync the lower boundaries in the docs
requirements and add some comments indicating where they come from/what
they're for.

Conflicts:
  requirements.txt

NOTE(stephenfin): Conflicts are due to the absence of change
I14a01f0a9989195e30c3ed0dadec35490f41bd48 ("[ussuri][goal] Drop python
2.7 support and testing"). As the name would suggest, this is an
Ussuri-only thing that we can't backport.

Changes:
  setup.cfg
  test-requirements.txt
  tooz/drivers/mysql.py

NOTE(stephenfin): We also need to set some caps for libraries that
failed to indicate that they had dropped Python 2.7 support using the
proper 'python_requires' setuptools keyword (or equivalent). This
includes 'grpcio' and 'sysv-ipc' (dependencies on the 'etcd3' and 'ipc'
extras in 'setup.cfg', respectively) and 'pifpaf' in
'test-requirements.txt'. We also need to blacklist etcd3gw 0.2.6 to keep
the requirements job happy. This was done elsewhere via change
I22b955419014dd34c63e406c488e0636ffe9013b. Finally, we need to fix a
breakage with PyMySQL 0.10.0. In that version a Connection is no longer
a context manager. We fix this by simply getting a Cursor out of it
(locks don't seem to interact with transactions, at least according to
MariaDB docs). This was done elsewhere via change
I5ea06ebd2b976465ff82f10a74e140f30e9e803f.

Change-Id: I8d6647118be22b0ce55e01b7e5451612ebe30e73
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
(cherry picked from commit e3de0e634af62e46162eb8d2354416ce4f2ba77c)
(cherry picked from commit 73dbe0e6fc1403cbb121985781d7b6ec996b099a)
(cherry picked from commit 8ca40da4bbe813a76cd48d3e8e54b78d10f25765)
(cherry picked from commit 59da4d8d6aae828c2fd36d1ddf12652c3cee7f7a)

5 files changed, 40 insertions, 19 deletions

diff --git a/doc/requirements.txt b/doc/requirements.txt
index 42d2dde..46a198b 100644
--- a/doc/requirements.txt
+++ b/doc/requirements.txt
@@ -4,12 +4,27 @@ openstackdocstheme>=1.11.0 # Apache-2.0
 reno>=1.8.0 # Apache-2.0
 
 # Install dependencies for tooz so that autodoc works.
-python-consul>=0.4.7 # MIT License
-sysv-ipc>=0.6.8 # BSD License
+## consul
+python-consul2>=0.0.16 # MIT License
+## etcd
+requests>=2.10.0 # Apache-2.0
+## etcd3
+etcd3>=0.12.0 # Apache-2.0
+grpcio>=1.18.0
+protobuf<4.0 # BSD License (3 clause)
+## etcd3gw
+etcd3gw!=0.2.6,>=0.1.0 # Apache-2.0
+## zake
 zake>=0.1.6 # Apache-2.0
-redis>=2.10.0 # MIT
+## redis
+redis>=3.1.0 # MIT
+## postgresql
 psycopg2>=2.5 # LGPL/ZPL
+## mysql
 PyMySQL>=0.6.2 # MIT License
+## zookeeper
+kazoo>=2.2 # Apache-2.0
+## memcached
 pymemcache!=1.3.0,>=1.2.9 # Apache 2.0 License
-etcd3>=0.6.2 # Apache-2.0
-etcd3gw>=0.1.0 # Apache-2.0
+## ipc
+sysv-ipc>=0.6.8 # BSD License
diff --git a/requirements.txt b/requirements.txt
index 1af5228..570349d 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -8,7 +8,8 @@ enum34>=1.0.4;python_version=='2.7' or python_version=='2.6' or python_version==
 voluptuous>=0.8.9 # BSD License
 msgpack>=0.4.0 # Apache-2.0
 fasteners>=0.7 # Apache-2.0
-tenacity>=3.2.1  # Apache-2.0
+tenacity>=3.2.1,<6.0.0;python_version=='2.7' # Apache-2.0
+tenacity>=3.2.1,<7.0.0;python_version>='3.6' # Apache-2.0
 futures>=3.0;python_version=='2.7' or python_version=='2.6' # BSD
 futurist>=1.2.0 # Apache-2.0
 oslo.utils>=3.15.0 # Apache-2.0
diff --git a/setup.cfg b/setup.cfg
index 5a2d968..769edb4 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -47,9 +47,11 @@ etcd =
   requests>=2.10.0 # Apache-2.0
 etcd3 =
   etcd3>=0.6.2 # Apache-2.0
-  grpcio>=1.18.0
+  grpcio>=1.18.0,<1.40.0;python_version=='2.7'
+  grpcio>=1.18.0;python_version>='3.4'
+  protobuf<4.0 # BSD License (3 clause)
 etcd3gw =
-  etcd3gw>=0.1.0 # Apache-2.0
+  etcd3gw!=0.2.6,>=0.1.0 # Apache-2.0
 zake =
   zake>=0.1.6 # Apache-2.0
 redis =
@@ -63,6 +65,7 @@ zookeeper =
 memcached =
   pymemcache!=1.3.0,>=1.2.9 # Apache 2.0 License
 ipc =
-  sysv-ipc>=0.6.8 # BSD License
+  sysv-ipc>=0.6.8,<1.0.0;python_version=='2.7' # BSD License
+  sysv-ipc>=0.6.8;python_version>='3.4' # BSD License
 [wheel]
 universal = 1
diff --git a/test-requirements.txt b/test-requirements.txt
index cdec5fd..dcaf721 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -7,7 +7,9 @@ python-subunit>=0.0.18 # Apache-2.0/BSD
 testtools>=1.4.0 # MIT
 coverage>=3.6 # Apache-2.0
 fixtures>=3.0.0 # Apache-2.0/BSD
-pifpaf>=0.10.0 # Apache-2.0
+pifpaf>=0.10.0,<3.0.0;python_version=='2.7' # Apache-2.0
+pifpaf>=0.10.0;python_version>='3.4' # Apache-2.0
+daiquiri<3.0.0;python_version=='2.7' # Apache-2.0
 os-testr>=0.8.0 # Apache-2.0
 stestr>=2.0.0,<3.0.0;python_version<'3.0'
 stestr>=2.0.0;python_version>='3.0'
diff --git a/tooz/drivers/mysql.py b/tooz/drivers/mysql.py
index 269ec57..5bc8892 100644
--- a/tooz/drivers/mysql.py
+++ b/tooz/drivers/mysql.py
@@ -61,12 +61,12 @@ class MySQLLock(locking.Lock):
             try:
                 if not self._conn.open:
                     self._conn.connect()
-                with self._conn as cur:
-                    cur.execute("SELECT GET_LOCK(%s, 0);", self.name)
-                    # Can return NULL on error
-                    if cur.fetchone()[0] is 1:
-                        self.acquired = True
-                        return True
+                cur = self._conn.cursor()
+                cur.execute("SELECT GET_LOCK(%s, 0);", self.name)
+                # Can return NULL on error
+                if cur.fetchone()[0] == 1:
+                    self.acquired = True
+                    return True
             except pymysql.MySQLError as e:
                 utils.raise_with_cause(
                     tooz.ToozError,
@@ -90,9 +90,9 @@ class MySQLLock(locking.Lock):
         if not self.acquired:
             return False
         try:
-            with self._conn as cur:
-                cur.execute("SELECT RELEASE_LOCK(%s);", self.name)
-                cur.fetchone()
+            cur = self._conn.cursor()
+            cur.execute("SELECT RELEASE_LOCK(%s);", self.name)
+            cur.fetchone()
             self.acquired = False
             self._conn.close()
             return True