diff options
| author | Andy McCurdy <andy@andymccurdy.com> | 2020-07-20 14:21:39 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-07-20 14:21:39 -0700 |
| commit | 75a2cfe6eeca842801372c997a42707b6dca8c05 (patch) | |
| tree | 9ec7cb476de049c724550e7f8e025dd103815308 | |
| parent | 142a44a075df4f371320d6da2b8341db62fa5d15 (diff) | |
| parent | 1f988d85662ec748a0111d4a6aa4e31effd020cd (diff) | |
| download | redis-py-75a2cfe6eeca842801372c997a42707b6dca8c05.tar.gz | |
Dockerize (#1365)
Provide a docker development and testing environment
* CI (Travis) now runs tests via the same docker environment that is available to developers.
* A simple Makefile has been added to make getting started easier.
* `make dev` will standup the development environment.
* `make test` will standup the development environment and also run the test suite.
* `make clean` will remove the development environment.
45 files changed, 636 insertions, 408 deletions
diff --git a/.coveragerc b/.coveragerc new file mode 100644 index 0000000..9be06b9 --- /dev/null +++ b/.coveragerc @@ -0,0 +1,2 @@ +[run] +source = redis diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..08ea4a6 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,3 @@ +**/__pycache__ +**/*.pyc +.tox @@ -11,3 +11,6 @@ vagrant/.vagrant .eggs .idea .coverage +env +venv +coverage.xml diff --git a/.travis.yml b/.travis.yml index 05dd94c..4b09cfc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,42 +1,14 @@ -language: python -cache: pip -matrix: - include: - - env: TOXENV=flake8 - - python: 2.7 - env: TOXENV=py27-plain - - python: 2.7 - env: TOXENV=py27-hiredis - - python: 3.5 - env: TOXENV=py35-plain - - python: 3.5 - env: TOXENV=py35-hiredis - - python: 3.6 - env: TOXENV=py36-plain - - python: 3.6 - env: TOXENV=py36-hiredis - - python: 3.7 - env: TOXENV=py37-plain - - python: 3.7 - env: TOXENV=py37-hiredis - - python: 3.8 - env: TOXENV=py38-plain - - python: 3.8 - env: TOXENV=py38-hiredis - - python: pypy - env: TOXENV=pypy-plain - - python: pypy - env: TOXENV=pypy-hiredis - - python: pypy3 - env: TOXENV=pypy3-plain - - python: pypy3 - env: TOXENV=pypy3-hiredis +env: + - DOCKER_COMPOSE_VERSION=1.26.2 + before_install: - - wget https://github.com/antirez/redis/archive/6.0.5.tar.gz && mkdir redis_install && tar -xvzf 6.0.5.tar.gz -C redis_install && cd redis_install/redis-6.0.5 && make && src/redis-server --daemonize yes && cd ../.. - - redis-cli info -install: - - pip install codecov tox + - sudo rm /usr/local/bin/docker-compose + - curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose + - chmod +x docker-compose + - sudo mv docker-compose /usr/local/bin + +services: + - docker + script: - - tox -after_success: - - "if [[ $TOXENV != 'flake8' ]]; then codecov; fi" + - make test @@ -1,3 +1,6 @@ +* (in development) + * Provide a development and testing environment via docker. Thanks + @abrookins. #1365 * 3.5.3 (June 1, 2020) * Restore try/except clauses to __del__ methods. These will be removed in 4.0 when more explicit resource management if enforced. #1339 diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst new file mode 100644 index 0000000..edaa7b6 --- /dev/null +++ b/CONTRIBUTING.rst @@ -0,0 +1,128 @@ +Contributing +============ + +Introduction +------------ + +First off, thank you for considering contributing to redis-py. We value community contributions! + +Contributions We Need +---------------------- + +You may already know what you want to contribute -- a fix for a bug you encountered, or a new feature your team wants to use. + +If you don't know what to contribute, keep an open mind! Improving documentation, bug triaging, and writing tutorials are all examples of helpful contributions that mean less work for you. + +Your First Contribution +----------------------- +Unsure where to begin contributing? You can start by looking through `help-wanted issues <https://github.com/andymccurdy/redis-py/issues?q=is%3Aopen+is%3Aissue+label%3ahelp-wanted>`_. + +Never contributed to open source before? Here are a couple of friendly tutorials: + +- http://makeapullrequest.com/ +- http://www.firsttimersonly.com/ + +Getting Started +--------------- + +Here's how to get started with your code contribution: + +1. Create your own fork of redis-py +2. Do the changes in your fork +3. If you need a development environment, run ``make dev`` +4. While developing, make sure the tests pass by running ``make test`` +5. If you like the change and think the project could use it, send a pull request + +The Development Environment +--------------------------- + +Running ``make dev`` will create a Docker-based development environment that starts the following containers: + +* A master Redis node +* A slave Redis node +* Three sentinel Redis nodes +* A test container + +The slave is a replica of the master node, using the `leader-follower replication <https://redis.io/topics/replication>`_ feature. + +The sentinels monitor the master node in a `sentinel high-availability configuration <https://redis.io/topics/sentinel>`_. + +Meanwhile, the `test` container hosts the code from your checkout of ``redis-py`` and allows running tests against many Python versions. + +Docker Tips +^^^^^^^^^^^ + +Following are a few tips that can help you work with the Docker-based development environment. + +To get a bash shell inside of a container: + +``$ docker-compose run <service> /bin/bash`` + +**Note**: The term "service" refers to the "services" defined in the ``docker-compose.yml`` file: "master", "slave", "sentinel_1", "sentinel_2", "sentinel_3", "test". + +Containers run a minimal Debian image that probably lacks tools you want to use. To install packages, first get a bash session (see previous tip) and then run: + +``$ apt update && apt install <package>`` + +You can see the combined logging output of all containers like this: + +``$ docker-compose logs`` + +The command `make test` runs all tests in all tested Python environments. To run the tests in a single environment, like Python 3.6, use a command like this: + +``$ docker-compose run test tox -e py36 -- --redis-url=redis://master:6379/9`` + +Here, the flag ``-e py36`` runs tests against the Python 3.6 tox environment. And note from the example that whenever you run tests like this, instead of using `make test`, you need to pass ``-- --redis-url=redis://master:6379/9``. This points the tests at the "master" container. + +Our test suite uses ``pytest``. You can run a specific test suite against a specific Python version like this: + +``$ docker-compose run test tox -e py36 -- --redis-url=redis://master:6379/9 tests/test_commands.py`` + +Troubleshooting +^^^^^^^^^^^^^^^ +If you get any errors when running ``make dev`` or ``make test``, make sure that you +are using supported versions of Docker and docker-compose. + +The included Dockerfiles and docker-compose.yml file work with the following +versions of Docker and docker-compose: + +* Docker 19.03.12 +* docker-compose 1.26.2 + +How to Report a Bug +------------------- + +Security Vulnerabilities +^^^^^^^^^^^^^^^^^^^^^^^^ + +**NOTE**: If you find a security vulnerability, do NOT open an issue. Email Andy McCurdy (sedrik@gmail.com) instead. + +In order to determine whether you are dealing with a security issue, ask yourself these two questions: + +* Can I access something that's not mine, or something I shouldn't have access to? +* Can I disable something for other people? + +If the answer to either of those two questions are "yes", then you're probably dealing with a security issue. Note that even if you answer "no" to both questions, you may still be dealing with a security issue, so if you're unsure, just email Andy at sedrik@gmail.com. + +Everything Else +^^^^^^^^^^^^^^^ + +When filing an issue, make sure to answer these five questions: + +1. What version of redis-py are you using? +2. What version of redis are you using? +3. What did you do? +4. What did you expect to see? +5. What did you see instead? + +How to Suggest a Feature or Enhancement +--------------------------------------- + +If you'd like to contribute a new feature, make sure you check our issue list to see if someone has already proposed it. Work may already be under way on the feature you want -- or we may have rejected a feature like it already. + +If you don't see anything, open a new issue that describes the feature you would like and how it should work. + +Code Review Process +------------------- + +The core team looks at Pull Requests on a regular basis. We will give feedback as as soon as possible. After feedback, we expect a response within two weeks. After that time, we may close your PR if it isn't showing any activity. diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..eba66f0 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,7 @@ +FROM fkrull/multi-python:latest + +RUN apt update && apt install -y pypy pypy-dev pypy3-dev + +WORKDIR /redis-py + +COPY . /redis-py diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..0d09643 --- /dev/null +++ b/Makefile @@ -0,0 +1,14 @@ +.PHONY: base clean dev test + +base: + docker build -t redis-py-base docker/base + +dev: base + docker-compose up -d --build + +test: dev + docker-compose run --rm test /redis-py/docker-entry.sh + +clean: + docker-compose stop + docker-compose rm -f @@ -47,6 +47,12 @@ or from source: $ python setup.py install +Contributing +------------ + +Want to contribute a feature, bug report, or report an issue? Check out our `guide to +contributing <https://github.com/andymccurdy/redis-py/blob/master/CONTRIBUTING.rst>`_. + Getting Started --------------- diff --git a/build_tools/.bash_profile b/build_tools/.bash_profile deleted file mode 100644 index b023cf7..0000000 --- a/build_tools/.bash_profile +++ /dev/null @@ -1 +0,0 @@ -PATH=$PATH:/var/lib/redis/bin diff --git a/build_tools/bootstrap.sh b/build_tools/bootstrap.sh deleted file mode 100755 index a5a0d2c..0000000 --- a/build_tools/bootstrap.sh +++ /dev/null @@ -1,4 +0,0 @@ -#!/usr/bin/env bash - -# need make to build redis -sudo apt-get install make diff --git a/build_tools/build_redis.sh b/build_tools/build_redis.sh deleted file mode 100755 index 379c6cc..0000000 --- a/build_tools/build_redis.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env bash - -source /home/vagrant/redis-py/build_tools/redis_vars.sh - -pushd /home/vagrant - -uninstall_all_sentinel_instances -uninstall_all_redis_instances - -# create a clean directory for redis -rm -rf $REDIS_DIR -mkdir -p $REDIS_BIN_DIR -mkdir -p $REDIS_CONF_DIR -mkdir -p $REDIS_SAVE_DIR - -# download, unpack and build redis -mkdir -p $REDIS_DOWNLOAD_DIR -cd $REDIS_DOWNLOAD_DIR -rm -f $REDIS_PACKAGE -rm -rf $REDIS_BUILD_DIR -wget http://download.redis.io/releases/$REDIS_PACKAGE -tar zxvf $REDIS_PACKAGE -cd $REDIS_BUILD_DIR -make -cp src/redis-server $REDIS_DIR/bin -cp src/redis-cli $REDIS_DIR/bin -cp src/redis-sentinel $REDIS_DIR/bin - -popd diff --git a/build_tools/install_redis.sh b/build_tools/install_redis.sh deleted file mode 100755 index fd53a1c..0000000 --- a/build_tools/install_redis.sh +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env bash - -source /home/vagrant/redis-py/build_tools/redis_vars.sh - -for filename in `ls $VAGRANT_REDIS_CONF_DIR`; do - # cuts the order prefix off of the filename, e.g. 001-master -> master - PROCESS_NAME=redis-`echo $filename | cut -f 2- -d -` - echo "======================================" - echo "INSTALLING REDIS SERVER: $PROCESS_NAME" - echo "======================================" - - # make sure the instance is uninstalled (it should be already) - uninstall_instance $PROCESS_NAME - - # base config - mkdir -p $REDIS_CONF_DIR - cp $REDIS_BUILD_DIR/redis.conf $REDIS_CONF_DIR/$PROCESS_NAME.conf - # override config values from file - cat $VAGRANT_REDIS_CONF_DIR/$filename >> $REDIS_CONF_DIR/$PROCESS_NAME.conf - - # replace placeholder variables in init.d script - cp $VAGRANT_DIR/redis_init_script /etc/init.d/$PROCESS_NAME - sed -i "s/{{ PROCESS_NAME }}/$PROCESS_NAME/g" /etc/init.d/$PROCESS_NAME - # need to read the config file to find out what port this instance will run on - port=`grep port $VAGRANT_REDIS_CONF_DIR/$filename | cut -f 2 -d " "` - sed -i "s/{{ PORT }}/$port/g" /etc/init.d/$PROCESS_NAME - chmod 755 /etc/init.d/$PROCESS_NAME - - # and tell update-rc.d about it - update-rc.d $PROCESS_NAME defaults 98 - - # save the $PROCESS_NAME into installed instances file - echo $PROCESS_NAME >> $REDIS_INSTALLED_INSTANCES_FILE - - # start redis - /etc/init.d/$PROCESS_NAME start -done diff --git a/build_tools/install_sentinel.sh b/build_tools/install_sentinel.sh deleted file mode 100755 index 0597208..0000000 --- a/build_tools/install_sentinel.sh +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env bash - -source /home/vagrant/redis-py/build_tools/redis_vars.sh - -for filename in `ls $VAGRANT_SENTINEL_CONF_DIR`; do - # cuts the order prefix off of the filename, e.g. 001-master -> master - PROCESS_NAME=sentinel-`echo $filename | cut -f 2- -d -` - echo "=========================================" - echo "INSTALLING SENTINEL SERVER: $PROCESS_NAME" - echo "=========================================" - - # make sure the instance is uninstalled (it should be already) - uninstall_instance $PROCESS_NAME - - # base config - mkdir -p $REDIS_CONF_DIR - cp $REDIS_BUILD_DIR/sentinel.conf $REDIS_CONF_DIR/$PROCESS_NAME.conf - # override config values from file - cat $VAGRANT_SENTINEL_CONF_DIR/$filename >> $REDIS_CONF_DIR/$PROCESS_NAME.conf - - # replace placeholder variables in init.d script - cp $VAGRANT_DIR/sentinel_init_script /etc/init.d/$PROCESS_NAME - sed -i "s/{{ PROCESS_NAME }}/$PROCESS_NAME/g" /etc/init.d/$PROCESS_NAME - # need to read the config file to find out what port this instance will run on - port=`grep port $VAGRANT_SENTINEL_CONF_DIR/$filename | cut -f 2 -d " "` - sed -i "s/{{ PORT }}/$port/g" /etc/init.d/$PROCESS_NAME - chmod 755 /etc/init.d/$PROCESS_NAME - - # and tell update-rc.d about it - update-rc.d $PROCESS_NAME defaults 99 - - # save the $PROCESS_NAME into installed instances file - echo $PROCESS_NAME >> $SENTINEL_INSTALLED_INSTANCES_FILE - - # start redis - /etc/init.d/$PROCESS_NAME start -done diff --git a/build_tools/redis-configs/001-master b/build_tools/redis-configs/001-master deleted file mode 100644 index 8591f1a..0000000 --- a/build_tools/redis-configs/001-master +++ /dev/null @@ -1,8 +0,0 @@ -pidfile /var/run/redis-master.pid -bind * -port 6379 -daemonize yes -unixsocket /tmp/redis_master.sock -unixsocketperm 777 -dbfilename master.rdb -dir /var/lib/redis/backups diff --git a/build_tools/redis-configs/002-slave b/build_tools/redis-configs/002-slave deleted file mode 100644 index 13eb77e..0000000 --- a/build_tools/redis-configs/002-slave +++ /dev/null @@ -1,10 +0,0 @@ -pidfile /var/run/redis-slave.pid -bind * -port 6380 -daemonize yes -unixsocket /tmp/redis-slave.sock -unixsocketperm 777 -dbfilename slave.rdb -dir /var/lib/redis/backups - -slaveof 127.0.0.1 6379 diff --git a/build_tools/redis_init_script b/build_tools/redis_init_script deleted file mode 100755 index 04cb2db..0000000 --- a/build_tools/redis_init_script +++ /dev/null @@ -1,49 +0,0 @@ -#!/bin/sh - -### BEGIN INIT INFO -# Provides: redis-server -# Required-Start: $syslog -# Required-Stop: $syslog -# Default-Start: 2 3 4 5 -# Default-Stop: 0 1 6 -# Short-Description: Start redis-server at boot time -# Description: Control redis-server. -### END INIT INFO - -REDISPORT={{ PORT }} -PIDFILE=/var/run/{{ PROCESS_NAME }}.pid -CONF=/var/lib/redis/conf/{{ PROCESS_NAME }}.conf - -EXEC=/var/lib/redis/bin/redis-server -CLIEXEC=/var/lib/redis/bin/redis-cli - -case "$1" in - start) - if [ -f $PIDFILE ] - then - echo "$PIDFILE exists, process is already running or crashed" - else - echo "Starting Redis server..." - $EXEC $CONF - fi - ;; - stop) - if [ ! -f $PIDFILE ] - then - echo "$PIDFILE does not exist, process is not running" - else - PID=$(cat $PIDFILE) - echo "Stopping ..." - $CLIEXEC -p $REDISPORT shutdown - while [ -x /proc/${PID} ] - do - echo "Waiting for Redis to shutdown ..." - sleep 1 - done - echo "Redis stopped" - fi - ;; - *) - echo "Please use start or stop as first argument" - ;; -esac diff --git a/build_tools/redis_vars.sh b/build_tools/redis_vars.sh deleted file mode 100755 index c52dd4c..0000000 --- a/build_tools/redis_vars.sh +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env bash - -VAGRANT_DIR=/home/vagrant/redis-py/build_tools -VAGRANT_REDIS_CONF_DIR=$VAGRANT_DIR/redis-configs -VAGRANT_SENTINEL_CONF_DIR=$VAGRANT_DIR/sentinel-configs -REDIS_VERSION=3.2.0 -REDIS_DOWNLOAD_DIR=/home/vagrant/redis-downloads -REDIS_PACKAGE=redis-$REDIS_VERSION.tar.gz -REDIS_BUILD_DIR=$REDIS_DOWNLOAD_DIR/redis-$REDIS_VERSION -REDIS_DIR=/var/lib/redis -REDIS_BIN_DIR=$REDIS_DIR/bin -REDIS_CONF_DIR=$REDIS_DIR/conf -REDIS_SAVE_DIR=$REDIS_DIR/backups -REDIS_INSTALLED_INSTANCES_FILE=$REDIS_DIR/redis-instances -SENTINEL_INSTALLED_INSTANCES_FILE=$REDIS_DIR/sentinel-instances - -function uninstall_instance() { - # Expects $1 to be the init.d filename, e.g. redis-nodename or - # sentinel-nodename - - if [ -a /etc/init.d/$1 ]; then - - echo "======================================" - echo "UNINSTALLING REDIS SERVER: $1" - echo "======================================" - - /etc/init.d/$1 stop - update-rc.d -f $1 remove - rm -f /etc/init.d/$1 - fi; - rm -f $REDIS_CONF_DIR/$1.conf -} - -function uninstall_all_redis_instances() { - if [ -a $REDIS_INSTALLED_INSTANCES_FILE ]; then - cat $REDIS_INSTALLED_INSTANCES_FILE | while read line; do - uninstall_instance $line; - done; - fi -} - -function uninstall_all_sentinel_instances() { - if [ -a $SENTINEL_INSTALLED_INSTANCES_FILE ]; then - cat $SENTINEL_INSTALLED_INSTANCES_FILE | while read line; do - uninstall_instance $line; - done; - fi -} diff --git a/build_tools/sentinel-configs/001-1 b/build_tools/sentinel-configs/001-1 deleted file mode 100644 index eccc3d1..0000000 --- a/build_tools/sentinel-configs/001-1 +++ /dev/null @@ -1,6 +0,0 @@ -pidfile /var/run/sentinel-1.pid -port 26379 -daemonize yes - -# short timeout for sentinel tests -sentinel down-after-milliseconds mymaster 500 diff --git a/build_tools/sentinel-configs/002-2 b/build_tools/sentinel-configs/002-2 deleted file mode 100644 index 0cd2801..0000000 --- a/build_tools/sentinel-configs/002-2 +++ /dev/null @@ -1,6 +0,0 @@ -pidfile /var/run/sentinel-2.pid -port 26380 -daemonize yes - -# short timeout for sentinel tests -sentinel down-after-milliseconds mymaster 500 diff --git a/build_tools/sentinel-configs/003-3 b/build_tools/sentinel-configs/003-3 deleted file mode 100644 index c7f4fcd..0000000 --- a/build_tools/sentinel-configs/003-3 +++ /dev/null @@ -1,6 +0,0 @@ -pidfile /var/run/sentinel-3.pid -port 26381 -daemonize yes - -# short timeout for sentinel tests -sentinel down-after-milliseconds mymaster 500 diff --git a/build_tools/sentinel_init_script b/build_tools/sentinel_init_script deleted file mode 100755 index 1d94804..0000000 --- a/build_tools/sentinel_init_script +++ /dev/null @@ -1,49 +0,0 @@ -#!/bin/sh - -### BEGIN INIT INFO -# Provides: redis-sentintel -# Required-Start: $syslog -# Required-Stop: $syslog -# Default-Start: 2 3 4 5 -# Default-Stop: 0 1 6 -# Short-Description: Start redis-sentinel at boot time -# Description: Control redis-sentinel. -### END INIT INFO - -SENTINELPORT={{ PORT }} -PIDFILE=/var/run/{{ PROCESS_NAME }}.pid -CONF=/var/lib/redis/conf/{{ PROCESS_NAME }}.conf - -EXEC=/var/lib/redis/bin/redis-sentinel -CLIEXEC=/var/lib/redis/bin/redis-cli - -case "$1" in - start) - if [ -f $PIDFILE ] - then - echo "$PIDFILE exists, process is already running or crashed" - else - echo "Starting Redis Sentinel..." - $EXEC $CONF - fi - ;; - stop) - if [ ! -f $PIDFILE ] - then - echo "$PIDFILE does not exist, process is not running" - else - PID=$(cat $PIDFILE) - echo "Stopping ..." - $CLIEXEC -p $SENTINELPORT shutdown - while [ -x /proc/${PID} ] - do - echo "Waiting for Sentinel to shutdown ..." - sleep 1 - done - echo "Sentinel stopped" - fi - ;; - *) - echo "Please use start or stop as first argument" - ;; -esac diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..247688a --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,66 @@ +version: "3.8" + +services: + master: + build: docker/master + ports: + - "6379:6379" + + slave: + build: docker/slave + depends_on: + - "master" + ports: + - "6380:6380" + + sentinel_1: + build: docker/sentinel_1 + depends_on: + - "slave" + ports: + - "26379:26379" + + sentinel_2: + build: docker/sentinel_2 + depends_on: + - "slave" + ports: + - "26380:26380" + + sentinel_3: + build: docker/sentinel_3 + depends_on: + - "slave" + ports: + - "26381:26381" + + test: + build: . + depends_on: + - "sentinel_3" + environment: + REDIS_MASTER_HOST: master + REDIS_MASTER_PORT: 6379 + CI: + CI_BUILD_ID: + CI_BUILD_URL: + CI_JOB_ID: + CODECOV_ENV: + CODECOV_SLUG: + CODECOV_TOKEN: + CODECOV_URL: + SHIPPABLE: + TRAVIS: + TRAVIS_BRANCH: + TRAVIS_COMMIT: + TRAVIS_JOB_ID: + TRAVIS_JOB_NUMBER: + TRAVIS_OS_NAME: + TRAVIS_PULL_REQUEST: + TRAVIS_REPO_SLUG: + TRAVIS_TAG: + VCS_BRANCH_NAME: + VCS_COMMIT_ID: + VCS_PULL_REQUEST: + VCS_SLUG: + VCS_TAG: diff --git a/docker-entry.sh b/docker-entry.sh new file mode 100755 index 0000000..f63a8c5 --- /dev/null +++ b/docker-entry.sh @@ -0,0 +1,19 @@ +#!/bin/bash + +# This is the entrypoint for "make test". It invokes Tox. If running +# outside the CI environment, it disables uploading the coverage report to codecov + +set -eu + +REDIS_MASTER="${REDIS_MASTER_HOST}":"${REDIS_MASTER_PORT}" +echo "Testing against Redis Server: ${REDIS_MASTER}" + +# skip the "codecov" env if not running on Travis +if [ -z ${TRAVIS-} ]; then + echo "Skipping codecov" + export TOX_SKIP_ENV="codecov" +fi + +# use the wait-for-it util to ensure the server is running before invoking Tox +util/wait-for-it.sh ${REDIS_MASTER} -- tox -- --redis-url=redis://"${REDIS_MASTER}"/9 + diff --git a/docker/base/Dockerfile b/docker/base/Dockerfile new file mode 100644 index 0000000..ab97386 --- /dev/null +++ b/docker/base/Dockerfile @@ -0,0 +1 @@ +FROM redis:6.0.5-buster diff --git a/docker/master/Dockerfile b/docker/master/Dockerfile new file mode 100644 index 0000000..1f4bd73 --- /dev/null +++ b/docker/master/Dockerfile @@ -0,0 +1,7 @@ +FROM redis-py-base:latest + +COPY redis.conf /usr/local/etc/redis/redis.conf + +EXPOSE 6379 + +CMD [ "redis-server", "/usr/local/etc/redis/redis.conf" ] diff --git a/docker/master/redis.conf b/docker/master/redis.conf new file mode 100644 index 0000000..ed00766 --- /dev/null +++ b/docker/master/redis.conf @@ -0,0 +1,3 @@ +bind master 127.0.0.1 +port 6379 +save "" diff --git a/docker/sentinel_1/Dockerfile b/docker/sentinel_1/Dockerfile new file mode 100644 index 0000000..66f6a75 --- /dev/null +++ b/docker/sentinel_1/Dockerfile @@ -0,0 +1,7 @@ +FROM redis-py-base:latest + +COPY sentinel.conf /usr/local/etc/redis/sentinel.conf + +EXPOSE 26379 + +CMD [ "redis-sentinel", "/usr/local/etc/redis/sentinel.conf" ] diff --git a/docker/sentinel_1/sentinel.conf b/docker/sentinel_1/sentinel.conf new file mode 100644 index 0000000..fc9aa68 --- /dev/null +++ b/docker/sentinel_1/sentinel.conf @@ -0,0 +1,7 @@ +bind sentinel_1 127.0.0.1 +port 26379 + +sentinel monitor redis-py-test master 6379 2 +sentinel down-after-milliseconds redis-py-test 5000 +sentinel failover-timeout redis-py-test 60000 +sentinel parallel-syncs redis-py-test 1 diff --git a/docker/sentinel_2/Dockerfile b/docker/sentinel_2/Dockerfile new file mode 100644 index 0000000..1c0bb92 --- /dev/null +++ b/docker/sentinel_2/Dockerfile @@ -0,0 +1,7 @@ +FROM redis-py-base:latest + +COPY sentinel.conf /usr/local/etc/redis/sentinel.conf + +EXPOSE 26380 + +CMD [ "redis-sentinel", "/usr/local/etc/redis/sentinel.conf" ] diff --git a/docker/sentinel_2/sentinel.conf b/docker/sentinel_2/sentinel.conf new file mode 100644 index 0000000..264443c --- /dev/null +++ b/docker/sentinel_2/sentinel.conf @@ -0,0 +1,7 @@ +bind sentinel_2 127.0.0.1 +port 26380 + +sentinel monitor redis-py-test master 6379 2 +sentinel down-after-milliseconds redis-py-test 5000 +sentinel failover-timeout redis-py-test 60000 +sentinel parallel-syncs redis-py-test 1 diff --git a/docker/sentinel_3/Dockerfile b/docker/sentinel_3/Dockerfile new file mode 100644 index 0000000..cf1ec21 --- /dev/null +++ b/docker/sentinel_3/Dockerfile @@ -0,0 +1,7 @@ +FROM redis-py-base:latest + +COPY sentinel.conf /usr/local/etc/redis/sentinel.conf + +EXPOSE 26381 + +CMD [ "redis-sentinel", "/usr/local/etc/redis/sentinel.conf" ] diff --git a/docker/sentinel_3/sentinel.conf b/docker/sentinel_3/sentinel.conf new file mode 100644 index 0000000..b0827f1 --- /dev/null +++ b/docker/sentinel_3/sentinel.conf @@ -0,0 +1,7 @@ +bind sentinel_3 127.0.0.1 +port 26381 + +sentinel monitor redis-py-test master 6379 2 +sentinel down-after-milliseconds redis-py-test 5000 +sentinel failover-timeout redis-py-test 60000 +sentinel parallel-syncs redis-py-test 1 diff --git a/docker/slave/Dockerfile b/docker/slave/Dockerfile new file mode 100644 index 0000000..7b6bdbe --- /dev/null +++ b/docker/slave/Dockerfile @@ -0,0 +1,7 @@ +FROM redis-py-base:latest + +COPY redis.conf /usr/local/etc/redis/redis.conf + +EXPOSE 6380 + +CMD [ "redis-server", "/usr/local/etc/redis/redis.conf" ] diff --git a/docker/slave/redis.conf b/docker/slave/redis.conf new file mode 100644 index 0000000..629ac70 --- /dev/null +++ b/docker/slave/redis.conf @@ -0,0 +1,4 @@ +bind slave 127.0.0.1 +port 6380 +save "" +slaveof master 6379 @@ -36,7 +36,7 @@ python_requires = >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.* hiredis = hiredis>=0.1.3 [flake8] -exclude = .venv,.tox,dist,docs,build,*.egg,redis_install +exclude = .venv,.tox,dist,docs,build,*.egg,redis_install,env,venv,.undodir [bdist_wheel] universal = 1 diff --git a/tests/conftest.py b/tests/conftest.py index 7d64609..caca8cc 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -4,6 +4,7 @@ import pytest import redis from mock import Mock +from redis._compat import urlparse from distutils.version import StrictVersion @@ -156,6 +157,13 @@ def mock_cluster_resp_slaves(request, **kwargs): return _gen_cluster_mock_resp(r, response) +@pytest.fixture(scope="session") +def master_host(request): + url = request.config.getoption("--redis-url") + parts = urlparse(url) + yield parts.hostname + + def wait_for_command(client, monitor, command): # issue a command with a key name that's local to this process. # if we find a command with our key before the command we're waiting diff --git a/tests/test_commands.py b/tests/test_commands.py index adaa9fc..c68f14c 100644 --- a/tests/test_commands.py +++ b/tests/test_commands.py @@ -433,12 +433,10 @@ class TestRedisCommands(object): def test_slowlog_get_limit(self, r, slowlog): assert r.slowlog_reset() r.get('foo') - r.get('bar') slowlog = r.slowlog_get(1) assert isinstance(slowlog, list) - commands = [log['command'] for log in slowlog] - assert b'GET foo' not in commands - assert b'GET bar' in commands + # only one command, based on the number we passed to slowlog_get() + assert len(slowlog) == 1 def test_slowlog_length(self, r, slowlog): r.get('foo') diff --git a/tests/test_connection_pool.py b/tests/test_connection_pool.py index 080e3df..c49ecda 100644 --- a/tests/test_connection_pool.py +++ b/tests/test_connection_pool.py @@ -43,21 +43,25 @@ class TestConnectionPool(object): assert isinstance(connection, DummyConnection) assert connection.kwargs == connection_kwargs - def test_multiple_connections(self): - pool = self.get_pool() + def test_multiple_connections(self, master_host): + connection_kwargs = {'host': master_host} + pool = self.get_pool(connection_kwargs=connection_kwargs) c1 = pool.get_connection('_') c2 = pool.get_connection('_') assert c1 != c2 - def test_max_connections(self): - pool = self.get_pool(max_connections=2) + def test_max_connections(self, master_host): + connection_kwargs = {'host': master_host} + pool = self.get_pool(max_connections=2, + connection_kwargs=connection_kwargs) pool.get_connection('_') pool.get_connection('_') with pytest.raises(redis.ConnectionError): pool.get_connection('_') - def test_reuse_previously_released_connection(self): - pool = self.get_pool() + def test_reuse_previously_released_connection(self, master_host): + connection_kwargs = {'host': master_host} + pool = self.get_pool(connection_kwargs=connection_kwargs) c1 = pool.get_connection('_') pool.release(c1) c2 = pool.get_connection('_') @@ -98,22 +102,25 @@ class TestBlockingConnectionPool(object): **connection_kwargs) return pool - def test_connection_creation(self): - connection_kwargs = {'foo': 'bar', 'biz': 'baz'} + def test_connection_creation(self, master_host): + connection_kwargs = {'foo': 'bar', 'biz': 'baz', 'host': master_host} pool = self.get_pool(connection_kwargs=connection_kwargs) connection = pool.get_connection('_') assert isinstance(connection, DummyConnection) assert connection.kwargs == connection_kwargs - def test_multiple_connections(self): - pool = self.get_pool() + def test_multiple_connections(self, master_host): + connection_kwargs = {'host': master_host} + pool = self.get_pool(connection_kwargs=connection_kwargs) c1 = pool.get_connection('_') c2 = pool.get_connection('_') assert c1 != c2 - def test_connection_pool_blocks_until_timeout(self): + def test_connection_pool_blocks_until_timeout(self, master_host): "When out of connections, block for timeout seconds, then raise" - pool = self.get_pool(max_connections=1, timeout=0.1) + connection_kwargs = {'host': master_host} + pool = self.get_pool(max_connections=1, timeout=0.1, + connection_kwargs=connection_kwargs) pool.get_connection('_') start = time.time() @@ -122,25 +129,28 @@ class TestBlockingConnectionPool(object): # we should have waited at least 0.1 seconds assert time.time() - start >= 0.1 - def connection_pool_blocks_until_another_connection_released(self): + def test_connection_pool_blocks_until_conn_available(self, master_host): """ When out of connections, block until another connection is released to the pool """ - pool = self.get_pool(max_connections=1, timeout=2) + connection_kwargs = {'host': master_host} + pool = self.get_pool(max_connections=1, timeout=2, + connection_kwargs=connection_kwargs) c1 = pool.get_connection('_') def target(): time.sleep(0.1) pool.release(c1) - Thread(target=target).start() start = time.time() + Thread(target=target).start() pool.get_connection('_') assert time.time() - start >= 0.1 - def test_reuse_previously_released_connection(self): - pool = self.get_pool() + def test_reuse_previously_released_connection(self, master_host): + connection_kwargs = {'host': master_host} + pool = self.get_pool(connection_kwargs=connection_kwargs) c1 = pool.get_connection('_') pool.release(c1) c2 = pool.get_connection('_') diff --git a/tests/test_multiprocessing.py b/tests/test_multiprocessing.py index 3f81606..235e3ce 100644 --- a/tests/test_multiprocessing.py +++ b/tests/test_multiprocessing.py @@ -30,12 +30,12 @@ class TestMultiprocessing(object): request=request, single_connection_client=False) - def test_close_connection_in_child(self): + def test_close_connection_in_child(self, master_host): """ A connection owned by a parent and closed by a child doesn't destroy the file descriptors so a parent can still use it. """ - conn = Connection() + conn = Connection(host=master_host) conn.send_command('ping') assert conn.read_response() == b'PONG' @@ -56,12 +56,12 @@ class TestMultiprocessing(object): conn.send_command('ping') assert conn.read_response() == b'PONG' - def test_close_connection_in_parent(self): + def test_close_connection_in_parent(self, master_host): """ A connection owned by a parent is unusable by a child if the parent (the owning process) closes the connection. """ - conn = Connection() + conn = Connection(host=master_host) conn.send_command('ping') assert conn.read_response() == b'PONG' @@ -84,12 +84,12 @@ class TestMultiprocessing(object): assert proc.exitcode == 0 @pytest.mark.parametrize('max_connections', [1, 2, None]) - def test_pool(self, max_connections): + def test_pool(self, max_connections, master_host): """ A child will create its own connections when using a pool created by a parent. """ - pool = ConnectionPool.from_url('redis://localhost', + pool = ConnectionPool.from_url('redis://{}'.format(master_host), max_connections=max_connections) conn = pool.get_connection('ping') @@ -119,12 +119,12 @@ class TestMultiprocessing(object): assert conn.read_response() == b'PONG' @pytest.mark.parametrize('max_connections', [1, 2, None]) - def test_close_pool_in_main(self, max_connections): + def test_close_pool_in_main(self, max_connections, master_host): """ A child process that uses the same pool as its parent isn't affected when the parent disconnects all connections within the pool. """ - pool = ConnectionPool.from_url('redis://localhost', + pool = ConnectionPool.from_url('redis://{}'.format(master_host), max_connections=max_connections) conn = pool.get_connection('ping') diff --git a/tests/test_pubsub.py b/tests/test_pubsub.py index 31b60be..bf13483 100644 --- a/tests/test_pubsub.py +++ b/tests/test_pubsub.py @@ -475,11 +475,8 @@ class TestPubSubSubcommands(object): p.subscribe('foo', 'bar', 'baz', 'quux') for i in range(4): assert wait_for_message(p)['type'] == 'subscribe' - channels = sorted(r.pubsub_channels()) - # assert channels == [b'bar', b'baz', b'foo', b'quux'] - if channels != [b'bar', b'baz', b'foo', b'quux']: - import pdb - pdb.set_trace() + expected = [b'bar', b'baz', b'foo', b'quux'] + assert all([channel in r.pubsub_channels() for channel in expected]) @skip_if_server_version_lt('2.8.0') def test_pubsub_numsub(self, r): diff --git a/tests/test_sentinel.py b/tests/test_sentinel.py index 1081e2b..c247c72 100644 --- a/tests/test_sentinel.py +++ b/tests/test_sentinel.py @@ -1,3 +1,5 @@ +import socket + import pytest from redis import exceptions @@ -7,6 +9,11 @@ from redis._compat import next import redis.sentinel +@pytest.fixture(scope="module") +def master_ip(master_host): + yield socket.gethostbyname(master_host) + + class SentinelTestClient(object): def __init__(self, cluster, id): self.cluster = cluster @@ -54,10 +61,10 @@ class SentinelTestCluster(object): @pytest.fixture() -def cluster(request): +def cluster(request, master_ip): def teardown(): redis.sentinel.Redis = saved_Redis - cluster = SentinelTestCluster() + cluster = SentinelTestCluster(ip=master_ip) saved_Redis = redis.sentinel.Redis redis.sentinel.Redis = cluster.client request.addfinalizer(teardown) @@ -69,9 +76,9 @@ def sentinel(request, cluster): return Sentinel([('foo', 26379), ('bar', 26379)]) -def test_discover_master(sentinel): +def test_discover_master(sentinel, master_ip): address = sentinel.discover_master('mymaster') - assert address == ('127.0.0.1', 6379) + assert address == (master_ip, 6379) def test_discover_master_error(sentinel): @@ -79,32 +86,32 @@ def test_discover_master_error(sentinel): sentinel.discover_master('xxx') -def test_discover_master_sentinel_down(cluster, sentinel): +def test_discover_master_sentinel_down(cluster, sentinel, master_ip): # Put first sentinel 'foo' down cluster.nodes_down.add(('foo', 26379)) address = sentinel.discover_master('mymaster') - assert address == ('127.0.0.1', 6379) + assert address == (master_ip, 6379) # 'bar' is now first sentinel assert sentinel.sentinels[0].id == ('bar', 26379) -def test_discover_master_sentinel_timeout(cluster, sentinel): +def test_discover_master_sentinel_timeout(cluster, sentinel, master_ip): # Put first sentinel 'foo' down cluster.nodes_timeout.add(('foo', 26379)) address = sentinel.discover_master('mymaster') - assert address == ('127.0.0.1', 6379) + assert address == (master_ip, 6379) # 'bar' is now first sentinel assert sentinel.sentinels[0].id == ('bar', 26379) -def test_master_min_other_sentinels(cluster): +def test_master_min_other_sentinels(cluster, master_ip): sentinel = Sentinel([('foo', 26379)], min_other_sentinels=1) # min_other_sentinels with pytest.raises(MasterNotFoundError): sentinel.discover_master('mymaster') cluster.master['num-other-sentinels'] = 2 address = sentinel.discover_master('mymaster') - assert address == ('127.0.0.1', 6379) + assert address == (master_ip, 6379) def test_master_odown(cluster, sentinel): @@ -153,10 +160,10 @@ def test_discover_slaves(cluster, sentinel): ('slave0', 1234), ('slave1', 1234)] -def test_master_for(cluster, sentinel): +def test_master_for(cluster, sentinel, master_ip): master = sentinel.master_for('mymaster', db=9) assert master.ping() - assert master.connection_pool.master_address == ('127.0.0.1', 6379) + assert master.connection_pool.master_address == (master_ip, 6379) # Use internal connection check master = sentinel.master_for('mymaster', db=9, check_connection=True) @@ -179,7 +186,7 @@ def test_slave_for_slave_not_found_error(cluster, sentinel): slave.ping() -def test_slave_round_robin(cluster, sentinel): +def test_slave_round_robin(cluster, sentinel, master_ip): cluster.slaves = [ {'ip': 'slave0', 'port': 6379, 'is_odown': False, 'is_sdown': False}, {'ip': 'slave1', 'port': 6379, 'is_odown': False, 'is_sdown': False}, @@ -189,6 +196,6 @@ def test_slave_round_robin(cluster, sentinel): assert next(rotator) in (('slave0', 6379), ('slave1', 6379)) assert next(rotator) in (('slave0', 6379), ('slave1', 6379)) # Fallback to master - assert next(rotator) == ('127.0.0.1', 6379) + assert next(rotator) == (master_ip, 6379) with pytest.raises(SlaveNotFoundError): next(rotator) @@ -1,6 +1,9 @@ +[pytest] +addopts = -s + [tox] minversion = 2.4 -envlist = {py27,py35,py36,py37,py38,py,py3}-{plain,hiredis}, flake8 +envlist = {py27,py35,py36,py37,py38,pypy,pypy3}-{plain,hiredis}, flake8, covreport, codecov [testenv] deps = @@ -9,7 +12,9 @@ deps = pytest >= 2.7.0 extras = hiredis: hiredis -commands = {envpython} -b -m coverage run -m pytest -W always {posargs} +commands = + {envpython} -b -m coverage run -p -m pytest -W always {posargs} + {envpython} -b -m coverage combine --append [testenv:flake8] basepython = python3.6 @@ -17,3 +22,32 @@ deps = flake8 commands = flake8 skipsdist = true skip_install = true + +[testenv:pypy-plain] +basepython = pypy + +[testenv:pypy-hiredis] +basepython = pypy + +[testenv:pypy3-plain] +basepython = pypy3 + +[testenv:pypy3-hiredis] +basepython = pypy3 + +[testenv:codecov] +deps = codecov +commands = codecov +passenv = + REDIS_* + CI + CI_* + CODECOV_* + SHIPPABLE + TRAVIS + TRAVIS_* + VCS_* + +[testenv:covreport] +deps = coverage +commands = coverage report diff --git a/util/wait-for-it.sh b/util/wait-for-it.sh new file mode 100755 index 0000000..c7a08d4 --- /dev/null +++ b/util/wait-for-it.sh @@ -0,0 +1,204 @@ +#!/usr/bin/env bash +# Use this script to test if a given TCP host/port are available +# +# Copyright (c) 2016 Giles Hall +# The MIT License (MIT) +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of +# this software and associated documentation files (the "Software"), to deal in +# the Software without restriction, including without limitation the rights to +# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +# of the Software, and to permit persons to whom the Software is furnished to do +# so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +WAITFORIT_cmdname=${0##*/} + +echoerr() { if [[ $WAITFORIT_QUIET -ne 1 ]]; then echo "$@" 1>&2; fi } + +usage() +{ + cat << USAGE >&2 +Usage: + $WAITFORIT_cmdname host:port [-s] [-t timeout] [-- command args] + -h HOST | --host=HOST Host or IP under test + -p PORT | --port=PORT TCP port under test + Alternatively, you specify the host and port as host:port + -s | --strict Only execute subcommand if the test succeeds + -q | --quiet Don't output any status messages + -t TIMEOUT | --timeout=TIMEOUT + Timeout in seconds, zero for no timeout + -- COMMAND ARGS Execute command with args after the test finishes +USAGE + exit 1 +} + +wait_for() +{ + if [[ $WAITFORIT_TIMEOUT -gt 0 ]]; then + echoerr "$WAITFORIT_cmdname: waiting $WAITFORIT_TIMEOUT seconds for $WAITFORIT_HOST:$WAITFORIT_PORT" + else + echoerr "$WAITFORIT_cmdname: waiting for $WAITFORIT_HOST:$WAITFORIT_PORT without a timeout" + fi + WAITFORIT_start_ts=$(date +%s) + while : + do + if [[ $WAITFORIT_ISBUSY -eq 1 ]]; then + nc -z $WAITFORIT_HOST $WAITFORIT_PORT + WAITFORIT_result=$? + else + (echo > /dev/tcp/$WAITFORIT_HOST/$WAITFORIT_PORT) >/dev/null 2>&1 + WAITFORIT_result=$? + fi + if [[ $WAITFORIT_result -eq 0 ]]; then + WAITFORIT_end_ts=$(date +%s) + echoerr "$WAITFORIT_cmdname: $WAITFORIT_HOST:$WAITFORIT_PORT is available after $((WAITFORIT_end_ts - WAITFORIT_start_ts)) seconds" + break + fi + sleep 1 + done + return $WAITFORIT_result +} + +wait_for_wrapper() +{ + # In order to support SIGINT during timeout: http://unix.stackexchange.com/a/57692 + if [[ $WAITFORIT_QUIET -eq 1 ]]; then + timeout $WAITFORIT_BUSYTIMEFLAG $WAITFORIT_TIMEOUT $0 --quiet --child --host=$WAITFORIT_HOST --port=$WAITFORIT_PORT --timeout=$WAITFORIT_TIMEOUT & + else + timeout $WAITFORIT_BUSYTIMEFLAG $WAITFORIT_TIMEOUT $0 --child --host=$WAITFORIT_HOST --port=$WAITFORIT_PORT --timeout=$WAITFORIT_TIMEOUT & + fi + WAITFORIT_PID=$! + trap "kill -INT -$WAITFORIT_PID" INT + wait $WAITFORIT_PID + WAITFORIT_RESULT=$? + if [[ $WAITFORIT_RESULT -ne 0 ]]; then + echoerr "$WAITFORIT_cmdname: timeout occurred after waiting $WAITFORIT_TIMEOUT seconds for $WAITFORIT_HOST:$WAITFORIT_PORT" + fi + return $WAITFORIT_RESULT +} + +# process arguments +while [[ $# -gt 0 ]] +do + case "$1" in + *:* ) + WAITFORIT_hostport=(${1//:/ }) + WAITFORIT_HOST=${WAITFORIT_hostport[0]} + WAITFORIT_PORT=${WAITFORIT_hostport[1]} + shift 1 + ;; + --child) + WAITFORIT_CHILD=1 + shift 1 + ;; + -q | --quiet) + WAITFORIT_QUIET=1 + shift 1 + ;; + -s | --strict) + WAITFORIT_STRICT=1 + shift 1 + ;; + -h) + WAITFORIT_HOST="$2" + if [[ $WAITFORIT_HOST == "" ]]; then break; fi + shift 2 + ;; + --host=*) + WAITFORIT_HOST="${1#*=}" + shift 1 + ;; + -p) + WAITFORIT_PORT="$2" + if [[ $WAITFORIT_PORT == "" ]]; then break; fi + shift 2 + ;; + --port=*) + WAITFORIT_PORT="${1#*=}" + shift 1 + ;; + -t) + WAITFORIT_TIMEOUT="$2" + if [[ $WAITFORIT_TIMEOUT == "" ]]; then break; fi + shift 2 + ;; + --timeout=*) + WAITFORIT_TIMEOUT="${1#*=}" + shift 1 + ;; + --) + shift + WAITFORIT_CLI=("$@") + break + ;; + --help) + usage + ;; + *) + echoerr "Unknown argument: $1" + usage + ;; + esac +done + +if [[ "$WAITFORIT_HOST" == "" || "$WAITFORIT_PORT" == "" ]]; then + echoerr "Error: you need to provide a host and port to test." + usage +fi + +WAITFORIT_TIMEOUT=${WAITFORIT_TIMEOUT:-15} +WAITFORIT_STRICT=${WAITFORIT_STRICT:-0} +WAITFORIT_CHILD=${WAITFORIT_CHILD:-0} +WAITFORIT_QUIET=${WAITFORIT_QUIET:-0} + +# Check to see if timeout is from busybox? +WAITFORIT_TIMEOUT_PATH=$(type -p timeout) +WAITFORIT_TIMEOUT_PATH=$(realpath $WAITFORIT_TIMEOUT_PATH 2>/dev/null || readlink -f $WAITFORIT_TIMEOUT_PATH) + +WAITFORIT_BUSYTIMEFLAG="" +if [[ $WAITFORIT_TIMEOUT_PATH =~ "busybox" ]]; then + WAITFORIT_ISBUSY=1 + # Check if busybox timeout uses -t flag + # (recent Alpine versions don't support -t anymore) + if timeout &>/dev/stdout | grep -q -e '-t '; then + WAITFORIT_BUSYTIMEFLAG="-t" + fi +else + WAITFORIT_ISBUSY=0 +fi + +if [[ $WAITFORIT_CHILD -gt 0 ]]; then + wait_for + WAITFORIT_RESULT=$? + exit $WAITFORIT_RESULT +else + if [[ $WAITFORIT_TIMEOUT -gt 0 ]]; then + wait_for_wrapper + WAITFORIT_RESULT=$? + else + wait_for + WAITFORIT_RESULT=$? + fi +fi + +if [[ $WAITFORIT_CLI != "" ]]; then + if [[ $WAITFORIT_RESULT -ne 0 && $WAITFORIT_STRICT -eq 1 ]]; then + echoerr "$WAITFORIT_cmdname: strict mode, refusing to execute subprocess" + exit $WAITFORIT_RESULT + fi + exec "${WAITFORIT_CLI[@]}" +else + exit $WAITFORIT_RESULT +fi + diff --git a/vagrant/Vagrantfile b/vagrant/Vagrantfile deleted file mode 100644 index 3ee7aee..0000000 --- a/vagrant/Vagrantfile +++ /dev/null @@ -1,27 +0,0 @@ -# -*- mode: ruby -*- -# vi: set ft=ruby : - -# Vagrantfile API/syntax version. Don't touch unless you know what you're doing! -VAGRANTFILE_API_VERSION = "2" - -Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| - # ubuntu 64bit image - config.vm.box = "hashicorp/precise64" - - # map the root of redis-py to /home/vagrant/redis-py - config.vm.synced_folder "../", "/home/vagrant/redis-py" - - # install the redis server - config.vm.provision :shell, :path => "../build_tools/bootstrap.sh" - config.vm.provision :shell, :path => "../build_tools/build_redis.sh" - config.vm.provision :shell, :path => "../build_tools/install_redis.sh" - config.vm.provision :shell, :path => "../build_tools/install_sentinel.sh" - config.vm.provision :file, :source => "../build_tools/.bash_profile", :destination => "/home/vagrant/.bash_profile" - - # setup forwarded ports - config.vm.network "forwarded_port", guest: 6379, host: 6379 - config.vm.network "forwarded_port", guest: 6380, host: 6380 - config.vm.network "forwarded_port", guest: 26379, host: 26379 - config.vm.network "forwarded_port", guest: 26380, host: 26380 - config.vm.network "forwarded_port", guest: 26381, host: 26381 -end |