diff options
| -rw-r--r-- | CONTRIBUTING.md | 125 | ||||
| -rw-r--r-- | fabfile.py | 111 | ||||
| -rwxr-xr-x | setup.py | 1 | ||||
| -rw-r--r-- | tasks.py | 178 |
4 files changed, 261 insertions, 154 deletions
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 24bd835e..56cb6ca4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -67,9 +67,9 @@ The tables below list all prerequisites along with the minimum required version If Python is already installed in your machine, run the following commands to validate the versions: -```shell -python -V -pip freeze | grep pyperclip +```sh +$ python -V +$ pip freeze | grep pyperclip ``` If your versions are lower than the prerequisite versions, you should update. @@ -95,7 +95,7 @@ If you do not already have Python installed on your machine, we recommend using 1. Open a Terminal / Command Line / Bash Shell in your projects directory (_i.e.: `/yourprojectdirectory/`_) 2. Clone your fork of cmd2 -```shell +```sh $ git clone https://github.com/yourUsername/cmd2.git ``` @@ -108,7 +108,7 @@ This will download the entire cmd2 repo to your projects directory. 1. Change directory to the new cmd2 directory (`cd cmd2`) 2. Add a remote to the official cmd2 repo: -```shell +```sh $ git remote add upstream https://github.com/python-cmd2/cmd2.git ``` @@ -124,7 +124,7 @@ Do this prior to every time you create a branch for a PR: 1. Make sure you are on the `master` branch - > ```shell + > ```sh > $ git status > On branch master > Your branch is up-to-date with 'origin/master'. @@ -132,13 +132,13 @@ Do this prior to every time you create a branch for a PR: > If your aren't on `master`, resolve outstanding files / commits and checkout the `master` branch - > ```shell + > ```sh > $ git checkout master > ``` 2. Do A Pull with Rebase Against `upstream` - > ```shell + > ```sh > $ git pull --rebase upstream master > ``` @@ -146,7 +146,7 @@ Do this prior to every time you create a branch for a PR: 3. (_Optional_) Force push your updated master branch to your GitHub fork - > ```shell + > ```sh > $ git push origin master --force > ``` @@ -164,13 +164,13 @@ Name the branch something like `fix/xxx` or `feature/xxx` where `xxx` is a short To create a branch on your local machine (and switch to this branch): -```shell +```sh $ git checkout -b [name_of_your_new_branch] ``` and to push to GitHub: -```shell +```sh $ git push origin [name_of_your_new_branch] ``` @@ -180,12 +180,12 @@ $ git push origin [name_of_your_new_branch] ### Setup for cmd2 development For doing cmd2 development, you actually do NOT want to have cmd2 installed as a Python package. So if you have previously installed cmd2, make sure to uninstall it: -```bash -pip uninstall cmd2 +```sh +$ pip uninstall cmd2 ``` Assuming you cloned the repository to `~/src/cmd2`: -```bash +```sh $ cd ~/src/cmd2 $ pip install -e . ``` @@ -195,18 +195,39 @@ imports `cmd2`, there is no need to re-install the module after every change. Th command will also install all of the runtime dependencies for `cmd2`. Next you should install all the modules used for development of `cmd2`: -```bash +```sh $ cd ~/src/cmd2 $ pip install -e .[dev] ``` -This will install `pytest` and `tox` for running unit tests, `pylint` for -static code analysis, and `sphinx` for building the documentation. +This project uses many python modules for various development tasks, including +testing, rendering documentation, and building and distributing releases. These +modules can be configured many different ways, which can make it difficult to +learn the specific incantations required for each project you are familiar with. + +This project uses `invoke <http://www.pyinvoke.org>`_ to provide a clean, high +level interface for these development tasks. To see the full list of functions +available:: +```sh +$ invoke -l +``` + +You can run multiple tasks in a single invocation, for example:: +```sh +$ invoke docs sdist wheel +``` + +That one command will remove all superflous cache, testing, and build +files, render the documentation, and build a source distribution and a +wheel distribution. + +If you want to see the details about what `invoke` is doing under the hood, +have a look at `tasks.py`. Now you can check if everything is installed and working: -```bash -cd ~src/cmd2 -python examples/example.py +```sh +$ cd ~src/cmd2 +$ python examples/example.py ``` If the example app loads, you should see a prompt that says "(Cmd)". You can @@ -220,20 +241,32 @@ This bit is up to you! #### How to find the code in the cmd2 codebase to fix/edit? -The cmd2 project directory structure is pretty simple and straightforward. All actual code for cmd2 -is located underneath the `cmd2` directory. The code to generate the documentation is in the `docs` directory. Unit tests are in the `tests` directory. The `examples` directory contains examples of how -to use cmd2. There are various other files in the root directory, but these are primarily related to -continuous integration and to release deployment. +The cmd2 project directory structure is pretty simple and straightforward. All +actual code for cmd2 is located underneath the `cmd2` directory. The code to +generate the documentation is in the `docs` directory. Unit tests are in the +`tests` directory. The `examples` directory contains examples of how to use +cmd2. There are various other files in the root directory, but these are +primarily related to continuous integration and to release deployment. #### Changes to the documentation files -If you made changes to any file in the `/docs` directory, you need to build the Sphinx documentation -and make sure your changes look good: -```shell -cd docs -make clean html + +If you made changes to any file in the `/docs` directory, you need to build the +Sphinx documentation and make sure your changes look good: +```sh +$ invoke docs ``` In order to see the changes, use your web browser of choice to open `<cmd2>/docs/_build/html/index.html`. +If you would rather use a webserver to view the documentation, including +automatic page refreshes as you edit the files, use: + +```sh +$ invoke livehtml +``` + +You will be shown the IP address and port number where the documents are now +served (usually [http://localhost:8000](http://localhost:8000). + ### Static Code Analysis You should have some sort of [PEP8](https://www.python.org/dev/peps/pep-0008/)-based linting running in your editor or IDE or at the command-line before you commit code. [pylint](https://www.pylint.org) is a good Python linter which can be run at the command-line but also can integrate with many IDEs and editors. @@ -242,21 +275,14 @@ You should have some sort of [PEP8](https://www.python.org/dev/peps/pep-0008/)-b ### Run The Test Suite When you're ready to share your code, run the test suite: -```shell -cd <cmd2> -py.test +```sh +$ cd <cmd2> +$ invoke pytest ``` and ensure all tests pass. -#### Measuring code coverage - -Code coverage can be measured as follows: - -```shell -py.test --cov=cmd2 --cov-report=term-missing --cov-report=html -``` - -Then use your web browser of choice to look at the results which are in `<cmd2>/htmlcov/index.html`. +Running the test suite also calculates test code coverage. A summary of coverage +is shown on the screen. A full report is available in `<cmd2>/htmlcov/index.html`. ### Squash Your Commits When you make a pull request, it is preferable for all of your changes to be in one commit. @@ -301,7 +327,7 @@ Instance of cmd2](#maintaining-your-fork). 1. Perform the maintenance step of rebasing `master`. 2. Ensure you are on the `master` branch using `git status`: -```bash +```sh $ git status On branch master Your branch is up-to-date with 'origin/master'. @@ -458,6 +484,19 @@ excellent support for debugging console applications. [PyCharm](https://www.jetbrains.com/pycharm/) is also quite good and has very nice [Code Inspection](https://www.jetbrains.com/help/pycharm/code-inspection.html) capabilities. -### Acknowledgement +## Publishing a new release + +Since 0.9.2, the process of publishing a new release of `cmd2` to [PyPi](https://pypi.org/) has been +mostly automated. The manual steps are all git operations. Here's the checklist: + +1. Make sure you are on the proper branch +2. Make sure the version strings in `cmd2.py`, `conf.py`, `setup.py`, and `test_cmd2.py` are correct. +3. Make sure all the unit tests pass. +4. Make sure `CHANGELOG.md` describes the version and has the correct release date. +5. Create and push a git tag that matches the version strings above. +6. (Optional) Run `invoke pypi_test` to clean, build, and upload a new release to [Test PyPi](https://test.pypi.org) +7. Run `invoke pypi` to clean, build, and upload a new release to [PyPi](https://pypi.org/) + +## Acknowledgement Thanks to the good folks at [freeCodeCamp](https://github.com/freeCodeCamp/freeCodeCamp) for creating an excellent `CONTRIBUTING` file which we have borrowed heavily from. diff --git a/fabfile.py b/fabfile.py deleted file mode 100644 index 729965e2..00000000 --- a/fabfile.py +++ /dev/null @@ -1,111 +0,0 @@ -# coding=utf-8 -from fabric.api import env, task, local -import os -import errno - -env.projname = local("python setup.py --name", capture=True) -env.version = local("python setup.py --version", capture=True) - - -def mkdirs(path): - try: - os.makedirs(path) - except OSError as exc: - if exc.errno == errno.EEXIST and os.path.isdir(path): - pass - else: - raise - - -@task -def clean(): - local("python setup.py clean") - local("find . -name '*.pyc' -delete") - local("rm -rf cmd2.egg-info") - local("rm -rf dist") - local("rm -rf htmlcov") - local("rm -rf docs/_build") - local("rm -rf .pytest_cache") - - -@task -def build(): - local("python setup.py sdist bdist_wheel") - - -@task -def rebuild(): - clean() - build() - - -@task -def prepare_cover_dir(): - # If there isn't a cover symlink, create and link the directory - if os.path.isdir('cover'): - return - - mkdirs('docs/_build/html/cover') - local("ln -s docs/_build/html/cover") - local("rm -rf cover/*") - - -@task -def coverage(): - prepare_cover_dir() - local("py.test -n4 --cov=%s --cov-report=term-missing " - "--cov-report=html" % env.projname) - - -@task -def pylint(): - local("pylint %s tests" % env.projname) - - -@task -def doc(): - local("sphinx-build -b html docs docs/_build/html") - - -@task -def docwithcoverage(): - coverage() - doc() - - -@task -def tox(): - local('tox') - - -@task -def release_check(): - tags = local("git tag", capture=True) - tags = set(tags.splitlines()) - if 'a' in env.version: - print("WARNING: alpha release %s" % env.version) - - # hacky CHANGELOG.md check - with open("CHANGELOG.md") as f: - raw_changes = f.read() - assert "%s\n---" % env.version in raw_changes, \ - "The current version %s is not in CHANGELOG.md" % env.version - if env.version in tags: - raise Exception("Already released v. %r" % env.version) - - -@task -def release(): - release_check() - clean() - build() - print("Releasing", env.projname, "version", env.version) - local("git tag %s" % env.version) - local("python setup.py sdist bdist_wheel upload") - local("git push --tags") - - -@task -def test_pip_install(): - local("d=$(mktemp -d) && cd $d && virtualenv . && . " - "bin/activate && pip install -v %s" % env.projname) @@ -73,6 +73,7 @@ EXTRAS_REQUIRE = { # install with 'pip install -e .[dev]' 'dev': [ 'pytest', 'pytest-cov', 'tox', 'pylint', 'sphinx', 'sphinx-rtd-theme', + 'sphinx-autobuild','invoke', 'twine', ] } diff --git a/tasks.py b/tasks.py new file mode 100644 index 00000000..000af0a5 --- /dev/null +++ b/tasks.py @@ -0,0 +1,178 @@ +# +# coding=utf-8 +"""Development related tasks to be run with 'invoke'""" + +import os +import shutil + +import invoke + +# shared function +def rmrf(items, verbose=True): + "Silently remove a list of directories or files" + if isinstance(items, str): + items = [items] + + for item in items: + if verbose: + print("Removing {}".format(item)) + shutil.rmtree(item, ignore_errors=True) + # rmtree doesn't remove bare files + try: + os.remove(item) + except FileNotFoundError: + pass + + +# create namespaces +namespace = invoke.Collection() +namespace_clean = invoke.Collection('clean') +namespace.add_collection(namespace_clean, 'clean') + +##### +# +# pytest, tox, pylint, and codecov +# +##### +@invoke.task +def pytest(context): + "Run tests and code coverage using pytest" + context.run("pytest --cov=cmd2 --cov-report=term --cov-report=html") +namespace.add_task(pytest) + +@invoke.task +def pytest_clean(context): + "Remove pytest cache and code coverage files and directories" + #pylint: disable=unused-argument + dirs = ['.pytest-cache', '.cache', 'htmlcov', '.coverage'] + rmrf(dirs) +namespace_clean.add_task(pytest_clean, 'pytest') + +@invoke.task +def tox(context): + "Run unit and integration tests on multiple python versions using tox" + context.run("tox") +namespace.add_task(tox) + +@invoke.task +def tox_clean(context): + "Remove tox virtualenvs and logs" + #pylint: disable=unused-argument + rmrf('.tox') +namespace_clean.add_task(tox_clean, 'tox') + + +##### +# +# documentation +# +##### +DOCS_SRCDIR = 'docs' +DOCS_BUILDDIR = os.path.join('docs', '_build') + +@invoke.task() +def docs(context, builder='html'): + "Build documentation using sphinx" + cmdline = 'python -msphinx -M {} {} {}'.format(builder, DOCS_SRCDIR, DOCS_BUILDDIR) + context.run(cmdline) +namespace.add_task(docs) + +@invoke.task +def docs_clean(context): + "Remove rendered documentation" + #pylint: disable=unused-argument + rmrf(DOCS_BUILDDIR) +namespace_clean.add_task(docs_clean, name='docs') + +@invoke.task +def livehtml(context): + "Launch webserver on http://localhost:8000 with rendered documentation" + builder = 'html' + outputdir = os.path.join(DOCS_BUILDDIR, builder) + cmdline = 'sphinx-autobuild -b {} {} {}'.format(builder, DOCS_SRCDIR, outputdir) + context.run(cmdline, pty=True) +namespace.add_task(livehtml) + + +##### +# +# build and distribute +# +##### +BUILDDIR = 'build' +DISTDIR = 'dist' + +@invoke.task +def build_clean(context): + "Remove the build directory" + #pylint: disable=unused-argument + rmrf(BUILDDIR) +namespace_clean.add_task(build_clean, 'build') + +@invoke.task +def dist_clean(context): + "Remove the dist directory" + #pylint: disable=unused-argument + rmrf(DISTDIR) +namespace_clean.add_task(dist_clean, 'dist') + +@invoke.task +def eggs_clean(context): + "Remove egg directories" + #pylint: disable=unused-argument + dirs = set() + dirs.add('.eggs') + for name in os.listdir(os.curdir): + if name.endswith('.egg-info'): + dirs.add(name) + if name.endswith('.egg'): + dirs.add(name) + rmrf(dirs) +namespace_clean.add_task(eggs_clean, 'eggs') + +@invoke.task +def pycache_clean(context): + "Remove __pycache__ directories" + #pylint: disable=unused-argument + dirs = set() + for root, dirnames, _ in os.walk(os.curdir): + if '__pycache__' in dirnames: + dirs.add(os.path.join(root, '__pycache__')) + print("Removing __pycache__ directories") + rmrf(dirs, verbose=False) +namespace_clean.add_task(pycache_clean, 'pycache') + +# +# make a dummy clean task which runs all the tasks in the clean namespace +clean_tasks = list(namespace_clean.tasks.values()) +@invoke.task(pre=list(namespace_clean.tasks.values()), default=True) +def clean_all(context): + "Run all clean tasks" + #pylint: disable=unused-argument + pass +namespace_clean.add_task(clean_all, 'all') + +@invoke.task(pre=[clean_all]) +def sdist(context): + "Create a source distribution" + context.run('python setup.py sdist') +namespace.add_task(sdist) + +@invoke.task(pre=[clean_all]) +def wheel(context): + "Build a wheel distribution" + context.run('python setup.py bdist_wheel') +namespace.add_task(wheel) + +@invoke.task(pre=[sdist, wheel]) +def pypi(context): + "Build and upload a distribution to pypi" + context.run('twine upload dist/*') +namespace.add_task(pypi) + +@invoke.task(pre=[sdist, wheel]) +def pypi_test(context): + "Build and upload a distribution to https://test.pypi.org" + context.run('twine upload --repository-url https://test.pypi.org/legacy/ dist/*') +namespace.add_task(pypi_test) + |