diff options
Diffstat (limited to 'docs/dev/testing.txt')
| -rw-r--r-- | docs/dev/testing.txt | 187 |
1 files changed, 187 insertions, 0 deletions
diff --git a/docs/dev/testing.txt b/docs/dev/testing.txt new file mode 100644 index 000000000..9feac9c6a --- /dev/null +++ b/docs/dev/testing.txt @@ -0,0 +1,187 @@ +=================== + Docutils_ Testing +=================== + +:Author: Felix Wiemann +:Revision: $Revision$ +:Date: $Date$ +:Copyright: This document has been placed in the public domain. + +.. _Docutils: http://docutils.sourceforge.net/ + +.. contents:: + +This document describes how the tests are organized and how to add new +tests or modify existing tests. + + +Unit Tests +========== + +Unit tests test single functions or modules (i.e. whitebox testing). + +If you are implementing a new feature, be sure to write a test case +covering its functionality. It happens very frequently that your +implementation (or even only a part of it) doesn't work with an older +(or even newer) Python version, and the only reliable way to detect +those cases is using tests. + +Often, it's easier to write the test first and then implement the +functionality required to make the test pass. + + +Writing New Tests +----------------- + +When writing new tests, it very often helps to see how a similar test +is implemented. For example, the files in the +``test_parsers/test_rst/`` directory all look very similar. So when +adding a test, you don't have to reinvent the wheel. + +If there is no similar test, you can write a new test from scratch +using Python's ``unittest`` module. For an example, please have a +look at the following imaginary ``test_square.py``:: + + #! /usr/bin/env python + + # Author: your name + # Contact: your email address + # Revision: $Revision$ + # Date: $Date$ + # Copyright: This module has been placed in the public domain. + + """ + Test module for docutils.square. + """ + + import unittest + import docutils.square + + + class SquareTest(unittest.TestCase): + + def test_square(self): + self.assertEqual(docutils.square.square(0), 0) + self.assertEqual(docutils.square.square(5), 25) + self.assertEqual(docutils.square.square(7), 49) + + def test_square_root(self): + self.assertEqual(docutils.square.sqrt(49), 7) + self.assertEqual(docutils.square.sqrt(0), 0) + self.assertRaises(docutils.square.SquareRootError, + docutils.square.sqrt, 20) + + + if __name__ == '__main__': + unittest.main() + +For more details on how to write tests, please refer to the +documentation of the ``unittest`` module. + + +.. _functional:: + +Functional Tests +================ + +The directory ``test/functional/`` contains data for functional tests. + +Performing functional testing means testing the Docutils system as a +whole (i.e. blackbox testing). + + +Directory Structure +------------------- + ++ ``functional/`` The main data directory. + + + ``input/`` The input files. + + - ``some_test.txt``, for example. + + + ``output/`` The actual output. + + - ``some_test.html``, for example. + + + ``expected/`` The expected output. + + - ``some_test.html``, for example. + + + ``tests/`` The config files for processing the input files. + + - ``some_test.py``, for example. + + - ``_default.py``, the `default configuration file`_. + + +The Testing Process +------------------- + +When running ``test_functional.py``, all config files in +``functional/tests/`` are processed. (Config files whose names begin +with an underscore are ignored.) The current working directory is +always Docutils' main test directory (``test/``). + +For example, ``functional/tests/some_test.py`` could read like this:: + + # Source and destination file names. + test_source = "some_test.txt" + test_destination = "some_test.html" + + # Keyword parameters passed to publish_file. + reader_name = "standalone" + parser_name = "rst" + writer_name = "html" + settings_overrides['output-encoding'] = 'utf-8' + # Relative to main ``test/`` directory. + settings_overrides['stylesheet_path'] = '../tools/stylesheets/default.css' + +The two variables ``test_source`` and ``test_destination`` contain the +input file name (relative to ``functional/input/``) and the output +file name (relative to ``functional/output/`` and +``functional/expected/``). Note that the file names can be chosen +arbitrarily. However, the file names in ``functional/output/`` *must* +match the file names in ``functional/expected/``. + +All other variables are passed as keyword arguments to +``docutils.core.publish_file``, so you can set reader, parser, +writer and anything else you want to configure. + +Note that ``settings_overrides`` is already initialized as a +dictionary *before* the execution of the config file. + + +Creating New Tests +------------------ + +In order to create a new test, put the input test file into +``functional/input/``. Then create a config file in +``functional/tests/`` which sets at least input and output file names, +reader, parser and writer. + +Now run ``test_functional.py``. The test will fail, of course, +because you do not have an expected output yet. However, an output +file will have been generated in ``functional/output/``. Check this +output file for validity and correctness. Then copy the file to +``functional/expected/``. + +If you rerun ``test_functional.py`` now, it should pass. + +If you run ``test_functional.py`` later and the actual output doesn't +match the expected output anymore, the test will fail. + +If this is the case and you made an intentional change, check the +actual output for validity and correctness, copy it to +``functional/expected/`` (overwriting the old expected output), and +commit the change. + + +.. _default configuration file: + +The Default Configuration File +------------------------------ + +The file ``functional/tests/_default.py`` contains default settings. +It is executed just before the actual configuration files, which has +the same effect as if the contents of ``_default.py`` were prepended +to every configuration file. |
