diff options
Diffstat (limited to 'docs/transcript.rst')
| -rw-r--r-- | docs/transcript.rst | 132 |
1 files changed, 132 insertions, 0 deletions
diff --git a/docs/transcript.rst b/docs/transcript.rst new file mode 100644 index 00000000..9aa050b1 --- /dev/null +++ b/docs/transcript.rst @@ -0,0 +1,132 @@ +======================== +Transcript based testing +======================== + +A transcript is both the input and output of a successful session of a +``cmd2``-based app which is saved to a text file. The transcript can be played +back into the app as a unit test. You can embed regular expressions into the +transcript to accomodate commands that produce dynamic or variable output. + + +Creating a transcript +===================== + +Here's a transcript created from ``python examples/example.py``: + +.. code-block:: none + + (Cmd) say -r 3 Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + (Cmd) mumble maybe we could go to lunch + like maybe we ... could go to hmmm lunch + (Cmd) mumble maybe we could go to lunch + well maybe we could like go to er lunch right? + +This transcript has three commands: you can see them on the lines that begin +with the prompt, which in this case is ``(Cmd) ``. Following each command is +the output generated by that command. + +Any lines in the transcript before the first line that begins with the prompt +are ignored. You can take advantage of this by using the first lines of the +transcript as comments. + +.. code-block:: none + + # Lines at the beginning of the transcript that do not + ; start with the prompt i.e. '(Cmd) ' are ignored. + /* You can use them for comments. */ + + All six of these lines before the first prompt are treated as comments. + + (Cmd) say -r 3 Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + (Cmd) mumble maybe we could go to lunch + like maybe we ... could go to hmmm lunch + (Cmd) mumble maybe we could go to lunch + maybe we could like go to er lunch right? + +In this example I've used several different commenting styles, and even bare +text. It doesn't matter what you put on those beginning lines. Everything before +the first line that starts with ``(Cmd) `` will be ignored. + +If we used this transcript as-is, it would likely fail. As you can see, the +``mumble`` command doesn't always return the same thing. The ``mumble`` command +inserts random words into the input. Transcripts can include regular +expressions as a way to check for output that can change. + +Regular expressions can be included in the response portion of a transcript, +and are surrounded by slashes. + +.. code-block:: none + + (Cmd) mumble maybe we could go to lunch + /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/ + (Cmd) mumble maybe we could go to lunch + /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/ + +Without creating a tutorial on regular expressions, this one matches anything +that has the words `maybe`, `could`, and `lunch` in that order. It doesn't +ensure that `we` or `go` or `to` appear in the output, but it does work if +mumble happens to add words to the beginning or the end of the output. + +Since the output could be multiple lines long, ``cmd2`` uses multiline regular +expression matching, and also uses the ``DOTALL`` flag, which subtly changes the behavior of commonly +used special characters like `.`, `^` and `$`, so you may want to double check the +`Python regular expression documentation +<https://docs.python.org/3/library/re.html>`_. + +If your output has slashes in it, you will need to escape those slashes so the +stuff between them is not interpred as a regular expression. In this transcript:: + + (Cmd) say cd /usr/local/lib/python3.6/site-packages + /usr/local/lib/python3.6/site-packages + +the output contains slashes. The text between the first slash and the second +slash, (``usr``) will be interpreted as a regular expression, and those two +slashes will not be included in the comparison. When replayed, this transcript +would therefore fail. To fix it, we could either write a regular expression to +match the path instead of specifying it verbatim, or we can escape the slashes:: + + (Cmd) say cd /usr/local/lib/python3.6/site-packages + \/usr\/local\/lib\/python3.6\/site-packages + + +Running a transcript +==================== + +Once you have created a transcript, it's easy to have your application play it +back and check the output. From within the ``examples/`` directory: + +.. code-block:: none + + $ python example.py --test transcript_regex.txt + . + ---------------------------------------------------------------------- + Ran 1 test in 0.013s + + OK + +The output will look familiar if you use ``unittest``, because that's exactly +what happens. Each command in the transcript is run, and the output is +``asserted`` to match expected result from the transcript. + +.. note:: + + If you have set ``allow_cli_args`` to False in order to disable parsing of + command line arguments at invocation, then the use of ``-t`` or ``--test`` + to run transcript testing is automatically disabled. In this case, you can + alternatively provide a value for the optional ``transcript_files`` when + constructing the instance of your ``cmd2.Cmd`` derived class in order to + cause a transcript test to run:: + + from cmd2 import Cmd + class App(Cmd): + # customized attributes and methods here + + if __name__ == '__main__': + app = App(transcript_files=['exampleSession.txt']) + app.cmdloop() |