diff options
Diffstat (limited to 'docs/features/transcripts.rst')
| -rw-r--r-- | docs/features/transcripts.rst | 201 |
1 files changed, 201 insertions, 0 deletions
diff --git a/docs/features/transcripts.rst b/docs/features/transcripts.rst new file mode 100644 index 00000000..18daeb78 --- /dev/null +++ b/docs/features/transcripts.rst @@ -0,0 +1,201 @@ +Transcripts +=========== + +A transcript is both the input and output of a successful session of a +``cmd2``-based app which is saved to a text file. With no extra work on your +part, your app can play back these transcripts as a unit test. Transcripts can +contain regular expressions, which provide the flexibility to match responses +from commands that produce dynamic or variable output. + +.. highlight:: none + + +Creating From History +--------------------- + +A transcript can automatically generated based upon commands previously +executed in the *history* using ``history -t``:: + + (Cmd) help + ... + (Cmd) help history + ... + (Cmd) history 1:2 -t transcript.txt + 2 commands and outputs saved to transcript file 'transcript.txt' + +This is by far the easiest way to generate a transcript. + +.. warning:: + + Make sure you use the **poutput()** method in your ``cmd2`` application for + generating command output. This method of the ``cmd2.Cmd`` class ensure + that output is properly redirected when redirecting to a file, piping to a + shell command, and when generating a transcript. + + +Creating From A Script File +--------------------------- + +A transcript can also be automatically generated from a script file using +``run_script -t``:: + + (Cmd) run_script scripts/script.txt -t transcript.txt + 2 commands and their outputs saved to transcript file 'transcript.txt' + (Cmd) + +This is a particularly attractive option for automatically regenerating +transcripts for regression testing as your ``cmd2`` application changes. + + +Creating Manually +----------------- + +Here's a transcript created from ``python examples/example.py``:: + + (Cmd) say -r 3 Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + (Cmd) mumble maybe we could go to lunch + like maybe we ... could go to hmmm lunch + (Cmd) mumble maybe we could go to lunch + well maybe we could like go to er lunch right? + +This transcript has three commands: they are on the lines that begin with the +prompt. The first command looks like this:: + + (Cmd) say -r 3 Goodnight, Gracie + +Following each command is the output generated by that command. + +The transcript ignores all lines in the file until it reaches the first line +that begins with the prompt. You can take advantage of this by using the first +lines of the transcript as comments:: + + # Lines at the beginning of the transcript that do not + ; start with the prompt i.e. '(Cmd) ' are ignored. + /* You can use them for comments. */ + + All six of these lines before the first prompt are treated as comments. + + (Cmd) say -r 3 Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + Goodnight, Gracie + (Cmd) mumble maybe we could go to lunch + like maybe we ... could go to hmmm lunch + (Cmd) mumble maybe we could go to lunch + maybe we could like go to er lunch right? + +In this example I've used several different commenting styles, and even bare +text. It doesn't matter what you put on those beginning lines. Everything +before:: + + (Cmd) say -r 3 Goodnight, Gracie + +will be ignored. + + +Regular Expressions +------------------- + +If we used the above transcript as-is, it would likely fail. As you can see, +the ``mumble`` command doesn't always return the same thing: it inserts random +words into the input. + +Regular expressions can be included in the response portion of a transcript, +and are surrounded by slashes:: + + (Cmd) mumble maybe we could go to lunch + /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/ + (Cmd) mumble maybe we could go to lunch + /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/ + +Without creating a tutorial on regular expressions, this one matches anything +that has the words ``maybe``, ``could``, and ``lunch`` in that order. It +doesn't ensure that ``we`` or ``go`` or ``to`` appear in the output, but it +does work if mumble happens to add words to the beginning or the end of the +output. + +Since the output could be multiple lines long, ``cmd2`` uses multiline regular +expression matching, and also uses the ``DOTALL`` flag. These two flags subtly +change the behavior of commonly used special characters like ``.``, ``^`` and +``$``, so you may want to double check the `Python regular expression +documentation <https://docs.python.org/3/library/re.html>`_. + +If your output has slashes in it, you will need to escape those slashes so the +stuff between them is not interpred as a regular expression. In this +transcript:: + + (Cmd) say cd /usr/local/lib/python3.6/site-packages + /usr/local/lib/python3.6/site-packages + +the output contains slashes. The text between the first slash and the second +slash, will be interpreted as a regular expression, and those two slashes will +not be included in the comparison. When replayed, this transcript would +therefore fail. To fix it, we could either write a regular expression to match +the path instead of specifying it verbatim, or we can escape the slashes:: + + (Cmd) say cd /usr/local/lib/python3.6/site-packages + \/usr\/local\/lib\/python3.6\/site-packages + +.. warning:: + + Be aware of trailing spaces and newlines. Your commands might output + trailing spaces which are impossible to see. Instead of leaving them + invisible, you can add a regular expression to match them, so that you can + see where they are when you look at the transcript:: + + (Cmd) set prompt + prompt: (Cmd)/ / + + Some terminal emulators strip trailing space when you copy text from them. + This could make the actual data generated by your app different than the + text you pasted into the transcript, and it might not be readily obvious why + the transcript is not passing. Consider using :ref:`output_redirection` to + the clipboard or to a file to ensure you accurately capture the output of + your command. + + If you aren't using regular expressions, make sure the newlines at the end + of your transcript exactly match the output of your commands. A common cause + of a failing transcript is an extra or missing newline. + + If you are using regular expressions, be aware that depending on how you + write your regex, the newlines after the regex may or may not matter. ``\Z`` + matches *after* the newline at the end of the string, whereas ``$`` matches + the end of the string *or* just before a newline. + + +Running A Transcript +-------------------- + +Once you have created a transcript, it's easy to have your application play it +back and check the output. From within the ``examples/`` directory:: + + $ python example.py --test transcript_regex.txt + . + ---------------------------------------------------------------------- + Ran 1 test in 0.013s + + OK + +The output will look familiar if you use ``unittest``, because that's exactly +what happens. Each command in the transcript is run, and we ``assert`` the +output matches the expected result from the transcript. + +.. note:: + + If you have set ``allow_cli_args`` to False in order to disable parsing of + command line arguments at invocation, then the use of ``-t`` or ``--test`` to + run transcript testing is automatically disabled. In this case, you can + alternatively provide a value for the optional ``transcript_files`` when + constructing the instance of your ``cmd2.Cmd`` derived class in order to + cause a transcript test to run:: + + from cmd2 import Cmd + class App(Cmd): + # customized attributes and methods here + + if __name__ == '__main__': + app = App(transcript_files=['exampleSession.txt']) + app.cmdloop() |