Write documentation for revised transcription feature

4 files changed, 176 insertions, 34 deletions

diff --git a/docs/freefeatures.rst b/docs/freefeatures.rst
index 7b6762ad..fbaca7f6 100644
--- a/docs/freefeatures.rst
+++ b/docs/freefeatures.rst
@@ -301,34 +301,20 @@ is equivalent to ``shell ls``.)
 Transcript-based testing
 ========================
 
-If the entire transcript (input and output) of a successful session of
-a ``cmd2``-based app is copied from the screen and pasted into a text
-file, ``transcript.txt``, then a transcript test can be run against it::
+A transcript is both the input and output of a successful session of a
+``cmd2``-based app which is saved to a text file. The transcript can be played
+back into the app as a unit test.
 
-  python app.py --test transcript.txt
+.. code-block:: none
 
-Any non-whitespace deviations between the output prescribed in ``transcript.txt`` and
-the actual output from a fresh run of the application will be reported
-as a unit test failure.  (Whitespace is ignored during the comparison.)
+   $ python example.py --test transcript_regex.txt
+   .
+   ----------------------------------------------------------------------
+   Ran 1 test in 0.013s
 
-Regular expressions can be embedded in the transcript inside paired ``/``
-slashes.  These regular expressions should not include any whitespace
-expressions.
+   OK
 
-.. note::
-
-   If you have set ``allow_cli_args`` to False in order to disable parsing of command line arguments at invocation,
-   then the use of ``-t`` or ``--test`` to run transcript testing is automatically disabled.  In this case, you can
-   alternatively provide a value for the optional ``transcript_files`` when constructing the instance of your
-   ``cmd2.Cmd`` derived class in order to cause a transcript test to run::
-
-       from cmd2 import Cmd
-       class App(Cmd):
-         # customized attributes and methods here
-
-       if __name__ == '__main__':
-           app = App(transcript_files=['exampleSession.txt'])
-           app.cmdloop()
+See :doc:`<transcription>` for more details.
 
 
 Tab-Completion
diff --git a/docs/index.rst b/docs/index.rst
index e89be557..12e1f832 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -66,6 +66,7 @@ Contents:
    freefeatures
    settingchanges
    unfreefeatures
+   transcription
    integrating
    hooks
    alternatives
diff --git a/docs/transcript.rst b/docs/transcript.rst
new file mode 100644
index 00000000..9aa050b1
--- /dev/null
+++ b/docs/transcript.rst
@@ -0,0 +1,132 @@
+========================
+Transcript based testing
+========================
+
+A transcript is both the input and output of a successful session of a
+``cmd2``-based app which is saved to a text file. The transcript can be played
+back into the app as a unit test. You can embed regular expressions into the
+transcript to accomodate commands that produce dynamic or variable output.
+
+
+Creating a transcript
+=====================
+
+Here's a transcript created from ``python examples/example.py``:
+
+.. code-block:: none
+
+   (Cmd) say -r 3 Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   (Cmd) mumble maybe we could go to lunch
+   like maybe we ... could go to hmmm lunch
+   (Cmd) mumble maybe we could go to lunch
+   well maybe we could like go to er lunch right?
+
+This transcript has three commands: you can see them on the lines that begin
+with the prompt, which in this case is ``(Cmd) ``. Following each command is
+the output generated by that command.
+
+Any lines in the transcript before the first line that begins with the prompt
+are ignored. You can take advantage of this by using the first lines of the
+transcript as comments.
+
+.. code-block:: none
+
+   # Lines at the beginning of the transcript that do not
+   ; start with the prompt i.e. '(Cmd) ' are ignored.
+   /* You can use them for comments. */
+   
+   All six of these lines before the first prompt are treated as comments.
+   
+   (Cmd) say -r 3 Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   (Cmd) mumble maybe we could go to lunch
+   like maybe we ... could go to hmmm lunch
+   (Cmd) mumble maybe we could go to lunch
+   maybe we could like go to er lunch right?
+
+In this example I've used several different commenting styles, and even bare
+text. It doesn't matter what you put on those beginning lines. Everything before
+the first line that starts with ``(Cmd) `` will be ignored.
+
+If we used this transcript as-is, it would likely fail. As you can see, the
+``mumble`` command doesn't always return the same thing. The ``mumble`` command
+inserts random words into the input. Transcripts can include regular
+expressions as a way to check for output that can change.
+
+Regular expressions can be included in the response portion of a transcript,
+and are surrounded by slashes.
+
+.. code-block:: none
+
+   (Cmd) mumble maybe we could go to lunch
+   /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/
+   (Cmd) mumble maybe we could go to lunch
+   /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/
+
+Without creating a tutorial on regular expressions, this one matches anything
+that has the words `maybe`, `could`, and `lunch` in that order. It doesn't
+ensure that `we` or `go` or `to` appear in the output, but it does work if
+mumble happens to add words to the beginning or the end of the output.
+
+Since the output could be multiple lines long, ``cmd2`` uses multiline regular
+expression matching, and also uses the ``DOTALL`` flag, which subtly changes the behavior of commonly
+used special characters like `.`, `^` and `$`, so you may want to double check the
+`Python regular expression documentation
+<https://docs.python.org/3/library/re.html>`_.
+
+If your output has slashes in it, you will need to escape those slashes so the
+stuff between them is not interpred as a regular expression. In this transcript::
+
+   (Cmd) say cd /usr/local/lib/python3.6/site-packages
+   /usr/local/lib/python3.6/site-packages
+
+the output contains slashes. The text between the first slash and the second
+slash, (``usr``) will be interpreted as a regular expression, and those two
+slashes will not be included in the comparison. When replayed, this transcript
+would therefore fail. To fix it, we could either write a regular expression to
+match the path instead of specifying it verbatim, or we can escape the slashes::
+
+   (Cmd) say cd /usr/local/lib/python3.6/site-packages
+   \/usr\/local\/lib\/python3.6\/site-packages
+
+
+Running a transcript
+====================
+
+Once you have created a transcript, it's easy to have your application play it
+back and check the output. From within the ``examples/`` directory:
+
+.. code-block:: none
+
+   $ python example.py --test transcript_regex.txt
+   .
+   ----------------------------------------------------------------------
+   Ran 1 test in 0.013s
+
+   OK
+
+The output will look familiar if you use ``unittest``, because that's exactly
+what happens. Each command in the transcript is run, and the output is
+``asserted`` to match expected result from the transcript.
+
+.. note::
+
+   If you have set ``allow_cli_args`` to False in order to disable parsing of
+   command line arguments at invocation, then the use of ``-t`` or ``--test``
+   to run transcript testing is automatically disabled. In this case, you can
+   alternatively provide a value for the optional ``transcript_files`` when
+   constructing the instance of your ``cmd2.Cmd`` derived class in order to
+   cause a transcript test to run::
+
+       from cmd2 import Cmd
+       class App(Cmd):
+         # customized attributes and methods here
+
+       if __name__ == '__main__':
+           app = App(transcript_files=['exampleSession.txt'])
+           app.cmdloop()
diff --git a/examples/example.py b/examples/example.py
index 482788cc..03508024 100755
--- a/examples/example.py
+++ b/examples/example.py
@@ -1,14 +1,18 @@
 #!/usr/bin/env python
 # coding=utf-8
-"""A sample application for cmd2.
+"""
+A sample application for cmd2.
 
-Thanks to cmd2's built-in transcript testing capability, it also serves as a test suite for example.py when used with
- the exampleSession.txt transcript.
+Thanks to cmd2's built-in transcript testing capability, it also serves as a
+test suite for example.py when used with the exampleSession.txt transcript.
 
-Running `python example.py -t exampleSession.txt` will run all the commands in the transcript against example.py,
-verifying that the output produced matches the transcript.
+Running `python example.py -t exampleSession.txt` will run all the commands in
+the transcript against example.py, verifying that the output produced matches
+the transcript.
 """
 
+import random
+
 from cmd2 import Cmd, make_option, options, set_use_arg_list
 
 
@@ -17,13 +21,16 @@ class CmdLineApp(Cmd):
 
     # Setting this true makes it run a shell command if a cmd2/cmd command doesn't exist
     # default_to_shell = True
+    MUMBLES = ['like', '...', 'um', 'er', 'hmmm', 'ahh']
+    MUMBLE_FIRST = ['so', 'like', 'well']
+    MUMBLE_LAST = ['right?']
 
     def __init__(self):
         self.abbrev = True
         self.multilineCommands = ['orate']
         self.maxrepeats = 3
 
-        # Add stuff to settable and shortcutgs before calling base class initializer
+        # Add stuff to settable and shortcuts before calling base class initializer
         self.settable['maxrepeats'] = 'max repetitions for speak command'
         self.shortcuts.update({'&': 'speak'})
 
@@ -46,14 +53,30 @@ class CmdLineApp(Cmd):
             arg = arg.upper()
         repetitions = opts.repeat or 1
         for i in range(min(repetitions, self.maxrepeats)):
-            self.stdout.write(arg)
-            self.stdout.write('\n')
-            # self.stdout.write is better than "print", because Cmd can be
-            # initialized with a non-standard output destination
+            self.poutput(arg)
+            # recommend using the poutput function instead of
+            # self.stdout.write or "print", because Cmd allows the user
+            # to redirect output
 
     do_say = do_speak  # now "say" is a synonym for "speak"
     do_orate = do_speak  # another synonym, but this one takes multi-line input
 
+    @options([ make_option('-r', '--repeat', type="int", help="output [n] times") ])
+    def do_mumble(self, arg, opts=None):
+        """Mumbles what you tell me to."""
+        repetitions = opts.repeat or 1
+        arg = arg.split()
+        for i in range(min(repetitions, self.maxrepeats)):
+            output = []
+            if (random.random() < .33):
+                output.append(random.choice(self.MUMBLE_FIRST))
+            for word in arg:
+                if (random.random() < .40):
+                    output.append(random.choice(self.MUMBLES))
+                output.append(word)
+            if (random.random() < .25):
+                output.append(random.choice(self.MUMBLE_LAST))
+            self.poutput(' '.join(output))
 
 if __name__ == '__main__':
     c = CmdLineApp()