Write documentation for revised transcription feature

3 files changed, 143 insertions, 24 deletions

diff --git a/docs/freefeatures.rst b/docs/freefeatures.rst
index 7b6762ad..fbaca7f6 100644
--- a/docs/freefeatures.rst
+++ b/docs/freefeatures.rst
@@ -301,34 +301,20 @@ is equivalent to ``shell ls``.)
 Transcript-based testing
 ========================
 
-If the entire transcript (input and output) of a successful session of
-a ``cmd2``-based app is copied from the screen and pasted into a text
-file, ``transcript.txt``, then a transcript test can be run against it::
+A transcript is both the input and output of a successful session of a
+``cmd2``-based app which is saved to a text file. The transcript can be played
+back into the app as a unit test.
 
-  python app.py --test transcript.txt
+.. code-block:: none
 
-Any non-whitespace deviations between the output prescribed in ``transcript.txt`` and
-the actual output from a fresh run of the application will be reported
-as a unit test failure.  (Whitespace is ignored during the comparison.)
+   $ python example.py --test transcript_regex.txt
+   .
+   ----------------------------------------------------------------------
+   Ran 1 test in 0.013s
 
-Regular expressions can be embedded in the transcript inside paired ``/``
-slashes.  These regular expressions should not include any whitespace
-expressions.
+   OK
 
-.. note::
-
-   If you have set ``allow_cli_args`` to False in order to disable parsing of command line arguments at invocation,
-   then the use of ``-t`` or ``--test`` to run transcript testing is automatically disabled.  In this case, you can
-   alternatively provide a value for the optional ``transcript_files`` when constructing the instance of your
-   ``cmd2.Cmd`` derived class in order to cause a transcript test to run::
-
-       from cmd2 import Cmd
-       class App(Cmd):
-         # customized attributes and methods here
-
-       if __name__ == '__main__':
-           app = App(transcript_files=['exampleSession.txt'])
-           app.cmdloop()
+See :doc:`<transcription>` for more details.
 
 
 Tab-Completion
diff --git a/docs/index.rst b/docs/index.rst
index e89be557..12e1f832 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -66,6 +66,7 @@ Contents:
    freefeatures
    settingchanges
    unfreefeatures
+   transcription
    integrating
    hooks
    alternatives
diff --git a/docs/transcript.rst b/docs/transcript.rst
new file mode 100644
index 00000000..9aa050b1
--- /dev/null
+++ b/docs/transcript.rst
@@ -0,0 +1,132 @@
+========================
+Transcript based testing
+========================
+
+A transcript is both the input and output of a successful session of a
+``cmd2``-based app which is saved to a text file. The transcript can be played
+back into the app as a unit test. You can embed regular expressions into the
+transcript to accomodate commands that produce dynamic or variable output.
+
+
+Creating a transcript
+=====================
+
+Here's a transcript created from ``python examples/example.py``:
+
+.. code-block:: none
+
+   (Cmd) say -r 3 Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   (Cmd) mumble maybe we could go to lunch
+   like maybe we ... could go to hmmm lunch
+   (Cmd) mumble maybe we could go to lunch
+   well maybe we could like go to er lunch right?
+
+This transcript has three commands: you can see them on the lines that begin
+with the prompt, which in this case is ``(Cmd) ``. Following each command is
+the output generated by that command.
+
+Any lines in the transcript before the first line that begins with the prompt
+are ignored. You can take advantage of this by using the first lines of the
+transcript as comments.
+
+.. code-block:: none
+
+   # Lines at the beginning of the transcript that do not
+   ; start with the prompt i.e. '(Cmd) ' are ignored.
+   /* You can use them for comments. */
+   
+   All six of these lines before the first prompt are treated as comments.
+   
+   (Cmd) say -r 3 Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   Goodnight, Gracie
+   (Cmd) mumble maybe we could go to lunch
+   like maybe we ... could go to hmmm lunch
+   (Cmd) mumble maybe we could go to lunch
+   maybe we could like go to er lunch right?
+
+In this example I've used several different commenting styles, and even bare
+text. It doesn't matter what you put on those beginning lines. Everything before
+the first line that starts with ``(Cmd) `` will be ignored.
+
+If we used this transcript as-is, it would likely fail. As you can see, the
+``mumble`` command doesn't always return the same thing. The ``mumble`` command
+inserts random words into the input. Transcripts can include regular
+expressions as a way to check for output that can change.
+
+Regular expressions can be included in the response portion of a transcript,
+and are surrounded by slashes.
+
+.. code-block:: none
+
+   (Cmd) mumble maybe we could go to lunch
+   /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/
+   (Cmd) mumble maybe we could go to lunch
+   /.*\bmaybe\b.*\bcould\b.*\blunch\b.*/
+
+Without creating a tutorial on regular expressions, this one matches anything
+that has the words `maybe`, `could`, and `lunch` in that order. It doesn't
+ensure that `we` or `go` or `to` appear in the output, but it does work if
+mumble happens to add words to the beginning or the end of the output.
+
+Since the output could be multiple lines long, ``cmd2`` uses multiline regular
+expression matching, and also uses the ``DOTALL`` flag, which subtly changes the behavior of commonly
+used special characters like `.`, `^` and `$`, so you may want to double check the
+`Python regular expression documentation
+<https://docs.python.org/3/library/re.html>`_.
+
+If your output has slashes in it, you will need to escape those slashes so the
+stuff between them is not interpred as a regular expression. In this transcript::
+
+   (Cmd) say cd /usr/local/lib/python3.6/site-packages
+   /usr/local/lib/python3.6/site-packages
+
+the output contains slashes. The text between the first slash and the second
+slash, (``usr``) will be interpreted as a regular expression, and those two
+slashes will not be included in the comparison. When replayed, this transcript
+would therefore fail. To fix it, we could either write a regular expression to
+match the path instead of specifying it verbatim, or we can escape the slashes::
+
+   (Cmd) say cd /usr/local/lib/python3.6/site-packages
+   \/usr\/local\/lib\/python3.6\/site-packages
+
+
+Running a transcript
+====================
+
+Once you have created a transcript, it's easy to have your application play it
+back and check the output. From within the ``examples/`` directory:
+
+.. code-block:: none
+
+   $ python example.py --test transcript_regex.txt
+   .
+   ----------------------------------------------------------------------
+   Ran 1 test in 0.013s
+
+   OK
+
+The output will look familiar if you use ``unittest``, because that's exactly
+what happens. Each command in the transcript is run, and the output is
+``asserted`` to match expected result from the transcript.
+
+.. note::
+
+   If you have set ``allow_cli_args`` to False in order to disable parsing of
+   command line arguments at invocation, then the use of ``-t`` or ``--test``
+   to run transcript testing is automatically disabled. In this case, you can
+   alternatively provide a value for the optional ``transcript_files`` when
+   constructing the instance of your ``cmd2.Cmd`` derived class in order to
+   cause a transcript test to run::
+
+       from cmd2 import Cmd
+       class App(Cmd):
+         # customized attributes and methods here
+
+       if __name__ == '__main__':
+           app = App(transcript_files=['exampleSession.txt'])
+           app.cmdloop()