summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--CHANGELOG.md14
-rw-r--r--CONTRIBUTING.md29
-rwxr-xr-xREADME.md8
-rw-r--r--SHLEX_TODO.txt13
-rwxr-xr-xcmd2/cmd2.py407
-rw-r--r--cmd2/constants.py2
-rw-r--r--cmd2/parsing.py419
-rw-r--r--cmd2/utils.py1
-rw-r--r--docs/freefeatures.rst8
-rw-r--r--docs/requirements.txt3
-rw-r--r--docs/settingchanges.rst2
-rw-r--r--docs/unfreefeatures.rst60
-rwxr-xr-xexamples/arg_print.py7
-rwxr-xr-xexamples/argparse_example.py2
-rwxr-xr-xexamples/example.py2
-rwxr-xr-xexamples/pirate.py2
-rwxr-xr-xsetup.py25
-rw-r--r--tests/redirect.txt1
-rw-r--r--tests/test_argparse.py14
-rw-r--r--tests/test_cmd2.py102
-rw-r--r--tests/test_completion.py38
-rw-r--r--tests/test_parsing.py518
-rw-r--r--tests/test_transcript.py9
-rw-r--r--tox.ini28
24 files changed, 925 insertions, 789 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e84825de..dac0756f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,10 +1,24 @@
## 0.9.0 (TBD, 2018)
+* Bug Fixes
+ * If self.default_to_shell is true, then redirection and piping are now properly passed to the shell. Previously it was truncated.
+ * Submenus now call all hooks, it used to just call precmd and postcmd.
* Enhancements
* Automatic completion of ``argparse`` arguments via ``cmd2.argparse_completer.AutoCompleter``
* See the [tab_autocompletion.py](https://github.com/python-cmd2/cmd2/blob/master/examples/tab_autocompletion.py) example for a demonstration of how to use this feature
* ``cmd2`` no longer depends on the ``six`` module
* ``cmd2`` is now a multi-file Python package instead of a single-file module
* New pyscript approach that provides a pythonic interface to commands in the cmd2 application.
+ * Switch command parsing from pyparsing to custom code which utilizes shlex.
+ * The object passed to do_* methods has changed. It no longer is the pyparsing object, it's a new Statement object, which is a subclass of ``str``. The statement object has many attributes which give you access to various components of the parsed input. If you were using anything but the string in your do_* methods, this change will require you to update your code.
+ * ``commentGrammers`` is no longer supported or available. Comments are C-style or python style.
+ * Input redirection no longer supported. Use the load command instead.
+ * ``multilineCommand`` attribute is ``now multiline_command``
+ * ``identchars`` is now ignored. The standardlibrary cmd uses those characters to split the first "word" of the input, but cmd2 hasn't used those for a while, and the new parsing logic parses on whitespace, which has the added benefit of full unicode support, unlike cmd or prior versions of cmd2.
+ * ``set_posix_shlex`` function and ``POSIX_SHLEX`` variable have been removed. Parsing behavior is now always the more forgiving ``posix=false``.
+ * ``set_strip_quotes`` function and ``STRIP_QUOTES_FOR_NON_POSIX`` have been removed. Quotes are stripped from arguments when presented as a list (a la ``sys.argv``), and present when arguments are presented as a string (like the string passed to do_*).
+* Changes
+ * ``strip_ansi()`` and ``strip_quotes()`` functions have moved to new utils module
+ * Several constants moved to new constants module
* Deletions (potentially breaking changes)
* Deleted all ``optparse`` code which had previously been deprecated in release 0.8.0
* The ``options`` decorator no longer exists
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index e0cbd57b..f4ebcbef 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -45,7 +45,6 @@ The tables below list all prerequisites along with the minimum required version
| Prerequisite | Minimum Version |
| --------------------------------------------------- | --------------- |
| [Python](https://www.python.org/downloads/) | `3.4` |
-| [pyparsing](http://pyparsing.wikispaces.com) | `2.1` |
| [pyperclip](https://github.com/asweigart/pyperclip) | `1.6` |
#### Additional prerequisites to run cmd2 unit tests
@@ -63,15 +62,13 @@ The tables below list all prerequisites along with the minimum required version
### Optional prerequisites for enhanced unit test features
| Prerequisite | Minimum Version |
| ------------------------------------------- | --------------- |
-| [pytest-forked](https://pypi.python.org/pypi/pytest-forked)| `0.2` |
-| [pytest-xdist](https://pypi.python.org/pypi/pytest-xdist)| `1.15` |
| [pytest-cov](https://pypi.python.org/pypi/pytest-cov) | `1.8` |
If Python is already installed in your machine, run the following commands to validate the versions:
```shell
python -V
-pip freeze | grep pyparsing
+pip freeze | grep pyperclip
```
If your versions are lower than the prerequisite versions, you should update.
@@ -190,7 +187,7 @@ Once you have cmd2 cloned, before you start any cmd2 application, you first need
```bash
# Install cmd2 prerequisites
-pip install -U pyparsing pyperclip
+pip install -U pyperclip
# Install prerequisites for running cmd2 unit tests
pip install -U pytest
@@ -198,8 +195,8 @@ pip install -U pytest
# Install prerequisites for building cmd2 documentation
pip install -U sphinx sphinx-rtd-theme
-# Install optional prerequisites for running unit tests in parallel and doing code coverage analysis
-pip install -U pytest-xdist pytest-cov pytest-forked
+# Install optional prerequisites for doing code coverage analysis
+pip install -U pytest-cov
```
For doing cmd2 development, you actually do NOT want to have cmd2 installed as a Python package.
@@ -259,27 +256,13 @@ py.test
and ensure all tests pass.
-If you have the `pytest-xdist` pytest distributed testing plugin installed, then you can use it to
-dramatically speed up test execution by running tests in parallel on multiple cores like so:
-
-```shell
-py.test -n4
-```
-where `4` should be replaced by the number of parallel threads you wish to run for testing.
-
-If you have the `pytest-forked` pytest plugin (not avilable on Windows) for running tests in isolated formed processes,
-you can speed things up even further:
-
-```shell
-py.test -nauto --forked
-```
#### Measuring code coverage
Code coverage can be measured as follows:
```shell
-py.test -nauto --cov=cmd2 --cov-report=term-missing --cov-report=html --forked
+py.test --cov=cmd2 --cov-report=term-missing --cov-report=html
```
Then use your web browser of choice to look at the results which are in `<cmd2>/htmlcov/index.html`.
@@ -287,7 +270,7 @@ Then use your web browser of choice to look at the results which are in `<cmd2>/
### Squash Your Commits
When you make a pull request, it is preferable for all of your changes to be in one commit.
-If you have made more then one commit, then you will can _squash_ your commits.
+If you have made more then one commit, then you can _squash_ your commits.
To do this, see [Squashing Your Commits](http://forum.freecodecamp.com/t/how-to-squash-multiple-commits-into-one-with-git/13231).
diff --git a/README.md b/README.md
index bf381ba5..e58d912c 100755
--- a/README.md
+++ b/README.md
@@ -23,7 +23,7 @@ Main Features
- Python scripting of your application with ``pyscript``
- Run shell commands with ``!``
- Pipe command output to shell commands with `|`
-- Redirect command output to file with `>`, `>>`; input from file with `<`
+- Redirect command output to file with `>`, `>>`
- Bare `>`, `>>` with no filename send output to paste buffer (clipboard)
- `py` enters interactive Python console (opt-in `ipy` for IPython console)
- Option to display long output using a pager with ``cmd2.Cmd.ppaged()``
@@ -57,7 +57,7 @@ pip install -U cmd2
```
cmd2 works with Python 3.4+ on Windows, macOS, and Linux. It is pure Python code with
-the only 3rd-party dependencies being on [pyparsing](http://pyparsing.wikispaces.com), and [pyperclip](https://github.com/asweigart/pyperclip).
+the only 3rd-party dependencies being on [colorama](https://github.com/tartley/colorama), and [pyperclip](https://github.com/asweigart/pyperclip).
Windows has an additional dependency on [pyreadline](https://pypi.python.org/pypi/pyreadline). Non-Windows platforms
have an additional dependency on [wcwidth](https://pypi.python.org/pypi/wcwidth). Finally, Python
3.4 has an additional dependency on [contextlib2](https://pypi.python.org/pypi/contextlib2).
@@ -91,7 +91,7 @@ Instructions for implementing each feature follow.
- Multi-line commands
- Any command accepts multi-line input when its name is listed in `Cmd.multilineCommands`.
+ Any command accepts multi-line input when its name is listed in `Cmd.multiline_commands`.
The program will keep expecting input until a line ends with any of the characters
in `Cmd.terminators` . The default terminators are `;` and `/n` (empty newline).
@@ -165,7 +165,7 @@ class CmdLineApp(cmd2.Cmd):
MUMBLE_LAST = ['right?']
def __init__(self):
- self.multilineCommands = ['orate']
+ self.multiline_commands = ['orate']
self.maxrepeats = 3
# Add stuff to settable and shortcuts before calling base class initializer
diff --git a/SHLEX_TODO.txt b/SHLEX_TODO.txt
new file mode 100644
index 00000000..70e439ce
--- /dev/null
+++ b/SHLEX_TODO.txt
@@ -0,0 +1,13 @@
+
+Notes on conversion from pyparsing to shlex taking place in the ply branch
+
+Todo List:
+- refactor Cmd2.parseline() to use StatementParser.parse()
+- refactor tab completion to use StatementParser instead of parseline()
+- self.redirector doesn't really work any more, either make it work or expire it
+- clarify Statement.args whether it should be None or '', and whether it should be a string or a list of arguments,
+ include verifying documentation in unfreefeatures.txt
+- make sure Statement.command_and_args can handle quoted string arguments well. It might have to return a list of args, not a string
+- delete SHLEX_TODO.txt once everything is done
+
+Questions:
diff --git a/cmd2/cmd2.py b/cmd2/cmd2.py
index 80fa5601..f4f30bd4 100755
--- a/cmd2/cmd2.py
+++ b/cmd2/cmd2.py
@@ -42,11 +42,10 @@ import subprocess
import sys
import tempfile
import traceback
-from typing import Callable, List, Optional, Union
+from typing import Callable, List, Optional, Union, Tuple
import unittest
from code import InteractiveConsole
-import pyparsing
import pyperclip
from . import constants
@@ -56,6 +55,8 @@ from . import utils
from .rl_utils import rl_force_redisplay, readline, rl_type, RlType
from .argparse_completer import AutoCompleter, ACArgumentParser
+from cmd2.parsing import StatementParser, Statement
+
if rl_type == RlType.PYREADLINE:
# Save the original pyreadline display completion function since we need to override it and restore it
@@ -116,22 +117,6 @@ except ImportError:
__version__ = '0.9.0'
-# Pyparsing enablePackrat() can greatly speed up parsing, but problems have been seen in Python 3 in the past
-pyparsing.ParserElement.enablePackrat()
-
-# Override the default whitespace chars in Pyparsing so that newlines are not treated as whitespace
-pyparsing.ParserElement.setDefaultWhitespaceChars(' \t')
-
-
-# The next 2 variables and associated setter functions effect how arguments are parsed for decorated commands
-# which use one of the decorators: @with_argument_list, @with_argparser, or @with_argparser_and_unknown_args
-# The defaults are sane and maximize ease of use for new applications based on cmd2.
-
-# Use POSIX or Non-POSIX (Windows) rules for splitting a command-line string into a list of arguments via shlex.split()
-POSIX_SHLEX = False
-
-# Strip outer quotes for convenience if POSIX_SHLEX = False
-STRIP_QUOTES_FOR_NON_POSIX = True
# optional attribute, when tagged on a function, allows cmd2 to categorize commands
HELP_CATEGORY = 'help_category'
@@ -153,24 +138,6 @@ def categorize(func: Union[Callable, Iterable], category: str) -> None:
setattr(func, HELP_CATEGORY, category)
-def set_posix_shlex(val: bool) -> None:
- """ Allows user of cmd2 to choose between POSIX and non-POSIX splitting of args for decorated commands.
-
- :param val: True => POSIX, False => Non-POSIX
- """
- global POSIX_SHLEX
- POSIX_SHLEX = val
-
-
-def set_strip_quotes(val: bool) -> None:
- """ Allows user of cmd2 to choose whether to automatically strip outer-quotes when POSIX_SHLEX is False.
-
- :param val: True => strip quotes on args for decorated commands if POSIX_SHLEX is False.
- """
- global STRIP_QUOTES_FOR_NON_POSIX
- STRIP_QUOTES_FOR_NON_POSIX = val
-
-
def _which(editor: str) -> Optional[str]:
try:
editor_path = subprocess.check_output(['which', editor], stderr=subprocess.STDOUT).strip()
@@ -187,13 +154,12 @@ def parse_quoted_string(cmdline: str) -> List[str]:
lexed_arglist = cmdline
else:
# Use shlex to split the command line into a list of arguments based on shell rules
- lexed_arglist = shlex.split(cmdline, posix=POSIX_SHLEX)
- # If not using POSIX shlex, make sure to strip off outer quotes for convenience
- if not POSIX_SHLEX and STRIP_QUOTES_FOR_NON_POSIX:
- temp_arglist = []
- for arg in lexed_arglist:
- temp_arglist.append(utils.strip_quotes(arg))
- lexed_arglist = temp_arglist
+ lexed_arglist = shlex.split(cmdline, posix=False)
+ # strip off outer quotes for convenience
+ temp_arglist = []
+ for arg in lexed_arglist:
+ temp_arglist.append(utils.strip_quotes(arg))
+ lexed_arglist = temp_arglist
return lexed_arglist
@@ -342,40 +308,6 @@ def write_to_paste_buffer(txt: str) -> None:
pyperclip.copy(txt)
-class ParsedString(str):
- """Subclass of str which also stores a pyparsing.ParseResults object containing structured parse results."""
- # pyarsing.ParseResults - structured parse results, to provide multiple means of access to the parsed data
- parsed = None
-
- # Function which did the parsing
- parser = None
-
- def full_parsed_statement(self):
- """Used to reconstruct the full parsed statement when a command isn't recognized."""
- new = ParsedString('%s %s' % (self.parsed.command, self.parsed.args))
- new.parsed = self.parsed
- new.parser = self.parser
- return new
-
-
-def replace_with_file_contents(fname: str) -> str:
- """Action to perform when successfully matching parse element definition for inputFrom parser.
-
- :param fname: filename
- :return: contents of file "fname"
- """
- try:
- # Any outer quotes are not part of the filename
- unquoted_file = utils.strip_quotes(fname[0])
- with open(os.path.expanduser(unquoted_file)) as source_file:
- result = source_file.read()
- except IOError:
- result = '< %s' % fname[0] # wasn't a file after all
-
- # TODO: IF pyparsing input parser logic gets fixed to support empty file, add support to get from paste buffer
- return result
-
-
class EmbeddedConsoleExit(SystemExit):
"""Custom exception class for use with the py command."""
pass
@@ -528,7 +460,7 @@ class AddSubmenu(object):
def __call__(self, cmd_obj):
"""Creates a subclass of Cmd wherein the given submenu can be accessed via the given command"""
- def enter_submenu(parent_cmd, line):
+ def enter_submenu(parent_cmd, statement):
"""
This function will be bound to do_<submenu> and will change the scope of the CLI to that of the
submenu.
@@ -547,12 +479,9 @@ class AddSubmenu(object):
# copy over any shared attributes
self._copy_in_shared_attrs(parent_cmd)
- if line.parsed.args:
+ if statement.args:
# Remove the menu argument and execute the command in the submenu
- line = submenu.parser_manager.parsed(line.parsed.args)
- submenu.precmd(line)
- ret = submenu.onecmd(line)
- submenu.postcmd(ret, line)
+ submenu.onecmd_plus_hooks(statement.args)
else:
if self.reformat_prompt is not None:
prompt = submenu.prompt
@@ -657,17 +586,13 @@ class Cmd(cmd.Cmd):
Line-oriented command interpreters are often useful for test harnesses, internal tools, and rapid prototypes.
"""
- # Attributes used to configure the ParserManager (all are not dynamically settable at runtime)
+ # Attributes used to configure the StatementParser, best not to change these at runtime
blankLinesAllowed = False
- commentGrammars = pyparsing.Or([pyparsing.pythonStyleComment, pyparsing.cStyleComment])
- commentInProgress = pyparsing.Literal('/*') + pyparsing.SkipTo(pyparsing.stringEnd ^ '*/')
- legalChars = u'!#$%.:?@_-' + pyparsing.alphanums + pyparsing.alphas8bit
- multilineCommands = []
- prefixParser = pyparsing.Empty()
+ multiline_commands = []
redirector = '>' # for sending output to file
shortcuts = {'?': 'help', '!': 'shell', '@': 'load', '@@': '_relative_load'}
aliases = dict()
- terminators = [';'] # make sure your terminators are not in legalChars!
+ terminators = [';']
# Attributes which are NOT dynamically settable at runtime
allow_cli_args = True # Should arguments passed on the command-line be processed as commands?
@@ -755,13 +680,13 @@ class Cmd(cmd.Cmd):
self.pystate = {}
self.pyscript_name = 'app'
self.keywords = self.reserved_words + [fname[3:] for fname in dir(self) if fname.startswith('do_')]
- self.parser_manager = ParserManager(redirector=self.redirector, terminators=self.terminators,
- multilineCommands=self.multilineCommands,
- legalChars=self.legalChars, commentGrammars=self.commentGrammars,
- commentInProgress=self.commentInProgress,
- blankLinesAllowed=self.blankLinesAllowed, prefixParser=self.prefixParser,
- preparse=self.preparse, postparse=self.postparse, aliases=self.aliases,
- shortcuts=self.shortcuts)
+ self.statement_parser = StatementParser(
+ allow_redirection=self.allow_redirection,
+ terminators=self.terminators,
+ multiline_commands=self.multiline_commands,
+ aliases=self.aliases,
+ shortcuts=self.shortcuts,
+ )
self._transcript_files = transcript_files
# Used to enable the ability for a Python script to quit the application
@@ -845,7 +770,6 @@ class Cmd(cmd.Cmd):
return utils.strip_ansi(self.prompt)
def _finalize_app_parameters(self):
- self.commentGrammars.ignore(pyparsing.quotedString).setParseAction(lambda x: '')
# noinspection PyUnresolvedReferences
self.shortcuts = sorted(self.shortcuts.items(), reverse=True)
@@ -1655,6 +1579,11 @@ class Cmd(cmd.Cmd):
# Parse the command line
command, args, expanded_line = self.parseline(line)
+ # use these lines instead of the one above
+ # statement = self.command_parser.parse_command_only(line)
+ # command = statement.command
+ # expanded_line = statement.command_and_args
+
# We overwrote line with a properly formatted but fully stripped version
# Restore the end spaces since line is only supposed to be lstripped when
# passed to completer functions according to Python docs
@@ -1928,18 +1857,18 @@ class Cmd(cmd.Cmd):
# Register a default SIGINT signal handler for Ctrl+C
signal.signal(signal.SIGINT, self.sigint_handler)
- def precmd(self, statement):
+ def precmd(self, statement: Statement) -> Statement:
"""Hook method executed just before the command is processed by ``onecmd()`` and after adding it to the history.
- :param statement: ParsedString - subclass of str which also contains pyparsing ParseResults instance
- :return: ParsedString - a potentially modified version of the input ParsedString statement
+ :param statement: Statement - subclass of str which also contains the parsed input
+ :return: Statement - a potentially modified version of the input Statement object
"""
return statement
# ----- Methods which are cmd2-specific lifecycle hooks which are not present in cmd -----
# noinspection PyMethodMayBeStatic
- def preparse(self, raw):
+ def preparse(self, raw: str) -> str:
"""Hook method executed just before the command line is interpreted, but after the input prompt is generated.
:param raw: str - raw command line input
@@ -1948,16 +1877,16 @@ class Cmd(cmd.Cmd):
return raw
# noinspection PyMethodMayBeStatic
- def postparse(self, parse_result):
- """Hook that runs immediately after parsing the command-line but before ``parsed()`` returns a ParsedString.
+ def postparse(self, statement: Statement) -> Statement:
+ """Hook that runs immediately after parsing the user input.
- :param parse_result: pyparsing.ParseResults - parsing results output by the pyparsing parser
- :return: pyparsing.ParseResults - potentially modified ParseResults object
+ :param statement: Statement object populated by parsing
+ :return: Statement - potentially modified Statement object
"""
- return parse_result
+ return statement
# noinspection PyMethodMayBeStatic
- def postparsing_precmd(self, statement):
+ def postparsing_precmd(self, statement: Statement) -> Tuple[bool, Statement]:
"""This runs after parsing the command-line, but before anything else; even before adding cmd to history.
NOTE: This runs before precmd() and prior to any potential output redirection or piping.
@@ -1969,14 +1898,14 @@ class Cmd(cmd.Cmd):
- raise EmptyStatement - will silently fail and do nothing
- raise <AnyOtherException> - will fail and print an error message
- :param statement: - the parsed command-line statement
- :return: (bool, statement) - (stop, statement) containing a potentially modified version of the statement
+ :param statement: - the parsed command-line statement as a Statement object
+ :return: (bool, statement) - (stop, statement) containing a potentially modified version of the statement object
"""
stop = False
return stop, statement
# noinspection PyMethodMayBeStatic
- def postparsing_postcmd(self, stop):
+ def postparsing_postcmd(self, stop: bool) -> bool:
"""This runs after everything else, including after postcmd().
It even runs when an empty line is entered. Thus, if you need to do something like update the prompt due
@@ -2136,24 +2065,52 @@ class Cmd(cmd.Cmd):
return stop
def _complete_statement(self, line):
- """Keep accepting lines of input until the command is complete."""
- if not line or (not pyparsing.Or(self.commentGrammars).setParseAction(lambda x: '').transformString(line)):
- raise EmptyStatement()
- statement = self.parser_manager.parsed(line)
- while statement.parsed.multilineCommand and (statement.parsed.terminator == ''):
- statement = '%s\n%s' % (statement.parsed.raw,
- self.pseudo_raw_input(self.continuation_prompt))
- statement = self.parser_manager.parsed(statement)
- if not statement.parsed.command:
+ """Keep accepting lines of input until the command is complete.
+
+ There is some pretty hacky code here to handle some quirks of
+ self.pseudo_raw_input(). It returns a literal 'eof' if the input
+ pipe runs out. We can't refactor it because we need to retain
+ backwards compatibility with the standard library version of cmd.
+ """
+ statement = self.statement_parser.parse(line)
+ while statement.multiline_command and not statement.terminator:
+ if not self.quit_on_sigint:
+ try:
+ newline = self.pseudo_raw_input(self.continuation_prompt)
+ if newline == 'eof':
+ # they entered either a blank line, or we hit an EOF
+ # for some other reason. Turn the literal 'eof'
+ # into a blank line, which serves as a command
+ # terminator
+ newline = '\n'
+ self.poutput(newline)
+ line = '{}\n{}'.format(statement.raw, newline)
+ except KeyboardInterrupt:
+ self.poutput('^C')
+ statement = self.statement_parser.parse('')
+ break
+ else:
+ newline = self.pseudo_raw_input(self.continuation_prompt)
+ if newline == 'eof':
+ # they entered either a blank line, or we hit an EOF
+ # for some other reason. Turn the literal 'eof'
+ # into a blank line, which serves as a command
+ # terminator
+ newline = '\n'
+ self.poutput(newline)
+ line = '{}\n{}'.format(statement.raw, newline)
+ statement = self.statement_parser.parse(line)
+
+ if not statement.command:
raise EmptyStatement()
return statement
def _redirect_output(self, statement):
"""Handles output redirection for >, >>, and |.
- :param statement: ParsedString - subclass of str which also contains pyparsing ParseResults instance
+ :param statement: Statement - a parsed statement from the user
"""
- if statement.parsed.pipeTo:
+ if statement.pipe_to:
self.kept_state = Statekeeper(self, ('stdout',))
# Create a pipe with read and write sides
@@ -2168,7 +2125,7 @@ class Cmd(cmd.Cmd):
# We want Popen to raise an exception if it fails to open the process. Thus we don't set shell to True.
try:
- self.pipe_proc = subprocess.Popen(shlex.split(statement.parsed.pipeTo), stdin=subproc_stdin)
+ self.pipe_proc = subprocess.Popen(shlex.split(statement.pipe_to), stdin=subproc_stdin)
except Exception as ex:
# Restore stdout to what it was and close the pipe
self.stdout.close()
@@ -2180,31 +2137,31 @@ class Cmd(cmd.Cmd):
# Re-raise the exception
raise ex
- elif statement.parsed.output:
- if (not statement.parsed.outputTo) and (not can_clip):
+ elif statement.output:
+ if (not statement.output_to) and (not can_clip):
raise EnvironmentError('Cannot redirect to paste buffer; install ``xclip`` and re-run to enable')
self.kept_state = Statekeeper(self, ('stdout',))
self.kept_sys = Statekeeper(sys, ('stdout',))
self.redirecting = True
- if statement.parsed.outputTo:
+ if statement.output_to:
mode = 'w'
- if statement.parsed.output == 2 * self.redirector:
+ if statement.output == 2 * self.redirector:
mode = 'a'
- sys.stdout = self.stdout = open(os.path.expanduser(statement.parsed.outputTo), mode)
+ sys.stdout = self.stdout = open(os.path.expanduser(statement.output_to), mode)
else:
sys.stdout = self.stdout = tempfile.TemporaryFile(mode="w+")
- if statement.parsed.output == '>>':
+ if statement.output == '>>':
self.poutput(get_paste_buffer())
def _restore_output(self, statement):
"""Handles restoring state after output redirection as well as the actual pipe operation if present.
- :param statement: ParsedString - subclass of str which also contains pyparsing ParseResults instance
+ :param statement: Statement object which contains the parsed input from the user
"""
# If we have redirected output to a file or the clipboard or piped it to a shell command, then restore state
if self.kept_state is not None:
# If we redirected output to the clipboard
- if statement.parsed.output and not statement.parsed.outputTo:
+ if statement.output and not statement.output_to:
self.stdout.seek(0)
write_to_paste_buffer(self.stdout.read())
@@ -2242,22 +2199,21 @@ class Cmd(cmd.Cmd):
result = target
return result
- def onecmd(self, line):
+ def onecmd(self, statement):
""" This executes the actual do_* method for a command.
If the command provided doesn't exist, then it executes _default() instead.
- :param line: ParsedString - subclass of string including the pyparsing ParseResults
+ :param statement: Command - a parsed command from the input stream
:return: bool - a flag indicating whether the interpretation of commands should stop
"""
- statement = self.parser_manager.parsed(line)
- funcname = self._func_named(statement.parsed.command)
+ funcname = self._func_named(statement.command)
if not funcname:
return self.default(statement)
# Since we have a valid command store it in the history
- if statement.parsed.command not in self.exclude_from_history:
- self.history.append(statement.parsed.raw)
+ if statement.command not in self.exclude_from_history:
+ self.history.append(statement.raw)
try:
func = getattr(self, funcname)
@@ -2270,10 +2226,10 @@ class Cmd(cmd.Cmd):
def default(self, statement):
"""Executed when the command given isn't a recognized command implemented by a do_* method.
- :param statement: ParsedString - subclass of string including the pyparsing ParseResults
+ :param statement: Statement object with parsed input
:return:
"""
- arg = statement.full_parsed_statement()
+ arg = statement.raw
if self.default_to_shell:
result = os.system(arg)
# If os.system() succeeded, then don't print warning about unknown command
@@ -2734,14 +2690,8 @@ Usage: Usage: unalias [-a] name [name ...]
read_only_settings = """
Commands may be terminated with: {}
Arguments at invocation allowed: {}
- Output redirection and pipes allowed: {}
- Parsing of command arguments:
- Shell lexer mode for command argument splitting: {}
- Strip Quotes after splitting arguments: {}
- """.format(str(self.terminators), self.allow_cli_args, self.allow_redirection,
- "POSIX" if POSIX_SHLEX else "non-POSIX",
- "True" if STRIP_QUOTES_FOR_NON_POSIX and not POSIX_SHLEX else "False")
- return read_only_settings
+ Output redirection and pipes allowed: {}"""
+ return read_only_settings.format(str(self.terminators), self.allow_cli_args, self.allow_redirection)
def show(self, args, parameter):
param = ''
@@ -2764,7 +2714,7 @@ Usage: Usage: unalias [-a] name [name ...]
if args.all:
self.poutput('\nRead only settings:{}'.format(self.cmdenvironment()))
else:
- raise LookupError("Parameter '%s' not supported (type 'show' for list of parameters)." % param)
+ raise LookupError("Parameter '%s' not supported (type 'set' for list of parameters)." % param)
set_parser = ACArgumentParser(formatter_class=argparse.RawTextHelpFormatter)
set_parser.add_argument('-a', '--all', action='store_true', help='display read-only settings as well')
@@ -3299,167 +3249,6 @@ Script should contain one command per line, just like command would be typed in
self.postloop()
-# noinspection PyPep8Naming
-class ParserManager:
- """
- Class which encapsulates all of the pyparsing parser functionality for cmd2 in a single location.
- """
- def __init__(self, redirector, terminators, multilineCommands, legalChars, commentGrammars, commentInProgress,
- blankLinesAllowed, prefixParser, preparse, postparse, aliases, shortcuts):
- """Creates and uses parsers for user input according to app's parameters."""
-
- self.commentGrammars = commentGrammars
- self.preparse = preparse
- self.postparse = postparse
- self.aliases = aliases
- self.shortcuts = shortcuts
-
- self.main_parser = self._build_main_parser(redirector=redirector, terminators=terminators,
- multilineCommands=multilineCommands, legalChars=legalChars,
- commentInProgress=commentInProgress,
- blankLinesAllowed=blankLinesAllowed, prefixParser=prefixParser)
- self.input_source_parser = self._build_input_source_parser(legalChars=legalChars,
- commentInProgress=commentInProgress)
-
- def _build_main_parser(self, redirector, terminators, multilineCommands, legalChars, commentInProgress,
- blankLinesAllowed, prefixParser):
- """Builds a PyParsing parser for interpreting user commands."""
-
- # Build several parsing components that are eventually compiled into overall parser
- output_destination_parser = (pyparsing.Literal(redirector * 2) |
- (pyparsing.WordStart() + redirector) |
- pyparsing.Regex('[^=]' + redirector))('output')
-
- terminator_parser = pyparsing.Or(
- [(hasattr(t, 'parseString') and t) or pyparsing.Literal(t) for t in terminators])('terminator')
- string_end = pyparsing.stringEnd ^ '\nEOF'
- multilineCommand = pyparsing.Or(
- [pyparsing.Keyword(c, caseless=False) for c in multilineCommands])('multilineCommand')
- oneline_command = (~multilineCommand + pyparsing.Word(legalChars))('command')
- pipe = pyparsing.Keyword('|', identChars='|')
- do_not_parse = self.commentGrammars | commentInProgress | pyparsing.quotedString
- after_elements = \
- pyparsing.Optional(pipe + pyparsing.SkipTo(output_destination_parser ^ string_end,
- ignore=do_not_parse)('pipeTo')) + \
- pyparsing.Optional(output_destination_parser +
- pyparsing.SkipTo(string_end, ignore=do_not_parse).
- setParseAction(lambda x: utils.strip_quotes(x[0].strip()))('outputTo'))
-
- multilineCommand.setParseAction(lambda x: x[0])
- oneline_command.setParseAction(lambda x: x[0])
-
- if blankLinesAllowed:
- blankLineTerminationParser = pyparsing.NoMatch
- else:
- blankLineTerminator = (pyparsing.lineEnd + pyparsing.lineEnd)('terminator')
- blankLineTerminator.setResultsName('terminator')
- blankLineTerminationParser = ((multilineCommand ^ oneline_command) +
- pyparsing.SkipTo(blankLineTerminator, ignore=do_not_parse).setParseAction(
- lambda x: x[0].strip())('args') + blankLineTerminator)('statement')
-
- multilineParser = (((multilineCommand ^ oneline_command) +
- pyparsing.SkipTo(terminator_parser,
- ignore=do_not_parse).setParseAction(lambda x: x[0].strip())('args') +
- terminator_parser)('statement') +
- pyparsing.SkipTo(output_destination_parser ^ pipe ^ string_end,
- ignore=do_not_parse).setParseAction(lambda x: x[0].strip())('suffix') +
- after_elements)
- multilineParser.ignore(commentInProgress)
-
- singleLineParser = ((oneline_command +
- pyparsing.SkipTo(terminator_parser ^ string_end ^ pipe ^ output_destination_parser,
- ignore=do_not_parse).setParseAction(
- lambda x: x[0].strip())('args'))('statement') +
- pyparsing.Optional(terminator_parser) + after_elements)
-
- blankLineTerminationParser = blankLineTerminationParser.setResultsName('statement')
-
- parser = prefixParser + (
- string_end |
- multilineParser |
- singleLineParser |
- blankLineTerminationParser |
- multilineCommand + pyparsing.SkipTo(string_end, ignore=do_not_parse)
- )
- parser.ignore(self.commentGrammars)
- return parser
-
- @staticmethod
- def _build_input_source_parser(legalChars, commentInProgress):
- """Builds a PyParsing parser for alternate user input sources (from file, pipe, etc.)"""
-
- input_mark = pyparsing.Literal('<')
- input_mark.setParseAction(lambda x: '')
-
- # Also allow spaces, slashes, and quotes
- file_name = pyparsing.Word(legalChars + ' /\\"\'')
-
- input_from = file_name('inputFrom')
- input_from.setParseAction(replace_with_file_contents)
- # a not-entirely-satisfactory way of distinguishing < as in "import from" from <
- # as in "lesser than"
- inputParser = input_mark + pyparsing.Optional(input_from) + pyparsing.Optional('>') + \
- pyparsing.Optional(file_name) + (pyparsing.stringEnd | '|')
- inputParser.ignore(commentInProgress)
- return inputParser
-
- def parsed(self, raw):
- """ This function is where the actual parsing of each line occurs.
-
- :param raw: str - the line of text as it was entered
- :return: ParsedString - custom subclass of str with extra attributes
- """
- if isinstance(raw, ParsedString):
- p = raw
- else:
- # preparse is an overridable hook; default makes no changes
- s = self.preparse(raw)
- s = self.input_source_parser.transformString(s.lstrip())
- s = self.commentGrammars.transformString(s)
-
- # Make a copy of aliases so we can edit it
- tmp_aliases = list(self.aliases.keys())
- keep_expanding = len(tmp_aliases) > 0
-
- # Expand aliases
- while keep_expanding:
- for cur_alias in tmp_aliases:
- keep_expanding = False
-
- if s == cur_alias or s.startswith(cur_alias + ' '):
- s = s.replace(cur_alias, self.aliases[cur_alias], 1)
-
- # Do not expand the same alias more than once
- tmp_aliases.remove(cur_alias)
- keep_expanding = len(tmp_aliases) > 0
- break
-
- # Expand command shortcut to its full command name
- for (shortcut, expansion) in self.shortcuts:
- if s.startswith(shortcut):
- # If the next character after the shortcut isn't a space, then insert one
- shortcut_len = len(shortcut)
- if len(s) == shortcut_len or s[shortcut_len] != ' ':
- expansion += ' '
-
- # Expand the shortcut
- s = s.replace(shortcut, expansion, 1)
- break
-
- try:
- result = self.main_parser.parseString(s)
- except pyparsing.ParseException:
- # If we have a parsing failure, treat it is an empty command and move to next prompt
- result = self.main_parser.parseString('')
- result['raw'] = raw
- result['command'] = result.multilineCommand or result.command
- result = self.postparse(result)
- p = ParsedString(result.args)
- p.parsed = result
- p.parser = self.parsed
- return p
-
-
class HistoryItem(str):
"""Class used to represent an item in the History list.
diff --git a/cmd2/constants.py b/cmd2/constants.py
index 6784264f..838650e5 100644
--- a/cmd2/constants.py
+++ b/cmd2/constants.py
@@ -6,7 +6,7 @@ import re
# Used for command parsing, tab completion and word breaks. Do not change.
QUOTES = ['"', "'"]
-REDIRECTION_CHARS = ['|', '<', '>']
+REDIRECTION_CHARS = ['|', '>']
# Regular expression to match ANSI escape codes
ANSI_ESCAPE_RE = re.compile(r'\x1b[^m]*m')
diff --git a/cmd2/parsing.py b/cmd2/parsing.py
new file mode 100644
index 00000000..7046b674
--- /dev/null
+++ b/cmd2/parsing.py
@@ -0,0 +1,419 @@
+#
+# -*- coding: utf-8 -*-
+"""Statement parsing classes for cmd2"""
+
+import re
+import shlex
+from typing import List, Tuple
+
+from . import constants
+
+LINE_FEED = '\n'
+
+
+class Statement(str):
+ """String subclass with additional attributes to store the results of parsing.
+
+ The cmd module in the standard library passes commands around as a
+ string. To retain backwards compatibility, cmd2 does the same. However, we
+ need a place to capture the additional output of the command parsing, so we add
+ our own attributes to this subclass.
+
+ The string portion of the class contains the arguments, but not the command, nor
+ the output redirection clauses.
+
+ :var raw: string containing exactly what we input by the user
+ :type raw: str
+ :var command: the command, i.e. the first whitespace delimited word
+ :type command: str or None
+ :var multiline_command: if the command is a multiline command, the name of the
+ command, otherwise None
+ :type command: str or None
+ :var args: the arguments to the command, not including any output
+ redirection or terminators. quoted arguments remain
+ quoted.
+ :type args: str
+ :var terminator: the charater which terminated the multiline command, if
+ there was one
+ :type terminator: str or None
+ :var suffix: characters appearing after the terminator but before output
+ redirection, if any
+ :type suffix: str or None
+ :var pipe_to: if output was piped to a shell command, the shell command
+ :type pipe_to: str or None
+ :var output: if output was redirected, the redirection token, i.e. '>>'
+ :type output: str or None
+ :var output_to: if output was redirected, the destination, usually a filename
+ :type output_to: str or None
+
+ """
+ def __init__(self, obj):
+ super().__init__()
+ self.raw = str(obj)
+ self.command = None
+ self.multiline_command = None
+ # has to be an empty string for compatibility with standard library cmd
+ self.args = ''
+ self.terminator = None
+ self.suffix = None
+ self.pipe_to = None
+ self.output = None
+ self.output_to = None
+
+ @property
+ def command_and_args(self):
+ """Combine command and args with a space separating them.
+
+ Quoted arguments remain quoted.
+ """
+ return '{} {}'.format('' if self.command is None else self.command, self.args).strip()
+
+
+class StatementParser():
+ """Parse raw text into command components.
+
+ Shortcuts is a list of tuples with each tuple containing the shortcut and the expansion.
+ """
+ def __init__(
+ self,
+ allow_redirection=True,
+ terminators=None,
+ multiline_commands=None,
+ aliases=None,
+ shortcuts=None,
+ ):
+ self.allow_redirection = allow_redirection
+ if terminators is None:
+ self.terminators = [';']
+ else:
+ self.terminators = terminators
+ if multiline_commands is None:
+ self.multiline_commands = []
+ else:
+ self.multiline_commands = multiline_commands
+ if aliases is None:
+ self.aliases = {}
+ else:
+ self.aliases = aliases
+ if shortcuts is None:
+ self.shortcuts = []
+ else:
+ self.shortcuts = shortcuts
+
+ # this regular expression matches C-style comments and quoted
+ # strings, i.e. stuff between single or double quote marks
+ # it's used with _comment_replacer() to strip out the C-style
+ # comments, while leaving C-style comments that are inside either
+ # double or single quotes.
+ #
+ # this big regular expression can be broken down into 3 regular
+ # expressions that are OR'ed together.
+ #
+ # /\*.*?(\*/|$) matches C-style comments, with an optional
+ # closing '*/'. The optional closing '*/' is
+ # there to retain backward compatibility with
+ # the pyparsing implementation of cmd2 < 0.9.0
+ # \'(?:\\.|[^\\\'])*\' matches a single quoted string, allowing
+ # for embedded backslash escaped single quote
+ # marks
+ # "(?:\\.|[^\\"])*" matches a double quoted string, allowing
+ # for embedded backslash escaped double quote
+ # marks
+ #
+ # by way of reminder the (?:...) regular expression syntax is just
+ # a non-capturing version of regular parenthesis. We need the non-
+ # capturing syntax because _comment_replacer() looks at match
+ # groups
+ self.comment_pattern = re.compile(
+ r'/\*.*?(\*/|$)|\'(?:\\.|[^\\\'])*\'|"(?:\\.|[^\\"])*"',
+ re.DOTALL | re.MULTILINE
+ )
+
+ # aliases have to be a word, so make a regular expression
+ # that matches the first word in the line. This regex has two
+ # parts, the first parenthesis enclosed group matches one
+ # or more non-whitespace characters, and the second group
+ # matches either a whitespace character or the end of the
+ # string. We use \A and \Z to ensure we always match the
+ # beginning and end of a string that may have multiple
+ # lines
+ self.command_pattern = re.compile(r'\A(\S+)(\s|\Z)')
+
+
+ def tokenize(self, line: str) -> List[str]:
+ """Lex a string into a list of tokens.
+
+ Comments are removed, and shortcuts and aliases are expanded.
+ """
+
+ # strip C-style comments
+ # shlex will handle the python/shell style comments for us
+ line = re.sub(self.comment_pattern, self._comment_replacer, line)
+
+ # expand shortcuts and aliases
+ line = self._expand(line)
+
+ # split on whitespace
+ lexer = shlex.shlex(line, posix=False)
+ lexer.whitespace_split = True
+
+ # custom lexing
+ tokens = self._split_on_punctuation(list(lexer))
+ return tokens
+
+ def parse(self, rawinput: str) -> Statement:
+ """Tokenize the input and parse it into a Statement object, stripping
+ comments, expanding aliases and shortcuts, and extracting output
+ redirection directives.
+ """
+
+ # handle the special case/hardcoded terminator of a blank line
+ # we have to do this before we tokenize because tokenizing
+ # destroys all unquoted whitespace in the input
+ terminator = None
+ if rawinput[-1:] == LINE_FEED:
+ terminator = LINE_FEED
+
+ command = None
+ args = ''
+
+ # lex the input into a list of tokens
+ tokens = self.tokenize(rawinput)
+
+ # of the valid terminators, find the first one to occur in the input
+ terminator_pos = len(tokens)+1
+ for test_terminator in self.terminators:
+ try:
+ pos = tokens.index(test_terminator)
+ if pos < terminator_pos:
+ terminator_pos = pos
+ terminator = test_terminator
+ break
+ except ValueError:
+ # the terminator is not in the tokens
+ pass
+
+ if terminator:
+ if terminator == LINE_FEED:
+ terminator_pos = len(tokens)+1
+ else:
+ terminator_pos = tokens.index(terminator)
+ # everything before the first terminator is the command and the args
+ (command, args) = self._command_and_args(tokens[:terminator_pos])
+ # we will set the suffix later
+ # remove all the tokens before and including the terminator
+ tokens = tokens[terminator_pos+1:]
+ else:
+ (testcommand, testargs) = self._command_and_args(tokens)
+ if testcommand in self.multiline_commands:
+ # no terminator on this line but we have a multiline command
+ # everything else on the line is part of the args
+ # because redirectors can only be after a terminator
+ command = testcommand
+ args = testargs
+ tokens = []
+
+ # check for output redirect
+ output = None
+ output_to = None
+ try:
+ output_pos = tokens.index('>')
+ output = '>'
+ output_to = ' '.join(tokens[output_pos+1:])
+ # remove all the tokens after the output redirect
+ tokens = tokens[:output_pos]
+ except ValueError:
+ pass
+
+ try:
+ output_pos = tokens.index('>>')
+ output = '>>'
+ output_to = ' '.join(tokens[output_pos+1:])
+ # remove all tokens after the output redirect
+ tokens = tokens[:output_pos]
+ except ValueError:
+ pass
+
+ # check for pipes
+ try:
+ # find the first pipe if it exists
+ pipe_pos = tokens.index('|')
+ # save everything after the first pipe
+ pipe_to = ' '.join(tokens[pipe_pos+1:])
+ # remove all the tokens after the pipe
+ tokens = tokens[:pipe_pos]
+ except ValueError:
+ # no pipe in the tokens
+ pipe_to = None
+
+ if terminator:
+ # whatever is left is the suffix
+ suffix = ' '.join(tokens)
+ else:
+ # no terminator, so whatever is left is the command and the args
+ suffix = None
+ if not command:
+ # command could already have been set, if so, don't set it again
+ (command, args) = self._command_and_args(tokens)
+
+ # set multiline
+ if command in self.multiline_commands:
+ multiline_command = command
+ else:
+ multiline_command = None
+
+ # build the statement
+ statement = Statement(args)
+ statement.raw = rawinput
+ statement.command = command
+ statement.args = args
+ statement.terminator = terminator
+ statement.output = output
+ statement.output_to = output_to
+ statement.pipe_to = pipe_to
+ statement.suffix = suffix
+ statement.multiline_command = multiline_command
+ return statement
+
+ def parse_command_only(self, rawinput: str) -> Statement:
+ """Partially parse input into a Statement object. The command is
+ identified, and shortcuts and aliases are expanded.
+ Terminators, multiline commands, and output redirection are not
+ parsed.
+ """
+ # lex the input into a list of tokens
+ tokens = self.tokenize(rawinput)
+
+ # parse out the command and everything else
+ (command, args) = self._command_and_args(tokens)
+
+ # build the statement
+ statement = Statement(args)
+ statement.raw = rawinput
+ statement.command = command
+ statement.args = args
+ return statement
+
+ def _expand(self, line: str) -> str:
+ """Expand shortcuts and aliases"""
+
+ # expand aliases
+ # make a copy of aliases so we can edit it
+ tmp_aliases = list(self.aliases.keys())
+ keep_expanding = bool(tmp_aliases)
+ while keep_expanding:
+ for cur_alias in tmp_aliases:
+ keep_expanding = False
+ # apply our regex to line
+ match = self.command_pattern.search(line)
+ if match:
+ # we got a match, extract the command
+ command = match.group(1)
+ if command == cur_alias:
+ # rebuild line with the expanded alias
+ line = self.aliases[cur_alias] + match.group(2) + line[match.end(2):]
+ tmp_aliases.remove(cur_alias)
+ keep_expanding = bool(tmp_aliases)
+ break
+
+ # expand shortcuts
+ for (shortcut, expansion) in self.shortcuts:
+ if line.startswith(shortcut):
+ # If the next character after the shortcut isn't a space, then insert one
+ shortcut_len = len(shortcut)
+ if len(line) == shortcut_len or line[shortcut_len] != ' ':
+ expansion += ' '
+
+ # Expand the shortcut
+ line = line.replace(shortcut, expansion, 1)
+ break
+ return line
+
+ @staticmethod
+ def _command_and_args(tokens: List[str]) -> Tuple[str, str]:
+ """given a list of tokens, and return a tuple of the command
+ and the args as a string.
+ """
+ command = None
+ args = ''
+
+ if tokens:
+ command = tokens[0]
+
+ if len(tokens) > 1:
+ args = ' '.join(tokens[1:])
+
+ return (command, args)
+
+ @staticmethod
+ def _comment_replacer(match):
+ matched_string = match.group(0)
+ if matched_string.startswith('/'):
+ # the matched string was a comment, so remove it
+ return ''
+ # the matched string was a quoted string, return the match
+ return matched_string
+
+ def _split_on_punctuation(self, tokens: List[str]) -> List[str]:
+ """
+ # Further splits tokens from a command line using punctuation characters
+ # as word breaks when they are in unquoted strings. Each run of punctuation
+ # characters is treated as a single token.
+
+ :param initial_tokens: the tokens as parsed by shlex
+ :return: the punctuated tokens
+ """
+ punctuation = []
+ punctuation.extend(self.terminators)
+ if self.allow_redirection:
+ punctuation.extend(constants.REDIRECTION_CHARS)
+
+ punctuated_tokens = []
+
+ for cur_initial_token in tokens:
+
+ # Save tokens up to 1 character in length or quoted tokens. No need to parse these.
+ if len(cur_initial_token) <= 1 or cur_initial_token[0] in constants.QUOTES:
+ punctuated_tokens.append(cur_initial_token)
+ continue
+
+ # Iterate over each character in this token
+ cur_index = 0
+ cur_char = cur_initial_token[cur_index]
+
+ # Keep track of the token we are building
+ new_token = ''
+
+ while True:
+ if cur_char not in punctuation:
+
+ # Keep appending to new_token until we hit a punctuation char
+ while cur_char not in punctuation:
+ new_token += cur_char
+ cur_index += 1
+ if cur_index < len(cur_initial_token):
+ cur_char = cur_initial_token[cur_index]
+ else:
+ break
+
+ else:
+ cur_punc = cur_char
+
+ # Keep appending to new_token until we hit something other than cur_punc
+ while cur_char == cur_punc:
+ new_token += cur_char
+ cur_index += 1
+ if cur_index < len(cur_initial_token):
+ cur_char = cur_initial_token[cur_index]
+ else:
+ break
+
+ # Save the new token
+ punctuated_tokens.append(new_token)
+ new_token = ''
+
+ # Check if we've viewed all characters
+ if cur_index >= len(cur_initial_token):
+ break
+
+ return punctuated_tokens
diff --git a/cmd2/utils.py b/cmd2/utils.py
index a975c6b8..dbe39213 100644
--- a/cmd2/utils.py
+++ b/cmd2/utils.py
@@ -5,7 +5,6 @@
import collections
from . import constants
-
def strip_ansi(text: str) -> str:
"""Strip ANSI escape codes from a string.
diff --git a/docs/freefeatures.rst b/docs/freefeatures.rst
index 419dc7d8..156b003e 100644
--- a/docs/freefeatures.rst
+++ b/docs/freefeatures.rst
@@ -32,11 +32,7 @@ Comments
Comments are omitted from the argument list
before it is passed to a ``do_`` method. By
default, both Python-style and C-style comments
-are recognized; you may change this by overriding
-``app.commentGrammars`` with a different pyparsing_
-grammar (see the arg_print_ example for specifically how to to this).
-
-Comments can be useful in :ref:`scripts`, but would
+are recognized. Comments can be useful in :ref:`scripts`, but would
be pointless within an interactive session.
::
@@ -49,7 +45,6 @@ be pointless within an interactive session.
(Cmd) speak it was /* not */ delicious! # Yuck!
it was delicious!
-.. _pyparsing: http://pyparsing.wikispaces.com/
.. _arg_print: https://github.com/python-cmd2/cmd2/blob/master/examples/arg_print.py
Startup Initialization Script
@@ -102,6 +97,7 @@ Output redirection
As in a Unix shell, output of a command can be redirected:
- sent to a file with ``>``, as in ``mycommand args > filename.txt``
+ - appended to a file with ``>>``, as in ``mycommand args >> filename.txt``
- piped (``|``) as input to operating-system commands, as in
``mycommand args | wc``
- sent to the paste buffer, ready for the next Copy operation, by
diff --git a/docs/requirements.txt b/docs/requirements.txt
index d22f9341..011603fb 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,4 +1,3 @@
-pyparsing
+colorama
pyperclip
wcwidth
-colorama
diff --git a/docs/settingchanges.rst b/docs/settingchanges.rst
index 8e87d341..02955273 100644
--- a/docs/settingchanges.rst
+++ b/docs/settingchanges.rst
@@ -44,7 +44,7 @@ To define more shortcuts, update the dict ``App.shortcuts`` with the
Shortcuts need to be created by updating the ``shortcuts`` dictionary attribute prior to calling the
``cmd2.Cmd`` super class ``__init__()`` method. Moreover, that super class init method needs to be called after
updating the ``shortcuts`` attribute This warning applies in general to many other attributes which are not
- settable at runtime such as ``commentGrammars``, ``multilineCommands``, etc.
+ settable at runtime such as ``multiline_commands``, etc.
Aliases
diff --git a/docs/unfreefeatures.rst b/docs/unfreefeatures.rst
index d420797d..6f22c1e0 100644
--- a/docs/unfreefeatures.rst
+++ b/docs/unfreefeatures.rst
@@ -7,7 +7,7 @@ Multiline commands
Command input may span multiple lines for the
commands whose names are listed in the
-parameter ``app.multilineCommands``. These
+parameter ``app.multiline_commands``. These
commands will be executed only
after the user has entered a *terminator*.
By default, the command terminators is
@@ -22,11 +22,9 @@ Parsed statements
=================
``cmd2`` passes ``arg`` to a ``do_`` method (or
-``default``) as a ParsedString, a subclass of
-string that includes an attribute ``parsed``.
-``parsed`` is a ``pyparsing.ParseResults``
-object produced by applying a pyparsing_
-grammar applied to ``arg``. It may include:
+``default``) as a Statement, a subclass of
+string that includes many attributes of the parsed
+input:
command
Name of the command called
@@ -37,47 +35,21 @@ raw
terminator
Character used to end a multiline command
-suffix
- Remnant of input after terminator
+command_and_args
+ A string of just the command and the arguments, with
+ output redirection or piping to shell commands removed
-::
-
- def do_parsereport(self, arg):
- self.stdout.write(arg.parsed.dump() + '\n')
+If ``Statement`` does not contain an attribute,
+querying for it will return ``None``.
-::
-
- (Cmd) parsereport A B /* C */ D; E
- ['parsereport', 'A B D', ';', 'E']
- - args: A B D
- - command: parsereport
- - raw: parsereport A B /* C */ D; E
- - statement: ['parsereport', 'A B D', ';']
- - args: A B D
- - command: parsereport
- - terminator: ;
- - suffix: E
- - terminator: ;
-
-If ``parsed`` does not contain an attribute,
-querying for it will return ``None``. (This
-is a characteristic of ``pyparsing.ParseResults``.)
-
-The parsing grammar and process currently employed
-by cmd2 is stable, but is likely significantly more
-complex than it needs to be. Future ``cmd2`` releases may
-change it somewhat (hopefully reducing complexity).
-
-(Getting ``arg`` as a ``ParsedString`` is
+(Getting ``arg`` as a ``Statement`` is
technically "free", in that it requires no application
changes from the cmd_ standard, but there will
be no result unless you change your application
-to *use* ``arg.parsed``.)
+to *use* any of the additional attributes.)
.. _cmd: https://docs.python.org/3/library/cmd.html
-.. _pyparsing: http://pyparsing.wikispaces.com/
-
Environment parameters
======================
@@ -129,16 +101,6 @@ that module.
``cmd2`` defines a few decorators which change the behavior of
how arguments get parsed for and passed to a ``do_`` method. See the section :ref:`decorators` for more information.
-Controlling how arguments are parsed for commands with flags
-------------------------------------------------------------
-There are a couple functions which can globally effect how arguments are parsed for commands with flags:
-
-.. autofunction:: cmd2.set_posix_shlex
-
-.. autofunction:: cmd2.set_strip_quotes
-
-.. _argparse: https://docs.python.org/3/library/argparse.html
-
poutput, pfeedback, perror, ppaged
==================================
diff --git a/examples/arg_print.py b/examples/arg_print.py
index 90df053a..b2f0fcda 100755
--- a/examples/arg_print.py
+++ b/examples/arg_print.py
@@ -19,16 +19,13 @@ class ArgumentAndOptionPrinter(cmd2.Cmd):
""" Example cmd2 application where we create commands that just print the arguments they are called with."""
def __init__(self):
- # Uncomment this line to disable Python-style comments but still allow C-style comments
- # self.commentGrammars = pyparsing.Or([pyparsing.cStyleComment])
-
# Create command aliases which are shorter
self.shortcuts.update({'$': 'aprint', '%': 'oprint'})
- # Make sure to call this super class __init__ *after* setting commentGrammars and/or updating shortcuts
+ # Make sure to call this super class __init__ *after* setting and/or updating shortcuts
super().__init__()
# NOTE: It is critical that the super class __init__ method be called AFTER updating certain parameters which
- # are not settable at runtime. This includes the commentGrammars, shortcuts, multilineCommands, etc.
+ # are not settable at runtime. This includes the shortcuts, multiline_commands, etc.
def do_aprint(self, arg):
"""Print the argument string this basic command is called with."""
diff --git a/examples/argparse_example.py b/examples/argparse_example.py
index e8afef5c..6e5dcf35 100755
--- a/examples/argparse_example.py
+++ b/examples/argparse_example.py
@@ -20,7 +20,7 @@ from cmd2.cmd2 import Cmd, with_argparser, with_argument_list
class CmdLineApp(Cmd):
""" Example cmd2 application. """
def __init__(self, ip_addr=None, port=None, transcript_files=None):
- self.multilineCommands = ['orate']
+ self.multiline_commands = ['orate']
self.shortcuts.update({'&': 'speak'})
self.maxrepeats = 3
diff --git a/examples/example.py b/examples/example.py
index 1fc6bf6d..f07b9c74 100755
--- a/examples/example.py
+++ b/examples/example.py
@@ -27,7 +27,7 @@ class CmdLineApp(Cmd):
MUMBLE_LAST = ['right?']
def __init__(self):
- self.multilineCommands = ['orate']
+ self.multiline_commands = ['orate']
self.maxrepeats = 3
# Add stuff to settable and shortcuts before calling base class initializer
diff --git a/examples/pirate.py b/examples/pirate.py
index 2daa8631..f6f4c629 100755
--- a/examples/pirate.py
+++ b/examples/pirate.py
@@ -14,7 +14,7 @@ class Pirate(Cmd):
"""A piratical example cmd2 application involving looting and drinking."""
def __init__(self):
self.default_to_shell = True
- self.multilineCommands = ['sing']
+ self.multiline_commands = ['sing']
self.terminators = Cmd.terminators + ['...']
self.songcolor = 'blue'
diff --git a/setup.py b/setup.py
index 3020cf24..639afe1c 100755
--- a/setup.py
+++ b/setup.py
@@ -3,16 +3,13 @@
"""
Setuptools setup file, used to install or test 'cmd2'
"""
-import sys
-
-import setuptools
from setuptools import setup
VERSION = '0.9.0'
DESCRIPTION = "cmd2 - a tool for building interactive command line applications in Python"
-LONG_DESCRIPTION = """cmd2 is a tool for building interactive command line applications in Python. Its goal is to make
-it quick and easy for developers to build feature-rich and user-friendly interactive command line applications. It
-provides a simple API which is an extension of Python's built-in cmd module. cmd2 provides a wealth of features on top
+LONG_DESCRIPTION = """cmd2 is a tool for building interactive command line applications in Python. Its goal is to make
+it quick and easy for developers to build feature-rich and user-friendly interactive command line applications. It
+provides a simple API which is an extension of Python's built-in cmd module. cmd2 provides a wealth of features on top
of cmd to make your life easier and eliminates much of the boilerplate code which would be necessary when using cmd.
The latest documentation for cmd2 can be read online here:
@@ -25,7 +22,7 @@ Main features:
- Python scripting of your application with ``pyscript``
- Run shell commands with ``!``
- Pipe command output to shell commands with `|`
- - Redirect command output to file with `>`, `>>`; input from file with `<`
+ - Redirect command output to file with `>`, `>>`
- Bare `>`, `>>` with no filename send output to paste buffer (clipboard)
- `py` enters interactive Python console (opt-in `ipy` for IPython console)
- Multi-line commands
@@ -61,7 +58,7 @@ Programming Language :: Python :: Implementation :: PyPy3
Topic :: Software Development :: Libraries :: Python Modules
""".splitlines())))
-INSTALL_REQUIRES = ['pyparsing >= 2.1.0', 'pyperclip >= 1.5.27', 'colorama']
+INSTALL_REQUIRES = ['pyperclip >= 1.5.27', 'colorama']
EXTRAS_REQUIRE = {
# Windows also requires pyreadline to ensure tab completion works
@@ -72,18 +69,7 @@ EXTRAS_REQUIRE = {
":python_version<'3.5'": ['contextlib2', 'typing'],
}
-if int(setuptools.__version__.split('.')[0]) < 18:
- EXTRAS_REQUIRE = {}
- if sys.platform.startswith('win'):
- INSTALL_REQUIRES.append('pyreadline')
- else:
- INSTALL_REQUIRES.append('wcwidth')
- if sys.version_info < (3, 5):
- INSTALL_REQUIRES.append('contextlib2')
- INSTALL_REQUIRES.append('typing')
-
TESTS_REQUIRE = ['pytest', 'pytest-xdist']
-DOCS_REQUIRE = ['sphinx', 'sphinx_rtd_theme', 'pyparsing', 'pyperclip', 'wcwidth']
setup(
name="cmd2",
@@ -98,6 +84,7 @@ setup(
platforms=['any'],
packages=['cmd2'],
keywords='command prompt console cmd',
+ python_requires='>=3.4',
install_requires=INSTALL_REQUIRES,
extras_require=EXTRAS_REQUIRE,
tests_require=TESTS_REQUIRE,
diff --git a/tests/redirect.txt b/tests/redirect.txt
deleted file mode 100644
index c375d5ba..00000000
--- a/tests/redirect.txt
+++ /dev/null
@@ -1 +0,0 @@
-history
diff --git a/tests/test_argparse.py b/tests/test_argparse.py
index e23c5d17..94a7b5ed 100644
--- a/tests/test_argparse.py
+++ b/tests/test_argparse.py
@@ -124,8 +124,6 @@ def test_argparse_basic_command(argparse_app):
assert out == ['hello']
def test_argparse_quoted_arguments(argparse_app):
- argparse_app.POSIX = False
- argparse_app.STRIP_QUOTES_FOR_NON_POSIX = True
out = run_cmd(argparse_app, 'say "hello there"')
assert out == ['hello there']
@@ -138,21 +136,9 @@ def test_argparse_with_list_and_empty_doc(argparse_app):
assert out == ['HELLO WORLD!']
def test_argparse_quoted_arguments_multiple(argparse_app):
- argparse_app.POSIX = False
- argparse_app.STRIP_QUOTES_FOR_NON_POSIX = True
out = run_cmd(argparse_app, 'say "hello there" "rick & morty"')
assert out == ['hello there rick & morty']
-def test_argparse_quoted_arguments_posix(argparse_app):
- argparse_app.POSIX = True
- out = run_cmd(argparse_app, 'tag strong this should be loud')
- assert out == ['<strong>this should be loud</strong>']
-
-def test_argparse_quoted_arguments_posix_multiple(argparse_app):
- argparse_app.POSIX = True
- out = run_cmd(argparse_app, 'tag strong this "should be" loud')
- assert out == ['<strong>this should be loud</strong>']
-
def test_argparse_help_docstring(argparse_app):
out = run_cmd(argparse_app, 'help say')
assert out[0].startswith('usage: say')
diff --git a/tests/test_cmd2.py b/tests/test_cmd2.py
index c8955497..b570ad3c 100644
--- a/tests/test_cmd2.py
+++ b/tests/test_cmd2.py
@@ -104,16 +104,53 @@ def test_base_show_readonly(base_app):
Commands may be terminated with: {}
Arguments at invocation allowed: {}
Output redirection and pipes allowed: {}
- Parsing of command arguments:
- Shell lexer mode for command argument splitting: {}
- Strip Quotes after splitting arguments: {}
-
-""".format(base_app.terminators, base_app.allow_cli_args, base_app.allow_redirection,
- "POSIX" if cmd2.POSIX_SHLEX else "non-POSIX",
- "True" if cmd2.STRIP_QUOTES_FOR_NON_POSIX and not cmd2.POSIX_SHLEX else "False"))
+""".format(base_app.terminators, base_app.allow_cli_args, base_app.allow_redirection))
assert out == expected
+def test_cast():
+ cast = cmd2.cast
+
+ # Boolean
+ assert cast(True, True) == True
+ assert cast(True, False) == False
+ assert cast(True, 0) == False
+ assert cast(True, 1) == True
+ assert cast(True, 'on') == True
+ assert cast(True, 'off') == False
+ assert cast(True, 'ON') == True
+ assert cast(True, 'OFF') == False
+ assert cast(True, 'y') == True
+ assert cast(True, 'n') == False
+ assert cast(True, 't') == True
+ assert cast(True, 'f') == False
+
+ # Non-boolean same type
+ assert cast(1, 5) == 5
+ assert cast(3.4, 2.7) == 2.7
+ assert cast('foo', 'bar') == 'bar'
+ assert cast([1,2], [3,4]) == [3,4]
+
+def test_cast_problems(capsys):
+ cast = cmd2.cast
+
+ expected = 'Problem setting parameter (now {}) to {}; incorrect type?\n'
+
+ # Boolean current, with new value not convertible to bool
+ current = True
+ new = [True, True]
+ assert cast(current, new) == current
+ out, err = capsys.readouterr()
+ assert out == expected.format(current, new)
+
+ # Non-boolean current, with new value not convertible to current type
+ current = 1
+ new = 'octopus'
+ assert cast(current, new) == current
+ out, err = capsys.readouterr()
+ assert out == expected.format(current, new)
+
+
def test_base_set(base_app):
out = run_cmd(base_app, 'set quiet True')
expected = normalize("""
@@ -129,7 +166,7 @@ def test_set_not_supported(base_app, capsys):
run_cmd(base_app, 'set qqq True')
out, err = capsys.readouterr()
expected = normalize("""
-EXCEPTION of type 'LookupError' occurred with message: 'Parameter 'qqq' not supported (type 'show' for list of parameters).'
+EXCEPTION of type 'LookupError' occurred with message: 'Parameter 'qqq' not supported (type 'set' for list of parameters).'
To enable full traceback, run the following command: 'set debug true'
""")
assert normalize(str(err)) == expected
@@ -217,6 +254,34 @@ def test_base_error(base_app):
assert out == ["*** Unknown syntax: meow"]
+@pytest.fixture
+def hist():
+ from cmd2.cmd2 import History, HistoryItem
+ h = History([HistoryItem('first'), HistoryItem('second'), HistoryItem('third'), HistoryItem('fourth')])
+ return h
+
+def test_history_span(hist):
+ h = hist
+ assert h == ['first', 'second', 'third', 'fourth']
+ assert h.span('-2..') == ['third', 'fourth']
+ assert h.span('2..3') == ['second', 'third'] # Inclusive of end
+ assert h.span('3') == ['third']
+ assert h.span(':') == h
+ assert h.span('2..') == ['second', 'third', 'fourth']
+ assert h.span('-1') == ['fourth']
+ assert h.span('-2..-3') == ['third', 'second']
+ assert h.span('*') == h
+
+def test_history_get(hist):
+ h = hist
+ assert h == ['first', 'second', 'third', 'fourth']
+ assert h.get('') == h
+ assert h.get('-2') == h[:-2]
+ assert h.get('5') == []
+ assert h.get('2-3') == ['second'] # Exclusive of end
+ assert h.get('ir') == ['first', 'third'] # Normal string search for all elements containing "ir"
+ assert h.get('/i.*d/') == ['third'] # Regex string search "i", then anything, then "d"
+
def test_base_history(base_app):
run_cmd(base_app, 'help')
run_cmd(base_app, 'shortcuts')
@@ -604,18 +669,6 @@ def test_allow_redirection(base_app):
# Verify that no file got created
assert not os.path.exists(filename)
-
-def test_input_redirection(base_app, request):
- test_dir = os.path.dirname(request.module.__file__)
- filename = os.path.join(test_dir, 'redirect.txt')
-
- # NOTE: File 'redirect.txt" contains 1 word "history"
-
- # Verify that redirecting input ffom a file works
- out = run_cmd(base_app, 'help < {}'.format(filename))
- assert out == normalize(HELP_HISTORY)
-
-
def test_pipe_to_shell(base_app, capsys):
if sys.platform == "win32":
# Windows
@@ -910,6 +963,7 @@ class SayApp(cmd2.Cmd):
@pytest.fixture
def say_app():
app = SayApp()
+ app.allow_cli_args = False
app.stdout = StdOut()
return app
@@ -965,7 +1019,7 @@ def test_default_to_shell_good(capsys):
line = 'dir'
else:
line = 'ls'
- statement = app.parser_manager.parsed(line)
+ statement = app.statement_parser.parse(line)
retval = app.default(statement)
assert not retval
out, err = capsys.readouterr()
@@ -975,7 +1029,7 @@ def test_default_to_shell_failure(capsys):
app = cmd2.Cmd()
app.default_to_shell = True
line = 'ls does_not_exist.xyz'
- statement = app.parser_manager.parsed(line)
+ statement = app.statement_parser.parse(line)
retval = app.default(statement)
assert not retval
out, err = capsys.readouterr()
@@ -1329,7 +1383,7 @@ def test_which_editor_bad():
class MultilineApp(cmd2.Cmd):
def __init__(self, *args, **kwargs):
- self.multilineCommands = ['orate']
+ self.multiline_commands = ['orate']
super().__init__(*args, **kwargs)
orate_parser = argparse.ArgumentParser()
@@ -1362,7 +1416,7 @@ def test_multiline_complete_statement_without_terminator(multiline_app):
line = '{} {}'.format(command, args)
statement = multiline_app._complete_statement(line)
assert statement == args
- assert statement.parsed.command == command
+ assert statement.command == command
def test_clipboard_failure(capsys):
diff --git a/tests/test_completion.py b/tests/test_completion.py
index 7026db48..a027d780 100644
--- a/tests/test_completion.py
+++ b/tests/test_completion.py
@@ -582,44 +582,6 @@ def test_tokens_for_completion_redirect_off(cmd2_app):
assert expected_tokens == tokens
assert expected_raw_tokens == raw_tokens
-def test_parseline_command_and_args(cmd2_app):
- line = 'help history'
- command, args, out_line = cmd2_app.parseline(line)
- assert command == 'help'
- assert args == 'history'
- assert line == out_line
-
-def test_parseline_emptyline(cmd2_app):
- line = ''
- command, args, out_line = cmd2_app.parseline(line)
- assert command is None
- assert args is None
- assert line is out_line
-
-def test_parseline_strips_line(cmd2_app):
- line = ' help history '
- command, args, out_line = cmd2_app.parseline(line)
- assert command == 'help'
- assert args == 'history'
- assert line.strip() == out_line
-
-def test_parseline_expands_alias(cmd2_app):
- # Create the alias
- cmd2_app.do_alias(['fake', 'pyscript'])
-
- line = 'fake foobar.py'
- command, args, out_line = cmd2_app.parseline(line)
- assert command == 'pyscript'
- assert args == 'foobar.py'
- assert line.replace('fake', 'pyscript') == out_line
-
-def test_parseline_expands_shortcuts(cmd2_app):
- line = '!cat foobar.txt'
- command, args, out_line = cmd2_app.parseline(line)
- assert command == 'shell'
- assert args == 'cat foobar.txt'
- assert line.replace('!', 'shell ') == out_line
-
def test_add_opening_quote_basic_no_text(cmd2_app):
text = ''
line = 'test_basic {}'.format(text)
diff --git a/tests/test_parsing.py b/tests/test_parsing.py
index b61e2d06..ab8ed098 100644
--- a/tests/test_parsing.py
+++ b/tests/test_parsing.py
@@ -1,327 +1,262 @@
# coding=utf-8
"""
-Unit/functional testing for helper functions/classes in the cmd2.py module.
-
-These are primarily tests related to parsing. Moreover, they are mostly a port of the old doctest tests which were
-problematic because they worked properly for some versions of pyparsing but not for others.
+Test the parsing logic in parsing.py
Copyright 2017 Todd Leonhardt <todd.leonhardt@gmail.com>
Released under MIT license, see LICENSE file
"""
-from cmd2 import cmd2
import pytest
+from cmd2 import cmd2
+from cmd2.parsing import StatementParser
-@pytest.fixture
-def hist():
- from cmd2.cmd2 import HistoryItem
- h = cmd2.History([HistoryItem('first'), HistoryItem('second'), HistoryItem('third'), HistoryItem('fourth')])
- return h
-# Case-sensitive parser
@pytest.fixture
def parser():
- c = cmd2.Cmd()
- c.multilineCommands = ['multiline']
- c.parser_manager = cmd2.ParserManager(redirector=c.redirector, terminators=c.terminators,
- multilineCommands=c.multilineCommands, legalChars=c.legalChars,
- commentGrammars=c.commentGrammars, commentInProgress=c.commentInProgress,
- blankLinesAllowed=c.blankLinesAllowed, prefixParser=c.prefixParser,
- preparse=c.preparse, postparse=c.postparse, aliases=c.aliases,
- shortcuts=c.shortcuts)
- return c.parser_manager.main_parser
-
-# Case-sensitive ParserManager
-@pytest.fixture
-def cs_pm():
- c = cmd2.Cmd()
- c.multilineCommands = ['multiline']
- c.parser_manager = cmd2.ParserManager(redirector=c.redirector, terminators=c.terminators,
- multilineCommands=c.multilineCommands, legalChars=c.legalChars,
- commentGrammars=c.commentGrammars, commentInProgress=c.commentInProgress,
- blankLinesAllowed=c.blankLinesAllowed, prefixParser=c.prefixParser,
- preparse=c.preparse, postparse=c.postparse, aliases=c.aliases,
- shortcuts=c.shortcuts)
- return c.parser_manager
-
-
-@pytest.fixture
-def input_parser():
- c = cmd2.Cmd()
- return c.parser_manager.input_source_parser
-
-
-def test_history_span(hist):
- h = hist
- assert h == ['first', 'second', 'third', 'fourth']
- assert h.span('-2..') == ['third', 'fourth']
- assert h.span('2..3') == ['second', 'third'] # Inclusive of end
- assert h.span('3') == ['third']
- assert h.span(':') == h
- assert h.span('2..') == ['second', 'third', 'fourth']
- assert h.span('-1') == ['fourth']
- assert h.span('-2..-3') == ['third', 'second']
- assert h.span('*') == h
-
-def test_history_get(hist):
- h = hist
- assert h == ['first', 'second', 'third', 'fourth']
- assert h.get('') == h
- assert h.get('-2') == h[:-2]
- assert h.get('5') == []
- assert h.get('2-3') == ['second'] # Exclusive of end
- assert h.get('ir') == ['first', 'third'] # Normal string search for all elements containing "ir"
- assert h.get('/i.*d/') == ['third'] # Regex string search "i", then anything, then "d"
-
-
-def test_cast():
- cast = cmd2.cast
-
- # Boolean
- assert cast(True, True) == True
- assert cast(True, False) == False
- assert cast(True, 0) == False
- assert cast(True, 1) == True
- assert cast(True, 'on') == True
- assert cast(True, 'off') == False
- assert cast(True, 'ON') == True
- assert cast(True, 'OFF') == False
- assert cast(True, 'y') == True
- assert cast(True, 'n') == False
- assert cast(True, 't') == True
- assert cast(True, 'f') == False
-
- # Non-boolean same type
- assert cast(1, 5) == 5
- assert cast(3.4, 2.7) == 2.7
- assert cast('foo', 'bar') == 'bar'
- assert cast([1,2], [3,4]) == [3,4]
-
-
-def test_cast_problems(capsys):
- cast = cmd2.cast
-
- expected = 'Problem setting parameter (now {}) to {}; incorrect type?\n'
-
- # Boolean current, with new value not convertible to bool
- current = True
- new = [True, True]
- assert cast(current, new) == current
- out, err = capsys.readouterr()
- assert out == expected.format(current, new)
-
- # Non-boolean current, with new value not convertible to current type
- current = 1
- new = 'octopus'
- assert cast(current, new) == current
- out, err = capsys.readouterr()
- assert out == expected.format(current, new)
-
+ parser = StatementParser(
+ allow_redirection=True,
+ terminators=[';'],
+ multiline_commands=['multiline'],
+ aliases={'helpalias': 'help',
+ '42': 'theanswer',
+ 'l': '!ls -al',
+ 'anothermultiline': 'multiline',
+ 'fake': 'pyscript'},
+ shortcuts=[('?', 'help'), ('!', 'shell')]
+ )
+ return parser
def test_parse_empty_string(parser):
- assert parser.parseString('').dump() == '[]'
-
-def test_parse_only_comment(parser):
- assert parser.parseString('/* empty command */').dump() == '[]'
-
-def test_parse_single_word(parser):
- line = 'plainword'
- results = parser.parseString(line)
- assert results.command == line
-
-def test_parse_word_plus_terminator(parser):
+ statement = parser.parse('')
+ assert not statement.command
+ assert not statement.args
+ assert statement.raw == ''
+
+@pytest.mark.parametrize('line,tokens', [
+ ('command', ['command']),
+ ('command /* with some comment */ arg', ['command', 'arg']),
+ ('command arg1 arg2 # comment at the end', ['command', 'arg1', 'arg2']),
+ ('42 arg1 arg2', ['theanswer', 'arg1', 'arg2']),
+ ('l', ['shell', 'ls', '-al'])
+])
+def test_tokenize(parser, line, tokens):
+ tokens_to_test = parser.tokenize(line)
+ assert tokens_to_test == tokens
+
+@pytest.mark.parametrize('tokens,command,args', [
+ ([], None, ''),
+ (['command'], 'command', ''),
+ (['command', 'arg1', 'arg2'], 'command', 'arg1 arg2')
+])
+def test_command_and_args(parser, tokens, command, args):
+ (parsed_command, parsed_args) = parser._command_and_args(tokens)
+ assert command == parsed_command
+ assert args == parsed_args
+
+@pytest.mark.parametrize('line', [
+ 'plainword',
+ '"one word"',
+ "'one word'",
+])
+def test_single_word(parser, line):
+ statement = parser.parse(line)
+ assert statement.command == line
+
+def test_word_plus_terminator(parser):
line = 'termbare;'
- results = parser.parseString(line)
- assert results.command == 'termbare'
- assert results.terminator == ';'
+ statement = parser.parse(line)
+ assert statement.command == 'termbare'
+ assert statement.terminator == ';'
-def test_parse_suffix_after_terminator(parser):
+def test_suffix_after_terminator(parser):
line = 'termbare; suffx'
- results = parser.parseString(line)
- assert results.command == 'termbare'
- assert results.terminator == ';'
- assert results.suffix == 'suffx'
+ statement = parser.parse(line)
+ assert statement.command == 'termbare'
+ assert statement.terminator == ';'
+ assert statement.suffix == 'suffx'
-def test_parse_command_with_args(parser):
+def test_command_with_args(parser):
line = 'command with args'
- results = parser.parseString(line)
- assert results.command == 'command'
- assert results.args == 'with args'
+ statement = parser.parse(line)
+ assert statement.command == 'command'
+ assert statement.args == 'with args'
+ assert not statement.pipe_to
+
+def test_command_with_quoted_args(parser):
+ line = 'command with "quoted args" and "some not"'
+ statement = parser.parse(line)
+ assert statement.command == 'command'
+ assert statement.args == 'with "quoted args" and "some not"'
def test_parse_command_with_args_terminator_and_suffix(parser):
line = 'command with args and terminator; and suffix'
- results = parser.parseString(line)
- assert results.command == 'command'
- assert results.args == "with args and terminator"
- assert results.terminator == ';'
- assert results.suffix == 'and suffix'
-
-def test_parse_simple_piped(parser):
- line = 'simple | piped'
- results = parser.parseString(line)
- assert results.command == 'simple'
- assert results.pipeTo == " piped"
-
-def test_parse_double_pipe_is_not_a_pipe(parser):
+ statement = parser.parse(line)
+ assert statement.command == 'command'
+ assert statement.args == "with args and terminator"
+ assert statement.terminator == ';'
+ assert statement.suffix == 'and suffix'
+
+def test_hashcomment(parser):
+ statement = parser.parse('hi # this is all a comment')
+ assert statement.command == 'hi'
+ assert not statement.args
+ assert not statement.pipe_to
+
+def test_c_comment(parser):
+ statement = parser.parse('hi /* this is | all a comment */')
+ assert statement.command == 'hi'
+ assert not statement.args
+ assert not statement.pipe_to
+
+def test_c_comment_empty(parser):
+ statement = parser.parse('/* this is | all a comment */')
+ assert not statement.command
+ assert not statement.args
+ assert not statement.pipe_to
+
+def test_parse_what_if_quoted_strings_seem_to_start_comments(parser):
+ statement = parser.parse('what if "quoted strings /* seem to " start comments?')
+ assert statement.command == 'what'
+ assert statement.args == 'if "quoted strings /* seem to " start comments?'
+ assert not statement.pipe_to
+
+def test_simple_piped(parser):
+ statement = parser.parse('simple | piped')
+ assert statement.command == 'simple'
+ assert not statement.args
+ assert statement.pipe_to == 'piped'
+
+def test_double_pipe_is_not_a_pipe(parser):
line = 'double-pipe || is not a pipe'
- results = parser.parseString(line)
- assert results.command == 'double-pipe'
- assert results.args == '|| is not a pipe'
- assert not 'pipeTo' in results
+ statement = parser.parse(line)
+ assert statement.command == 'double-pipe'
+ assert statement.args == '|| is not a pipe'
+ assert not statement.pipe_to
-def test_parse_complex_pipe(parser):
+def test_complex_pipe(parser):
line = 'command with args, terminator;sufx | piped'
- results = parser.parseString(line)
- assert results.command == 'command'
- assert results.args == "with args, terminator"
- assert results.terminator == ';'
- assert results.suffix == 'sufx'
- assert results.pipeTo == ' piped'
-
-def test_parse_output_redirect(parser):
+ statement = parser.parse(line)
+ assert statement.command == 'command'
+ assert statement.args == "with args, terminator"
+ assert statement.terminator == ';'
+ assert statement.suffix == 'sufx'
+ assert statement.pipe_to == 'piped'
+
+def test_output_redirect(parser):
line = 'output into > afile.txt'
- results = parser.parseString(line)
- assert results.command == 'output'
- assert results.args == 'into'
- assert results.output == '>'
- assert results.outputTo == 'afile.txt'
+ statement = parser.parse(line)
+ assert statement.command == 'output'
+ assert statement.args == 'into'
+ assert statement.output == '>'
+ assert statement.output_to == 'afile.txt'
-def test_parse_output_redirect_with_dash_in_path(parser):
+def test_output_redirect_with_dash_in_path(parser):
line = 'output into > python-cmd2/afile.txt'
- results = parser.parseString(line)
- assert results.command == 'output'
- assert results.args == 'into'
- assert results.output == '>'
- assert results.outputTo == 'python-cmd2/afile.txt'
-
-
-def test_case_sensitive_parsed_single_word(cs_pm):
- line = 'HeLp'
- statement = cs_pm.parsed(line)
- assert statement.parsed.command == line
-
-
-def test_parse_input_redirect(input_parser):
- line = '< afile.txt'
- results = input_parser.parseString(line)
- assert results.inputFrom == line
-
-def test_parse_input_redirect_with_dash_in_path(input_parser):
- line = "< python-cmd2/afile.txt"
- results = input_parser.parseString(line)
- assert results.inputFrom == line
-
-def test_parse_pipe_and_redirect(parser):
+ statement = parser.parse(line)
+ assert statement.command == 'output'
+ assert statement.args == 'into'
+ assert statement.output == '>'
+ assert statement.output_to == 'python-cmd2/afile.txt'
+
+def test_output_redirect_append(parser):
+ line = 'output appended to >> /tmp/afile.txt'
+ statement = parser.parse(line)
+ assert statement.command == 'output'
+ assert statement.args == 'appended to'
+ assert statement.output == '>>'
+ assert statement.output_to == '/tmp/afile.txt'
+
+def test_pipe_and_redirect(parser):
line = 'output into;sufx | pipethrume plz > afile.txt'
- results = parser.parseString(line)
- assert results.command == 'output'
- assert results.args == 'into'
- assert results.terminator == ';'
- assert results.suffix == 'sufx'
- assert results.pipeTo == ' pipethrume plz'
- assert results.output == '>'
- assert results.outputTo == 'afile.txt'
+ statement = parser.parse(line)
+ assert statement.command == 'output'
+ assert statement.args == 'into'
+ assert statement.terminator == ';'
+ assert statement.suffix == 'sufx'
+ assert statement.pipe_to == 'pipethrume plz'
+ assert statement.output == '>'
+ assert statement.output_to == 'afile.txt'
def test_parse_output_to_paste_buffer(parser):
line = 'output to paste buffer >> '
- results = parser.parseString(line)
- assert results.command == 'output'
- assert results.args == 'to paste buffer'
- assert results.output == '>>'
-
-def test_parse_ignore_commented_redirectors(parser):
- line = 'ignore the /* commented | > */ stuff;'
- results = parser.parseString(line)
- assert results.command == 'ignore'
- assert results.args == 'the /* commented | > */ stuff'
- assert results.terminator == ';'
-
-def test_parse_has_redirect_inside_terminator(parser):
+ statement = parser.parse(line)
+ assert statement.command == 'output'
+ assert statement.args == 'to paste buffer'
+ assert statement.output == '>>'
+
+def test_has_redirect_inside_terminator(parser):
"""The terminator designates the end of the commmand/arguments portion. If a redirector
occurs before a terminator, then it will be treated as part of the arguments and not as a redirector."""
line = 'has > inside;'
- results = parser.parseString(line)
- assert results.command == 'has'
- assert results.args == '> inside'
- assert results.terminator == ';'
-
-def test_parse_what_if_quoted_strings_seem_to_start_comments(parser):
- line = 'what if "quoted strings /* seem to " start comments?'
- results = parser.parseString(line)
- assert results.command == 'what'
- assert results.args == 'if "quoted strings /* seem to " start comments?'
+ statement = parser.parse(line)
+ assert statement.command == 'has'
+ assert statement.args == '> inside'
+ assert statement.terminator == ';'
def test_parse_unfinished_multiliine_command(parser):
line = 'multiline has > inside an unfinished command'
- results = parser.parseString(line)
- assert results.multilineCommand == 'multiline'
- assert not 'args' in results
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.command == 'multiline'
+ assert statement.args == 'has > inside an unfinished command'
+ assert not statement.terminator
def test_parse_multiline_command_ignores_redirectors_within_it(parser):
line = 'multiline has > inside;'
- results = parser.parseString(line)
- assert results.multilineCommand == 'multiline'
- assert results.args == 'has > inside'
- assert results.terminator == ';'
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.args == 'has > inside'
+ assert statement.terminator == ';'
def test_parse_multiline_with_incomplete_comment(parser):
"""A terminator within a comment will be ignored and won't terminate a multiline command.
Un-closed comments effectively comment out everything after the start."""
line = 'multiline command /* with comment in progress;'
- results = parser.parseString(line)
- assert results.multilineCommand == 'multiline'
- assert not 'args' in results
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.args == 'command'
+ assert not statement.terminator
def test_parse_multiline_with_complete_comment(parser):
line = 'multiline command /* with comment complete */ is done;'
- results = parser.parseString(line)
- assert results.multilineCommand == 'multiline'
- assert results.args == 'command /* with comment complete */ is done'
- assert results.terminator == ';'
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.args == 'command is done'
+ assert statement.terminator == ';'
def test_parse_multiline_termninated_by_empty_line(parser):
line = 'multiline command ends\n\n'
- results = parser.parseString(line)
- assert results.multilineCommand == 'multiline'
- assert results.args == 'command ends'
- assert len(results.terminator) == 2
- assert results.terminator[0] == '\n'
- assert results.terminator[1] == '\n'
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.args == 'command ends'
+ assert statement.terminator == '\n'
def test_parse_multiline_ignores_terminators_in_comments(parser):
line = 'multiline command "with term; ends" now\n\n'
- results = parser.parseString(line)
- assert results.multilineCommand == 'multiline'
- assert results.args == 'command "with term; ends" now'
- assert len(results.terminator) == 2
- assert results.terminator[0] == '\n'
- assert results.terminator[1] == '\n'
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.args == 'command "with term; ends" now'
+ assert statement.terminator == '\n'
def test_parse_command_with_unicode_args(parser):
line = 'drink café'
- results = parser.parseString(line)
- assert results.command == 'drink'
- assert results.args == 'café'
+ statement = parser.parse(line)
+ assert statement.command == 'drink'
+ assert statement.args == 'café'
def test_parse_unicode_command(parser):
line = 'café au lait'
- results = parser.parseString(line)
- assert results.command == 'café'
- assert results.args == 'au lait'
+ statement = parser.parse(line)
+ assert statement.command == 'café'
+ assert statement.args == 'au lait'
def test_parse_redirect_to_unicode_filename(parser):
line = 'dir home > café'
- results = parser.parseString(line)
- assert results.command == 'dir'
- assert results.args == 'home'
- assert results.output == '>'
- assert results.outputTo == 'café'
-
-def test_parse_input_redirect_from_unicode_filename(input_parser):
- line = '< café'
- results = input_parser.parseString(line)
- assert results.inputFrom == line
-
+ statement = parser.parse(line)
+ assert statement.command == 'dir'
+ assert statement.args == 'home'
+ assert statement.output == '>'
+ assert statement.output_to == 'café'
def test_empty_statement_raises_exception():
app = cmd2.Cmd()
@@ -330,3 +265,66 @@ def test_empty_statement_raises_exception():
with pytest.raises(cmd2.EmptyStatement):
app._complete_statement(' ')
+
+@pytest.mark.parametrize('line,command,args', [
+ ('helpalias', 'help', ''),
+ ('helpalias mycommand', 'help', 'mycommand'),
+ ('42', 'theanswer', ''),
+ ('42 arg1 arg2', 'theanswer', 'arg1 arg2'),
+ ('!ls', 'shell', 'ls'),
+ ('!ls -al /tmp', 'shell', 'ls -al /tmp'),
+ ('l', 'shell', 'ls -al')
+])
+def test_alias_and_shortcut_expansion(parser, line, command, args):
+ statement = parser.parse(line)
+ assert statement.command == command
+ assert statement.args == args
+
+def test_alias_on_multiline_command(parser):
+ line = 'anothermultiline has > inside an unfinished command'
+ statement = parser.parse(line)
+ assert statement.multiline_command == 'multiline'
+ assert statement.command == 'multiline'
+ assert statement.args == 'has > inside an unfinished command'
+ assert not statement.terminator
+
+def test_parse_command_only_command_and_args(parser):
+ line = 'help history'
+ statement = parser.parse_command_only(line)
+ assert statement.command == 'help'
+ assert statement.args == 'history'
+ assert statement.command_and_args == line
+
+def test_parse_command_only_emptyline(parser):
+ line = ''
+ statement = parser.parse_command_only(line)
+ assert statement.command is None
+ assert statement.args is ''
+ assert statement.command_and_args is line
+
+def test_parse_command_only_strips_line(parser):
+ line = ' help history '
+ statement = parser.parse_command_only(line)
+ assert statement.command == 'help'
+ assert statement.args == 'history'
+ assert statement.command_and_args == line.strip()
+
+def test_parse_command_only_expands_alias(parser):
+ line = 'fake foobar.py'
+ statement = parser.parse_command_only(line)
+ assert statement.command == 'pyscript'
+ assert statement.args == 'foobar.py'
+
+def test_parse_command_only_expands_shortcuts(parser):
+ line = '!cat foobar.txt'
+ statement = parser.parse_command_only(line)
+ assert statement.command == 'shell'
+ assert statement.args == 'cat foobar.txt'
+ assert statement.command_and_args == line.replace('!', 'shell ')
+
+def test_parse_command_only_quoted_args(parser):
+ line = 'l "/tmp/directory with spaces/doit.sh"'
+ statement = parser.parse_command_only(line)
+ assert statement.command == 'shell'
+ assert statement.args == 'ls -al "/tmp/directory with spaces/doit.sh"'
+ assert statement.command_and_args == line.replace('l', 'shell ls -al')
diff --git a/tests/test_transcript.py b/tests/test_transcript.py
index 4f821c06..c0fb49c1 100644
--- a/tests/test_transcript.py
+++ b/tests/test_transcript.py
@@ -15,7 +15,6 @@ from unittest import mock
import pytest
from cmd2 import cmd2
-from cmd2.cmd2 import set_posix_shlex, set_strip_quotes
from .conftest import run_cmd, StdOut, normalize
class CmdLineApp(cmd2.Cmd):
@@ -25,7 +24,7 @@ class CmdLineApp(cmd2.Cmd):
MUMBLE_LAST = ['right?']
def __init__(self, *args, **kwargs):
- self.multilineCommands = ['orate']
+ self.multiline_commands = ['orate']
self.maxrepeats = 3
self.redirector = '->'
@@ -35,10 +34,6 @@ class CmdLineApp(cmd2.Cmd):
super().__init__(*args, **kwargs)
self.intro = 'This is an intro banner ...'
- # Configure how arguments are parsed for commands using decorators
- set_posix_shlex(False)
- set_strip_quotes(True)
-
speak_parser = argparse.ArgumentParser()
speak_parser.add_argument('-p', '--piglatin', action="store_true", help="atinLay")
speak_parser.add_argument('-s', '--shout', action="store_true", help="N00B EMULATION MODE")
@@ -130,7 +125,7 @@ def test_base_with_transcript(_cmdline_app):
Documented commands (type help <topic>):
========================================
-alias help load orate pyscript say shell speak
+alias help load orate pyscript say shell speak
edit history mumble py quit set shortcuts unalias
(Cmd) help say
diff --git a/tox.ini b/tox.ini
index 6749418b..e74ce16f 100644
--- a/tox.ini
+++ b/tox.ini
@@ -12,72 +12,56 @@ setenv =
[testenv:py34]
deps =
codecov
- pyparsing
pyperclip
pytest
pytest-cov
- pytest-forked
- pytest-xdist
wcwidth
commands =
- py.test {posargs: -n 2} --cov --cov-report=term-missing --forked
+ py.test {posargs} --cov
codecov
[testenv:py35]
deps =
mock
- pyparsing
pyperclip
pytest
- pytest-forked
- pytest-xdist
wcwidth
-commands = py.test -v -n2 --forked
+commands = py.test -v
[testenv:py35-win]
deps =
mock
- pyparsing
pyperclip
pyreadline
pytest
- pytest-xdist
-commands = py.test -v -n2
+commands = py.test -v
[testenv:py36]
deps =
codecov
- pyparsing
pyperclip
pytest
pytest-cov
- pytest-forked
- pytest-xdist
wcwidth
commands =
- py.test {posargs: -n 2} --cov --cov-report=term-missing --forked
+ py.test {posargs} --cov
codecov
[testenv:py36-win]
deps =
codecov
- pyparsing
pyperclip
pyreadline
pytest
pytest-cov
- pytest-xdist
commands =
- py.test {posargs: -n 2} --cov --cov-report=term-missing
+ py.test {posargs} --cov
codecov
[testenv:py37]
deps =
- pyparsing
pyperclip
pytest
- pytest-forked
- pytest-xdist
wcwidth
-commands = py.test -v -n2 --forked
+commands = py.test -v
View Lorry Controller Status