summaryrefslogtreecommitdiff
path: root/sphinx/pycode/pgen2/tokenize.py
Commit message (Collapse)AuthorAgeFilesLines
* Fix: pgen2 tokenizer doesn't recognize `...` literal (Ellipsis for py3). ↵shimizukawa2014-08-281-2/+9
| | | | | | Closes #1547 I think pgen2 code derived from lib2to3 package. Basically, the package only support python2 code then it doesn't recognize `...` literal.
* wrap py3 iterators with list() for each places that expect a list object. ↵shimizukawa2014-04-301-2/+3
| | | | refs #1350.
* consistency nitsGeorg Brandl2014-01-221-1/+2
|
* Modernize the code now that Python 2.5 is no longer supportedDmitry Shachnev2014-01-191-2/+4
| | | | | | | | | | | - Use print function instead of print statement; - Use new exception handling; - Use in operator instead of has_key(); - Do not use tuple arguments in functions; - Other miscellaneous improvements. This is based on output of `futurize --stage1`, with some manual corrections.
* Fixed issue #1DasIch2010-05-011-1/+3
|
* merge in Ben's bundle with more py3k compatibilityGeorg Brandl2009-01-101-0/+5
|\
| * Changed the tokenizer in pgen2 to add a newline to lines even if it's ↵mitsuhiko2009-01-091-0/+5
| | | | | | | | missing. This makes sure that a file that does not end with a newline is properly parsed.
* | normalize raise statementsBenjamin Peterson2009-01-101-2/+2
|/
* Add pgen2 and custom utilities.Georg Brandl2008-12-291-0/+405