delta/cpython-git.git/Parser, branch revert-11318-fast-enum

bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-10370)

2018-12-22T09:18:40+00:00

"Include/token.h", "Lib/token.py" (containing now some data moved from
"Lib/tokenize.py") and new files "Parser/token.c" (containing the code
moved from "Parser/tokenizer.c") and "Doc/library/token-list.inc" (included
in "Doc/library/token.rst") are now generated from "Grammar/Tokens" by
"Tools/scripts/generate_token.py". The script overwrites files only if
needed and can be used on the read-only sources tree.

"Lib/symbol.py" is now generated by "Tools/scripts/generate_symbol_py.py"
instead of been executable itself.

Added new make targets "regen-token" and "regen-symbol" which are now
dependencies of "regen-all".

The documentation contains now strings for operators and punctuation tokens.

bpo-33306: Improve SyntaxError messages for unbalanced parentheses. (GH-6516)

2018-12-17T15:34:14+00:00

bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015)

2018-12-07T10:11:30+00:00

Set MemoryError when appropriate, add missing failure checks,
and fix some potential leaks.

bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)

2018-11-22T17:38:38+00:00

Partially revert commit 5f2df88b63e50d23914e97ec778861a52abdeaad:
add "#undef Yield" to .c files after including Python-ast.h.

Fix the warning:

    winbase.h(102): warning C4005: 'Yield': macro redefinition

bpo-35081: Rename internal headers (GH-10275)

2018-11-12T15:53:38+00:00

Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h

bpo-35177: Add dependencies between header files (GH-10361)

2018-11-11T23:56:19+00:00

* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:

  * Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
    twice
  * Add "extern { ... }" for C++
  * Undefine "Yield" macro conflicting with winbase.h

* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files

bpo-35081: Add _PyThreadState_GET() internal macro (GH-10266)

2018-11-01T00:51:40+00:00

If Py_BUILD_CORE is defined, the PyThreadState_GET() macro access
_PyRuntime which comes from the internal pycore_state.h header.
Public headers must not require internal headers.

Move PyThreadState_GET() and _PyInterpreterState_GET_UNSAFE() from
Include/pystate.h to Include/internal/pycore_state.h, and rename
PyThreadState_GET() to _PyThreadState_GET() there.

The PyThreadState_GET() macro of pystate.h is now redefined when
pycore_state.h is included, to use the fast _PyThreadState_GET().

Changes:

* Add _PyThreadState_GET() macro
* Replace "PyThreadState_GET()->interp" with
  _PyInterpreterState_GET_UNSAFE()
* Replace PyThreadState_GET() with _PyThreadState_GET() in internal C
  files (compiled with Py_BUILD_CORE defined), but keep
  PyThreadState_GET() in the public header files.
* _testcapimodule.c: replace PyThreadState_GET() with
  PyThreadState_Get(); the module is not compiled with Py_BUILD_CORE
  defined.
* pycore_state.h now requires Py_BUILD_CORE to be defined.

bpo-35081: Add pycore_ prefix to internal header files (GH-10263)

2018-10-31T23:52:28+00:00

* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".

bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445)

2018-09-27T14:42:37+00:00

bpo-34683: Make SyntaxError column offsets consistently 1-indexed (gh-9338)

2018-09-24T21:12:49+00:00

Also point to start of tokens in parsing errors.

Fixes bpo-34683