<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/cpython-git.git/Python/ast_opt.c, branch Mariatta-patch-2</title>
<subtitle>github.com: python/cpython.git
</subtitle>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/'/>
<entry>
<title>bpo-38605: Make 'from __future__ import annotations' the default (GH-20434)</title>
<updated>2020-10-06T20:03:02+00:00</updated>
<author>
<name>Batuhan Taskaya</name>
<email>batuhanosmantaskaya@gmail.com</email>
</author>
<published>2020-10-06T20:03:02+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=044a1048ca93d466965afc027b91a5a9eb9ce23c'/>
<id>044a1048ca93d466965afc027b91a5a9eb9ce23c</id>
<content type='text'>
The hard part was making all the tests pass; there are some subtle issues here, because apparently the future import wasn't tested very thoroughly in previous Python versions.

For example, `inspect.signature()` returned type objects normally (except for forward references), but strings with the future import. We changed it to try and return type objects by calling `typing.get_type_hints()`, but fall back on returning strings if that function fails (which it may do if there are future references in the annotations that require passing in a specific namespace to resolve).</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The hard part was making all the tests pass; there are some subtle issues here, because apparently the future import wasn't tested very thoroughly in previous Python versions.

For example, `inspect.signature()` returned type objects normally (except for forward references), but strings with the future import. We changed it to try and return type objects by calling `typing.get_type_hints()`, but fall back on returning strings if that function fails (which it may do if there are future references in the annotations that require passing in a specific namespace to resolve).</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-41746: Add type information to asdl_seq objects (GH-22223)</title>
<updated>2020-09-16T18:42:00+00:00</updated>
<author>
<name>Pablo Galindo</name>
<email>Pablogsal@gmail.com</email>
</author>
<published>2020-09-16T18:42:00+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=a5634c406767ef694df49b624adf9cfa6c0d9064'/>
<id>a5634c406767ef694df49b624adf9cfa6c0d9064</id>
<content type='text'>
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)</title>
<updated>2020-04-22T22:29:27+00:00</updated>
<author>
<name>Pablo Galindo</name>
<email>Pablogsal@gmail.com</email>
</author>
<published>2020-04-22T22:29:27+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=c5fc15685202cda73f7c3f5c6f299b0945f58508'/>
<id>c5fc15685202cda73f7c3f5c6f299b0945f58508</id>
<content type='text'>
Co-authored-by: Guido van Rossum &lt;guido@python.org&gt;
Co-authored-by: Lysandros Nikolaou &lt;lisandrosnik@gmail.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Co-authored-by: Guido van Rossum &lt;guido@python.org&gt;
Co-authored-by: Lysandros Nikolaou &lt;lisandrosnik@gmail.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-39522: Always initialise kind attribute in constant ast nodes (GH-19525)</title>
<updated>2020-04-14T20:40:41+00:00</updated>
<author>
<name>Pablo Galindo</name>
<email>Pablogsal@gmail.com</email>
</author>
<published>2020-04-14T20:40:41+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=33986465bde2a2188537c4ef6cdb6055e348f31f'/>
<id>33986465bde2a2188537c4ef6cdb6055e348f31f</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-39220: Do not optimise annotation if 'from __future__ import annotations' is used (GH-17866)</title>
<updated>2020-03-18T23:02:09+00:00</updated>
<author>
<name>Pablo Galindo</name>
<email>Pablogsal@gmail.com</email>
</author>
<published>2020-03-18T23:02:09+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=d112c600ab3f5e2910345bcc4bfc39439917ed87'/>
<id>d112c600ab3f5e2910345bcc4bfc39439917ed87</id>
<content type='text'>
Do not apply AST-based optimizations if 'from __future__ import annotations' is used in order to
prevent information lost in the final version of the annotations.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Do not apply AST-based optimizations if 'from __future__ import annotations' is used in order to
prevent information lost in the final version of the annotations.</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-34822: Simplify AST for subscription. (GH-9605)</title>
<updated>2020-03-10T16:52:34+00:00</updated>
<author>
<name>Serhiy Storchaka</name>
<email>storchaka@gmail.com</email>
</author>
<published>2020-03-10T16:52:34+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=13d52c268699f199a8e917a0f1dc4c51e5346c42'/>
<id>13d52c268699f199a8e917a0f1dc4c51e5346c42</id>
<content type='text'>
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.

All non-terminal nodes in AST for expressions are now of the expr type.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.

All non-terminal nodes in AST for expressions are now of the expr type.
</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-39639: Remove the AST "Suite" node and associated code (GH-18513)</title>
<updated>2020-03-04T16:16:46+00:00</updated>
<author>
<name>Batuhan Taşkaya</name>
<email>47358913+isidentical@users.noreply.github.com</email>
</author>
<published>2020-03-04T16:16:47+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=d82e469048e0e034d8c0020cd33b733be1adf68b'/>
<id>d82e469048e0e034d8c0020cd33b733be1adf68b</id>
<content type='text'>
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).

Co-Authored-By: Victor Stinner &lt;vstinner@python.org&gt;
Co-authored-by: Brett Cannon &lt;54418+brettcannon@users.noreply.github.com&gt;
Co-authored-by: Pablo Galindo &lt;Pablogsal@gmail.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).

Co-Authored-By: Victor Stinner &lt;vstinner@python.org&gt;
Co-authored-by: Brett Cannon &lt;54418+brettcannon@users.noreply.github.com&gt;
Co-authored-by: Pablo Galindo &lt;Pablogsal@gmail.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix constant folding optimization for positional only arguments (GH-17837)</title>
<updated>2020-01-05T17:03:56+00:00</updated>
<author>
<name>Anthony Sottile</name>
<email>asottile@umich.edu</email>
</author>
<published>2020-01-05T17:03:56+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=b121a4a45ff4bab8812a9b26ceffe5ad642f5d5a'/>
<id>b121a4a45ff4bab8812a9b26ceffe5ad642f5d5a</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-33416: Add end positions to Python AST (GH-11605)</title>
<updated>2019-01-22T11:18:22+00:00</updated>
<author>
<name>Ivan Levkivskyi</name>
<email>levkivskyi@gmail.com</email>
</author>
<published>2019-01-22T11:18:22+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=9932a22897ef9905161dac7476e6976370e13515'/>
<id>9932a22897ef9905161dac7476e6976370e13515</id>
<content type='text'>
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-35177: Add dependencies between header files (GH-10361)</title>
<updated>2018-11-11T23:56:19+00:00</updated>
<author>
<name>Victor Stinner</name>
<email>vstinner@redhat.com</email>
</author>
<published>2018-11-11T23:56:19+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/cpython-git.git/commit/?id=5f2df88b63e50d23914e97ec778861a52abdeaad'/>
<id>5f2df88b63e50d23914e97ec778861a52abdeaad</id>
<content type='text'>
* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:

  * Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
    twice
  * Add "extern { ... }" for C++
  * Undefine "Yield" macro conflicting with winbase.h

* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:

  * Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
    twice
  * Add "extern { ... }" for C++
  * Undefine "Yield" macro conflicting with winbase.h

* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files</pre>
</div>
</content>
</entry>
</feed>
