<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/pygments-git.git/tests/examplefiles/ragel-cpp, branch 2.12.0</title>
<subtitle>github.com: pygments/pygments.git
</subtitle>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/python-packages/pygments-git.git/'/>
<entry>
<title>CFamily: Lex identifiers after `case` as constants (#2084)</title>
<updated>2022-03-12T14:00:31+00:00</updated>
<author>
<name>amitkummer</name>
<email>49096391+amitkummer@users.noreply.github.com</email>
</author>
<published>2022-03-12T14:00:31+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/python-packages/pygments-git.git/commit/?id=62dd19e9d3e0400fff0aeddabae490797425c87e'/>
<id>62dd19e9d3e0400fff0aeddabae490797425c87e</id>
<content type='text'>
* Lex identifiers after `case` as constants

Add a state for marking identifiers preceded by  a `case` keyword as
constants.

Additionally, refactor the `label` rule to no longer permit a `case`
keyword before a label.

Consequentially, identifiers after a `case` keyword (like `foo` in
`case foo:`) are no longer wrongly lexed as `Name.Label`, but as
`Name.Constant`.

In addition, this fixes #2076, as multiple `case` keywords in one
line are lexed the same.

* Add test for multiple `case` keywords in one line

* Fix existing tests

* Lex `::` as Operator and not Name.Constant

After a `case`, when lexing a namespaced name, like `foo::bar`, lex the
namespace operator `::` as Operator, and not Name.Constant.

* Regenerate tokens</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Lex identifiers after `case` as constants

Add a state for marking identifiers preceded by  a `case` keyword as
constants.

Additionally, refactor the `label` rule to no longer permit a `case`
keyword before a label.

Consequentially, identifiers after a `case` keyword (like `foo` in
`case foo:`) are no longer wrongly lexed as `Name.Label`, but as
`Name.Constant`.

In addition, this fixes #2076, as multiple `case` keywords in one
line are lexed the same.

* Add test for multiple `case` keywords in one line

* Fix existing tests

* Lex `::` as Operator and not Name.Constant

After a `case`, when lexing a namespaced name, like `foo::bar`, lex the
namespace operator `::` as Operator, and not Name.Constant.

* Regenerate tokens</pre>
</div>
</content>
</entry>
<entry>
<title>Use the correct whitespace token for the C family.</title>
<updated>2021-06-20T09:42:04+00:00</updated>
<author>
<name>Matthäus G. Chajdas</name>
<email>dev@anteru.net</email>
</author>
<published>2021-06-20T09:42:04+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/python-packages/pygments-git.git/commit/?id=a537d5e9663e888706df75c40826be3aed177959'/>
<id>a537d5e9663e888706df75c40826be3aed177959</id>
<content type='text'>
The CFamilyLexer was matching whitespace as Text instead of Whitespace.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The CFamilyLexer was matching whitespace as Text instead of Whitespace.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix CFamilyLexer preprocessor tokenization errors (#1830)</title>
<updated>2021-06-20T09:38:00+00:00</updated>
<author>
<name>Henrik Lievonen</name>
<email>henrik.lievonen@hotmail.com</email>
</author>
<published>2021-06-20T09:38:00+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/python-packages/pygments-git.git/commit/?id=fea1fbc0576bb9d6d1dc84ac8cc3825e0a4e5232'/>
<id>fea1fbc0576bb9d6d1dc84ac8cc3825e0a4e5232</id>
<content type='text'>
CFamilyLexer failed to tokenize preprocessor macros when they were
preceded by line break surrounded by spaces. This was the case because
prerpocessor regex rule expected to start at the beginning of the line,
but the space regex rule matched also the whitespace after the line
break. Now the space rule has been refined not to match the line break.
Because of this, the preprocessor regex rule correctly matches
prerpocessor tokens even when they are preceded by white spaces, at the
cost of adding some more tokens in the token stream in some cases. This
change preserves the behavior of invalid preprocessor usage failing to
tokenize.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
CFamilyLexer failed to tokenize preprocessor macros when they were
preceded by line break surrounded by spaces. This was the case because
prerpocessor regex rule expected to start at the beginning of the line,
but the space regex rule matched also the whitespace after the line
break. Now the space rule has been refined not to match the line break.
Because of this, the preprocessor regex rule correctly matches
prerpocessor tokens even when they are preceded by white spaces, at the
cost of adding some more tokens in the token stream in some cases. This
change preserves the behavior of invalid preprocessor usage failing to
tokenize.</pre>
</div>
</content>
</entry>
<entry>
<title>Fixes for C and C++ functions and namespaces (#1722)</title>
<updated>2021-02-27T16:32:41+00:00</updated>
<author>
<name>amitkummer</name>
<email>49096391+amitkummer@users.noreply.github.com</email>
</author>
<published>2021-02-27T16:32:41+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/python-packages/pygments-git.git/commit/?id=c1a0d82acbe6fa026a1f6843fee00cc1df626549'/>
<id>c1a0d82acbe6fa026a1f6843fee00cc1df626549</id>
<content type='text'>
* Fix lexing of function names

This fixes #1561.

Add a keywords state that matches inside and outside functions for
keywords.

Before this, when a keyword would appear the lexer would go to the
statements state, in which functions were not matched.

* Add tests for lexing of function names

* Unbreak previous tests

* Allow namespaced names in function statements

Add a second identifiers regex that matces all the previous identifiers
and also '::'.

I took the decision to create a second identifiers regex with '::'
inside, simply because using the old identifiers regex would hurt
performance massively on every solution I tried to craft.

* Add tests for namespaced names in functions

* Unbreak previous tests

* Add support for namespaces in namespace declarations

Add a namespace state that is entered each time the namespace keyword
is matched and lexes all name matches as namespaces.

Cases this approach doesn't cover:
* Namespaces in using declarations.
* Namespaces that prefix names in random code.
Unfortunately, in both of these cases the names before and after '::'
are not always namespaces.

* Add tests for namespace declartions

* Unbreak previous tests

* Tidy functions regex

Remove group nesting that became unneeded after fc56ab8 (the last big
refactor).

* Remove f string usage I introduced by mistake</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Fix lexing of function names

This fixes #1561.

Add a keywords state that matches inside and outside functions for
keywords.

Before this, when a keyword would appear the lexer would go to the
statements state, in which functions were not matched.

* Add tests for lexing of function names

* Unbreak previous tests

* Allow namespaced names in function statements

Add a second identifiers regex that matces all the previous identifiers
and also '::'.

I took the decision to create a second identifiers regex with '::'
inside, simply because using the old identifiers regex would hurt
performance massively on every solution I tried to craft.

* Add tests for namespaced names in functions

* Unbreak previous tests

* Add support for namespaces in namespace declarations

Add a namespace state that is entered each time the namespace keyword
is matched and lexes all name matches as namespaces.

Cases this approach doesn't cover:
* Namespaces in using declarations.
* Namespaces that prefix names in random code.
Unfortunately, in both of these cases the names before and after '::'
are not always namespaces.

* Add tests for namespace declartions

* Unbreak previous tests

* Tidy functions regex

Remove group nesting that became unneeded after fc56ab8 (the last big
refactor).

* Remove f string usage I introduced by mistake</pre>
</div>
</content>
</entry>
<entry>
<title>Also add auto-updatable output-based tests to examplefiles (#1689)</title>
<updated>2021-01-20T09:48:45+00:00</updated>
<author>
<name>Oleh Prypin</name>
<email>oleh@pryp.in</email>
</author>
<published>2021-01-20T09:48:45+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/python-packages/pygments-git.git/commit/?id=6f4309217326430145564ae8b1bb393ea684f39f'/>
<id>6f4309217326430145564ae8b1bb393ea684f39f</id>
<content type='text'>
Co-authored-by: Georg Brandl &lt;georg@python.org&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Co-authored-by: Georg Brandl &lt;georg@python.org&gt;</pre>
</div>
</content>
</entry>
</feed>
