pyupgrade is a tool to automatically upgrade syntax for newer versions
of the Python language.

The project has been Python 3 only since
35544e2fc6eed0ce4a27ec7285aac71ff0ddc473, allowing for several cleanups:

- Remove unnecessary "-*- coding: utf-8 -*-" cookie. Python 3 reads all
  source files as utf-8 by default.

- Replace IOError/EnvironmentError with OSError. Python 3 unified these
  exceptions. The old names are aliases only.

- Use the Python 3 shorter super() syntax.

- Remove "utf8" argument form encode/decode. In Python 3, this value is
  the default.

- Remove "r" from open() calls. In Python 3, this value is the default.

- Remove u prefix from Unicode strings. In Python 3, all strings are
  Unicode.

- Replace io.open() with builtin open(). In Python 3, these functions
  are functionally equivalent.

Co-authored-by: Matthäus G. Chajdas <Anteru@users.noreply.github.com>

This should revert the behavior of the function without losing
the overall caching behavior that was intended.

Closes #1644

This seems to break some themes which were not expecting Pygments to
change margins, and it doesn't look like it makes a difference for
standalone Pygments.

* Update the JSON-LD keyword list to match JSON-LD 1.1

Changes in this patch:

* Update the JSON-LD URL to HTTPS
* Update the list of JSON-LD keywords
* Make the JSON-LD parser less dependent on the JSON lexer implementation
* Add unit tests for the JSON-LD lexer

* Add unit tests for the JSON parser

This includes:

* Testing valid literals
* Testing valid string escapes
* Testing that object keys are tokenized differently from string values

* Rewrite the JSON lexer

Related to #1425

Included in this change:

* The JSON parser is rewritten
* The JSON bare object parser no longer requires additional code
* `get_tokens_unprocessed()` returns as much as it can to reduce yields
  (for example, side-by-side punctuation is not returned separately)
* The unit tests were updated
* Add unit tests based on Hypothesis test results

* Reduce HTML formatter memory consumption by ~33% and speed it up

Related to #1425

Tested on a 118MB JSON file. Memory consumption tops out at ~3GB before
this patch and drops to only ~2GB with this patch. These were the command
lines used:

python -m pygments -l json -f html -o .\new-code-classes.html .\jc-output.txt
python -m pygments -l json -f html -O "noclasses" -o .\new-code-styles.html .\jc-output.txt

* Add an LRU cache to the HTML formatter's HTML-escaping and line-splitting

For a 118MB JSON input file, this reduces memory consumption by ~500MB
and reduces formatting time by ~15 seconds.

* JSON: Add a catastrophic backtracking test back to the test suite

* JSON: Update the comment that documents the internal queue

* JSON: Document in comments that ints/floats/constants are not validated

This removes the top/bottom padding changes, and only keeps left/right
padding, in the hope that this does not break all Sphinx themes.

* Fix LatexEmbeddedLexer to not call the nested tokenizer piecewise

* Reuse the existing do_insertions function

* Add a test for the `LatexEmbeddedLexer`

fixes #1223

* all: remove "u" string prefix

* util: remove unirange

Since Python 3.3, all builds are wide unicode compatible.

* unistring: remove support for narrow-unicode builds

which stopped being relevant with Python 3.3