From e130dbff2eb591c7cdca4a0bef90b63502cd3d4e Mon Sep 17 00:00:00 2001 From: gbrandl Date: Thu, 3 May 2007 18:23:03 +0200 Subject: [svn] Add whitespace filter. Add a few more filter unit tests. --- docs/src/tokens.txt | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) (limited to 'docs/src') diff --git a/docs/src/tokens.txt b/docs/src/tokens.txt index f98b4d6e..3a1cd4af 100644 --- a/docs/src/tokens.txt +++ b/docs/src/tokens.txt @@ -57,8 +57,9 @@ For some tokens aliases are already defined: Inside the `pygments.token` module the following aliases are defined: -============= ============================ ================================== +============= ============================ ==================================== `Text` `Token.Text` for any type of text data +`Whitespace` `Token.Text.Whitespace` for specially highlighted whitespace `Error` `Token.Error` represents lexer errors `Other` `Token.Other` special token for data not matched by a parser (e.g. HTML @@ -73,7 +74,10 @@ Inside the `pygments.token` module the following aliases are defined: `Comment` `Token.Comment` any kind of comments `Generic` `Token.Generic` generic tokens (have a look at the explanation below) -============= ============================ ================================== +============= ============================ ==================================== + +The `Whitespace` token type is new in Pygments 0.8. It is used only by the +`VisibleWhitespaceFilter` currently. Normally you just create token types using the already defined aliases. For each of those token aliases, a number of subtypes exists (excluding the special tokens -- cgit v1.2.1