Merged in Reedbeta/pygments-main/hlsl-lexer (pull request #675)

Add HLSL lexer

Approved-by: Anteru <bitbucket@ca.sh13.net>

5 files changed, 84 insertions, 5 deletions

diff --git a/doc/_themes/pygments14/layout.html b/doc/_themes/pygments14/layout.html
index 2cc03e03..e8860827 100644
--- a/doc/_themes/pygments14/layout.html
+++ b/doc/_themes/pygments14/layout.html
@@ -82,7 +82,7 @@
 
 {% block footer %}
     <div class="footer" role="contentinfo">
-      &copy; Copyright 2006-2015, Georg Brandl and Pygments contributors.
+      &copy; Copyright 2006-2017, Georg Brandl and Pygments contributors.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> {{
       sphinx_version }}. <br/>
       Pygments logo created by <a href="http://joelunger.com">Joel Unger</a>.
diff --git a/doc/_themes/pygments14/static/pygments14.css_t b/doc/_themes/pygments14/static/pygments14.css_t
index 5c37aaf9..7f09f623 100644
--- a/doc/_themes/pygments14/static/pygments14.css_t
+++ b/doc/_themes/pygments14/static/pygments14.css_t
@@ -4,7 +4,7 @@
  *
  * Sphinx stylesheet -- pygments14 theme.  Heavily copied from sphinx13.
  *
- * :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS.
+ * :copyright: Copyright 2006-2017 by the Pygments team, see AUTHORS.
  * :license: BSD, see LICENSE for details.
  *
  */
diff --git a/doc/docs/api.rst b/doc/docs/api.rst
index dd831bd1..a6b242dd 100644
--- a/doc/docs/api.rst
+++ b/doc/docs/api.rst
@@ -62,6 +62,18 @@ Functions from :mod:`pygments.lexers`:
     Will raise :exc:`pygments.util.ClassNotFound` if not lexer for that mimetype
     is found.
 
+.. function:: load_lexer_from_file(filename, lexername="CustomLexer", **options)
+
+    Return a `Lexer` subclass instance loaded from the provided file, relative
+    to the current directory. The file is expected to contain a Lexer class
+    named `lexername` (by default, CustomLexer). Users should be very careful with
+    the input, because this method is equivalent to running eval on the input file.
+    The lexer is given the `options` at its instantiation.
+
+    :exc:`ClassNotFound` is raised if there are any errors loading the Lexer
+
+    .. versionadded:: 2.2
+
 .. function:: guess_lexer(text, **options)
 
     Return a `Lexer` subclass instance that's guessed from the text in
@@ -125,6 +137,17 @@ Functions from :mod:`pygments.formatters`:
     Will raise :exc:`pygments.util.ClassNotFound` if no formatter for that filename
     is found.
 
+.. function:: load_formatter_from_file(filename, formattername="CustomFormatter", **options)
+
+    Return a `Formatter` subclass instance loaded from the provided file, relative
+    to the current directory. The file is expected to contain a Formatter class
+    named ``formattername`` (by default, CustomFormatter). Users should be very
+    careful with the input, because this method is equivalent to running eval
+    on the input file. The formatter is given the `options` at its instantiation.
+
+    :exc:`ClassNotFound` is raised if there are any errors loading the Formatter
+
+    .. versionadded:: 2.2
 
 .. module:: pygments.styles
 
diff --git a/doc/docs/cmdline.rst b/doc/docs/cmdline.rst
index 165af969..e4f94ea5 100644
--- a/doc/docs/cmdline.rst
+++ b/doc/docs/cmdline.rst
@@ -99,6 +99,23 @@ The ``-N`` option guesses a lexer name for a given filename, so that ::
 will print out ``python``.  It won't highlight anything yet.  If no specific
 lexer is known for that filename, ``text`` is printed.
 
+Custom Lexers and Formatters
+----------------------------
+
+.. versionadded:: 2.2
+
+The ``-x`` flag enables custom lexers and formatters to be loaded
+from files relative to the current directory. Create a file with a class named
+CustomLexer or CustomFormatter, then specify it on the command line::
+
+    $ pygmentize -l your_lexer.py -f your_formatter.py -x
+
+You can also specify the name of your class with a colon::
+
+    $ pygmentize -l your_lexer.py:SomeLexer -x
+
+For more information, see :doc:`the Pygments documentation on Lexer development
+<lexerdevelopment>`.
 
 Getting help
 ------------
diff --git a/doc/docs/lexerdevelopment.rst b/doc/docs/lexerdevelopment.rst
index fd6e76b9..63bd01a3 100644
--- a/doc/docs/lexerdevelopment.rst
+++ b/doc/docs/lexerdevelopment.rst
@@ -88,8 +88,47 @@ one.
 Adding and testing a new lexer
 ==============================
 
-Using a lexer that is not part of Pygments can be done via the Python API.  You
-can import and instantiate the lexer, and pass it to :func:`pygments.highlight`.
+The easiest way to use a new lexer is to use Pygments' support for loading
+the lexer from a file relative to your current directory.
+
+First, change the name of your lexer class to CustomLexer:
+
+.. code-block:: python
+
+    from pygments.lexer import RegexLexer
+    from pygments.token import *
+
+    class CustomLexer(RegexLexer):
+        """All your lexer code goes here!"""
+
+Then you can load the lexer from the command line with the additional
+flag ``-x``:
+
+.. code-block:: console
+
+    $ pygmentize -l your_lexer_file.py -x
+
+To specify a class name other than CustomLexer, append it with a colon:
+
+.. code-block:: console
+
+    $ pygmentize -l your_lexer.py:SomeLexer -x
+
+Or, using the Python API:
+
+.. code-block:: python
+
+    # For a lexer named CustomLexer
+    your_lexer = load_lexer_from_file(filename, **options)
+
+    # For a lexer named MyNewLexer
+    your_named_lexer = load_lexer_from_file(filename, "MyNewLexer", **options)
+
+When loading custom lexers and formatters, be extremely careful to use only
+trusted files; Pygments will perform the equivalent of ``eval`` on them.
+
+If you only want to use your lexer with the Pygments API, you can import and
+instantiate the lexer yourself, then pass it to :func:`pygments.highlight`.
 
 To prepare your new lexer for inclusion in the Pygments distribution, so that it
 will be found when passing filenames or lexer aliases from the command line, you
@@ -361,7 +400,7 @@ There are a few more things you can do with states:
           tokens = {...}
 
           def get_tokens_unprocessed(self, text, stack=('root', 'otherstate')):
-              for item in RegexLexer.get_tokens_unprocessed(text, stack):
+              for item in RegexLexer.get_tokens_unprocessed(self, text, stack):
                   yield item
 
   Some lexers like the `PhpLexer` use this to make the leading ``<?php``