diff options
Diffstat (limited to 'doc/extdev')
| -rw-r--r-- | doc/extdev/appapi.rst | 537 | ||||
| -rw-r--r-- | doc/extdev/builderapi.rst | 30 | ||||
| -rw-r--r-- | doc/extdev/domainapi.rst | 14 | ||||
| -rw-r--r-- | doc/extdev/envapi.rst | 54 | ||||
| -rw-r--r-- | doc/extdev/index.rst | 33 | ||||
| -rw-r--r-- | doc/extdev/markupapi.rst | 11 | ||||
| -rw-r--r-- | doc/extdev/nodes.rst | 57 | ||||
| -rw-r--r-- | doc/extdev/tutorial.rst | 397 |
8 files changed, 1133 insertions, 0 deletions
diff --git a/doc/extdev/appapi.rst b/doc/extdev/appapi.rst new file mode 100644 index 000000000..eca191bbd --- /dev/null +++ b/doc/extdev/appapi.rst @@ -0,0 +1,537 @@ +.. highlight:: rest + +Application API +=============== + +.. module:: sphinx.application + :synopsis: Application class and extensibility interface. + + +Each Sphinx extension is a Python module with at least a :func:`setup` function. +This function is called at initialization time with one argument, the +application object representing the Sphinx process. + +.. class:: Sphinx + + This application object has the public API described in the following. + +Extension setup +--------------- + +These methods are usually called in an extension's ``setup()`` function. + +Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext` +package. + +.. method:: Sphinx.setup_extension(name) + + Load the extension given by the module *name*. Use this if your extension + needs the features provided by another extension. + +.. method:: Sphinx.add_builder(builder) + + Register a new builder. *builder* must be a class that inherits from + :class:`~sphinx.builders.Builder`. + +.. method:: Sphinx.add_config_value(name, default, rebuild) + + Register a configuration value. This is necessary for Sphinx to recognize + new values and set default values accordingly. The *name* should be prefixed + with the extension name, to avoid clashes. The *default* value can be any + Python object. The string value *rebuild* must be one of those values: + + * ``'env'`` if a change in the setting only takes effect when a document is + parsed -- this means that the whole environment must be rebuilt. + * ``'html'`` if a change in the setting needs a full rebuild of HTML + documents. + * ``''`` if a change in the setting will not need any special rebuild. + + .. versionchanged:: 0.4 + If the *default* value is a callable, it will be called with the config + object as its argument in order to get the default value. This can be + used to implement config values whose default depends on other values. + + .. versionchanged:: 0.6 + Changed *rebuild* from a simple boolean (equivalent to ``''`` or + ``'env'``) to a string. However, booleans are still accepted and + converted internally. + +.. method:: Sphinx.add_domain(domain) + + Make the given *domain* (which must be a class; more precisely, a subclass of + :class:`~sphinx.domains.Domain`) known to Sphinx. + + .. versionadded:: 1.0 + +.. method:: Sphinx.override_domain(domain) + + Make the given *domain* class known to Sphinx, assuming that there is already + a domain with its ``.name``. The new domain must be a subclass of the + existing one. + + .. versionadded:: 1.0 + +.. method:: Sphinx.add_index_to_domain(domain, index) + + Add a custom *index* class to the domain named *domain*. *index* must be a + subclass of :class:`~sphinx.domains.Index`. + + .. versionadded:: 1.0 + +.. method:: Sphinx.add_event(name) + + Register an event called *name*. This is needed to be able to emit it. + +.. method:: Sphinx.add_node(node, **kwds) + + Register a Docutils node class. This is necessary for Docutils internals. + It may also be used in the future to validate nodes in the parsed documents. + + Node visitor functions for the Sphinx HTML, LaTeX, text and manpage writers + can be given as keyword arguments: the keyword must be one or more of + ``'html'``, ``'latex'``, ``'text'``, ``'man'``, ``'texinfo'``, the value a + 2-tuple of ``(visit, depart)`` methods. ``depart`` can be ``None`` if the + ``visit`` function raises :exc:`docutils.nodes.SkipNode`. Example: + + .. code-block:: python + + class math(docutils.nodes.Element): pass + + def visit_math_html(self, node): + self.body.append(self.starttag(node, 'math')) + def depart_math_html(self, node): + self.body.append('</math>') + + app.add_node(math, html=(visit_math_html, depart_math_html)) + + Obviously, translators for which you don't specify visitor methods will choke + on the node when encountered in a document to translate. + + .. versionchanged:: 0.5 + Added the support for keyword arguments giving visit functions. + +.. method:: Sphinx.add_directive(name, func, content, arguments, **options) + Sphinx.add_directive(name, directiveclass) + + Register a Docutils directive. *name* must be the prospective directive + name. There are two possible ways to write a directive: + + * In the docutils 0.4 style, *obj* is the directive function. *content*, + *arguments* and *options* are set as attributes on the function and + determine whether the directive has content, arguments and options, + respectively. **This style is deprecated.** + + * In the docutils 0.5 style, *directiveclass* is the directive class. It + must already have attributes named *has_content*, *required_arguments*, + *optional_arguments*, *final_argument_whitespace* and *option_spec* that + correspond to the options for the function way. See `the Docutils docs + <http://docutils.sourceforge.net/docs/howto/rst-directives.html>`_ for + details. + + The directive class must inherit from the class + ``docutils.parsers.rst.Directive``. + + For example, the (already existing) :rst:dir:`literalinclude` directive would be + added like this: + + .. code-block:: python + + from docutils.parsers.rst import directives + add_directive('literalinclude', literalinclude_directive, + content = 0, arguments = (1, 0, 0), + linenos = directives.flag, + language = direcitves.unchanged, + encoding = directives.encoding) + + .. versionchanged:: 0.6 + Docutils 0.5-style directive classes are now supported. + +.. method:: Sphinx.add_directive_to_domain(domain, name, func, content, arguments, **options) + Sphinx.add_directive_to_domain(domain, name, directiveclass) + + Like :meth:`add_directive`, but the directive is added to the domain named + *domain*. + + .. versionadded:: 1.0 + +.. method:: Sphinx.add_role(name, role) + + Register a Docutils role. *name* must be the role name that occurs in the + source, *role* the role function (see the `Docutils documentation + <http://docutils.sourceforge.net/docs/howto/rst-roles.html>`_ on details). + +.. method:: Sphinx.add_role_to_domain(domain, name, role) + + Like :meth:`add_role`, but the role is added to the domain named *domain*. + + .. versionadded:: 1.0 + +.. method:: Sphinx.add_generic_role(name, nodeclass) + + Register a Docutils role that does nothing but wrap its contents in the + node given by *nodeclass*. + + .. versionadded:: 0.6 + +.. method:: Sphinx.add_object_type(directivename, rolename, indextemplate='', parse_node=None, \ + ref_nodeclass=None, objname='', doc_field_types=[]) + + This method is a very convenient way to add a new :term:`object` type that + can be cross-referenced. It will do this: + + * Create a new directive (called *directivename*) for documenting an object. + It will automatically add index entries if *indextemplate* is nonempty; if + given, it must contain exactly one instance of ``%s``. See the example + below for how the template will be interpreted. + * Create a new role (called *rolename*) to cross-reference to these + object descriptions. + * If you provide *parse_node*, it must be a function that takes a string and + a docutils node, and it must populate the node with children parsed from + the string. It must then return the name of the item to be used in + cross-referencing and index entries. See the :file:`conf.py` file in the + source for this documentation for an example. + * The *objname* (if not given, will default to *directivename*) names the + type of object. It is used when listing objects, e.g. in search results. + + For example, if you have this call in a custom Sphinx extension:: + + app.add_object_type('directive', 'dir', 'pair: %s; directive') + + you can use this markup in your documents:: + + .. rst:directive:: function + + Document a function. + + <...> + + See also the :rst:dir:`function` directive. + + For the directive, an index entry will be generated as if you had prepended :: + + .. index:: pair: function; directive + + The reference node will be of class ``literal`` (so it will be rendered in a + proportional font, as appropriate for code) unless you give the + *ref_nodeclass* argument, which must be a docutils node class. Most useful + are ``docutils.nodes.emphasis`` or ``docutils.nodes.strong`` -- you can also + use ``docutils.nodes.generated`` if you want no further text decoration. If + the text should be treated as literal (e.g. no smart quote replacement), but + not have typewriter styling, use ``sphinx.addnodes.literal_emphasis`` or + ``sphinx.addnodes.literal_strong``. + + For the role content, you have the same syntactical possibilities as for + standard Sphinx roles (see :ref:`xref-syntax`). + + This method is also available under the deprecated alias + ``add_description_unit``. + +.. method:: Sphinx.add_crossref_type(directivename, rolename, indextemplate='', ref_nodeclass=None, objname='') + + This method is very similar to :meth:`add_object_type` except that the + directive it generates must be empty, and will produce no output. + + That means that you can add semantic targets to your sources, and refer to + them using custom roles instead of generic ones (like :rst:role:`ref`). Example + call:: + + app.add_crossref_type('topic', 'topic', 'single: %s', docutils.nodes.emphasis) + + Example usage:: + + .. topic:: application API + + The application API + ------------------- + + <...> + + See also :topic:`this section <application API>`. + + (Of course, the element following the ``topic`` directive needn't be a + section.) + +.. method:: Sphinx.add_transform(transform) + + Add the standard docutils :class:`Transform` subclass *transform* to the list + of transforms that are applied after Sphinx parses a reST document. + +.. method:: Sphinx.add_javascript(filename) + + Add *filename* to the list of JavaScript files that the default HTML template + will include. The filename must be relative to the HTML static path, see + :confval:`the docs for the config value <html_static_path>`. A full URI with + scheme, like ``http://example.org/foo.js``, is also supported. + + .. versionadded:: 0.5 + +.. method:: Sphinx.add_stylesheet(filename) + + Add *filename* to the list of CSS files that the default HTML template will + include. Like for :meth:`add_javascript`, the filename must be relative to + the HTML static path, or a full URI with scheme. + + .. versionadded:: 1.0 + +.. method:: Sphinx.add_lexer(alias, lexer) + + Use *lexer*, which must be an instance of a Pygments lexer class, to + highlight code blocks with the given language *alias*. + + .. versionadded:: 0.6 + +.. method:: Sphinx.add_autodocumenter(cls) + + Add *cls* as a new documenter class for the :mod:`sphinx.ext.autodoc` + extension. It must be a subclass of :class:`sphinx.ext.autodoc.Documenter`. + This allows to auto-document new types of objects. See the source of the + autodoc module for examples on how to subclass :class:`Documenter`. + + .. XXX add real docs for Documenter and subclassing + + .. versionadded:: 0.6 + +.. method:: Sphinx.add_autodoc_attrgetter(type, getter) + + Add *getter*, which must be a function with an interface compatible to the + :func:`getattr` builtin, as the autodoc attribute getter for objects that are + instances of *type*. All cases where autodoc needs to get an attribute of a + type are then handled by this function instead of :func:`getattr`. + + .. versionadded:: 0.6 + +.. method:: Sphinx.add_search_language(cls) + + Add *cls*, which must be a subclass of :class:`sphinx.search.SearchLanguage`, + as a support language for building the HTML full-text search index. The + class must have a *lang* attribute that indicates the language it should be + used for. See :confval:`html_search_language`. + + .. versionadded:: 1.1 + +.. method:: Sphinx.require_sphinx(version) + + Compare *version* (which must be a ``major.minor`` version string, + e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build + when it is too old. + + .. versionadded:: 1.0 + +.. method:: Sphinx.connect(event, callback) + + Register *callback* to be called when *event* is emitted. For details on + available core events and the arguments of callback functions, please see + :ref:`events`. + + The method returns a "listener ID" that can be used as an argument to + :meth:`disconnect`. + +.. method:: Sphinx.disconnect(listener_id) + + Unregister callback *listener_id*. + + +.. exception:: ExtensionError + + All these methods raise this exception if something went wrong with the + extension API. + + +Emitting events +--------------- + +.. method:: Sphinx.emit(event, *arguments) + + Emit *event* and pass *arguments* to the callback functions. Return the + return values of all callbacks as a list. Do not emit core Sphinx events + in extensions! + +.. method:: Sphinx.emit_firstresult(event, *arguments) + + Emit *event* and pass *arguments* to the callback functions. Return the + result of the first callback that doesn't return ``None``. + + .. versionadded:: 0.5 + + +Producing messages / logging +---------------------------- + +The application object also provides support for emitting leveled messages. + +.. automethod:: Sphinx.warn + +.. automethod:: Sphinx.info + +.. automethod:: Sphinx.verbose + +.. automethod:: Sphinx.debug + +.. automethod:: Sphinx.debug2 + + +.. _events: + +Sphinx core events +------------------ + +These events are known to the core. The arguments shown are given to the +registered event handlers. + +.. event:: builder-inited (app) + + Emitted when the builder object has been created. It is available as + ``app.builder``. + +.. event:: env-get-outdated (app, env, added, changed, removed) + + Emitted when the environment determines which source files have changed and + should be re-read. *added*, *changed* and *removed* are sets of docnames + that the environment has determined. You can return a list of docnames to + re-read in addition to these. + + .. versionadded:: 1.1 + +.. event:: env-purge-doc (app, env, docname) + + Emitted when all traces of a source file should be cleaned from the + environment, that is, if the source file is removed or before it is freshly + read. This is for extensions that keep their own caches in attributes of the + environment. + + For example, there is a cache of all modules on the environment. When a + source file has been changed, the cache's entries for the file are cleared, + since the module declarations could have been removed from the file. + + .. versionadded:: 0.5 + +.. event:: source-read (app, docname, source) + + Emitted when a source file has been read. The *source* argument is a list + whose single element is the contents of the source file. You can process the + contents and replace this item to implement source-level transformations. + + For example, if you want to use ``$`` signs to delimit inline math, like in + LaTeX, you can use a regular expression to replace ``$...$`` by + ``:math:`...```. + + .. versionadded:: 0.5 + +.. event:: doctree-read (app, doctree) + + Emitted when a doctree has been parsed and read by the environment, and is + about to be pickled. The *doctree* can be modified in-place. + +.. event:: missing-reference (app, env, node, contnode) + + Emitted when a cross-reference to a Python module or object cannot be + resolved. If the event handler can resolve the reference, it should return a + new docutils node to be inserted in the document tree in place of the node + *node*. Usually this node is a :class:`reference` node containing *contnode* + as a child. + + :param env: The build environment (``app.builder.env``). + :param node: The :class:`pending_xref` node to be resolved. Its attributes + ``reftype``, ``reftarget``, ``modname`` and ``classname`` attributes + determine the type and target of the reference. + :param contnode: The node that carries the text and formatting inside the + future reference and should be a child of the returned reference node. + + .. versionadded:: 0.5 + +.. event:: doctree-resolved (app, doctree, docname) + + Emitted when a doctree has been "resolved" by the environment, that is, all + references have been resolved and TOCs have been inserted. The *doctree* can + be modified in place. + + Here is the place to replace custom nodes that don't have visitor methods in + the writers, so that they don't cause errors when the writers encounter them. + +.. event:: env-updated (app, env) + + Emitted when the :meth:`update` method of the build environment has + completed, that is, the environment and all doctrees are now up-to-date. + + .. versionadded:: 0.5 + +.. event:: html-collect-pages (app) + + Emitted when the HTML builder is starting to write non-document pages. You + can add pages to write by returning an iterable from this event consisting of + ``(pagename, context, templatename)``. + + .. versionadded:: 1.0 + +.. event:: html-page-context (app, pagename, templatename, context, doctree) + + Emitted when the HTML builder has created a context dictionary to render a + template with -- this can be used to add custom elements to the context. + + The *pagename* argument is the canonical name of the page being rendered, + that is, without ``.html`` suffix and using slashes as path separators. The + *templatename* is the name of the template to render, this will be + ``'page.html'`` for all pages from reST documents. + + The *context* argument is a dictionary of values that are given to the + template engine to render the page and can be modified to include custom + values. Keys must be strings. + + The *doctree* argument will be a doctree when the page is created from a reST + documents; it will be ``None`` when the page is created from an HTML template + alone. + + .. versionadded:: 0.4 + +.. event:: build-finished (app, exception) + + Emitted when a build has finished, before Sphinx exits, usually used for + cleanup. This event is emitted even when the build process raised an + exception, given as the *exception* argument. The exception is reraised in + the application after the event handlers have run. If the build process + raised no exception, *exception* will be ``None``. This allows to customize + cleanup actions depending on the exception status. + + .. versionadded:: 0.5 + + +Checking the Sphinx version +--------------------------- + +.. currentmodule:: sphinx + +Use this to adapt your extension to API changes in Sphinx. + +.. data:: version_info + + A tuple of five elements; for Sphinx version 1.2.1 beta 3 this would be + ``(1, 2, 1, 'beta', 3)``. + + .. versionadded:: 1.2 + Before version 1.2, check the string ``sphinx.__version__``. + + +The Config object +----------------- + +.. module:: sphinx.config + +.. class:: Config + + The config object makes the values of all config values available as + attributes. + + It is available as the ``config`` attribute on the application and + environment objects. For example, to get the value of :confval:`language`, + use either ``app.config.language`` or ``env.config.language``. + + +.. _template-bridge: + +The template bridge +------------------- + +.. currentmodule:: sphinx.application + +.. autoclass:: TemplateBridge + :members: diff --git a/doc/extdev/builderapi.rst b/doc/extdev/builderapi.rst new file mode 100644 index 000000000..cd8688a36 --- /dev/null +++ b/doc/extdev/builderapi.rst @@ -0,0 +1,30 @@ +.. _writing-builders: + +Builder API +=========== + +.. todo:: Expand this. + +.. currentmodule:: sphinx.builders + +.. class:: Builder + + This is the base class for all builders. + + These methods are predefined and will be called from the application: + + .. automethod:: get_relative_uri + .. automethod:: build_all + .. automethod:: build_specific + .. automethod:: build_update + .. automethod:: build + + These methods can be overridden in concrete builder classes: + + .. automethod:: init + .. automethod:: get_outdated_docs + .. automethod:: get_target_uri + .. automethod:: prepare_writing + .. automethod:: write_doc + .. automethod:: finish + diff --git a/doc/extdev/domainapi.rst b/doc/extdev/domainapi.rst new file mode 100644 index 000000000..d6ecf0633 --- /dev/null +++ b/doc/extdev/domainapi.rst @@ -0,0 +1,14 @@ +.. _domain-api: + +Domain API +---------- + +.. module:: sphinx.domains + +.. autoclass:: Domain + :members: + +.. autoclass:: ObjType + +.. autoclass:: Index + :members: diff --git a/doc/extdev/envapi.rst b/doc/extdev/envapi.rst new file mode 100644 index 000000000..56771f733 --- /dev/null +++ b/doc/extdev/envapi.rst @@ -0,0 +1,54 @@ +Build environment API +===================== + +.. module:: sphinx.environment + +.. class:: BuildEnvironment + + **Attributes** + + .. attribute:: app + + Reference to the application object. + + .. attribute:: config + + Reference to the :class:`.Config` object. + + .. attribute:: srcdir + + Source directory (the directory containing ``conf.py``). + + .. attribute:: doctreedir + + Directory for storing pickled doctrees. + + .. attribute:: found_docs + + A set of all existing docnames. + + .. attribute:: metadata + + Dictionary mapping docnames to "metadata" (see :ref:`metadata`). + + .. attribute:: titles + + Dictionary mapping docnames to the docutils node for their main title. + + .. autoattribute:: docname + + **Utility methods** + + .. automethod:: warn + + .. automethod:: warn_node + + .. automethod:: doc2path + + .. automethod:: relfn2path + + .. automethod:: note_dependency + + .. automethod:: new_serialno + + .. automethod:: note_reread diff --git a/doc/extdev/index.rst b/doc/extdev/index.rst new file mode 100644 index 000000000..b76928c03 --- /dev/null +++ b/doc/extdev/index.rst @@ -0,0 +1,33 @@ +.. _dev-extensions: + +Developing extensions for Sphinx +================================ + +Since many projects will need special features in their documentation, Sphinx is +designed to be extensible on several levels. + +This is what you can do in an extension: First, you can add new +:term:`builder`\s to support new output formats or actions on the parsed +documents. Then, it is possible to register custom reStructuredText roles and +directives, extending the markup. And finally, there are so-called "hook +points" at strategic places throughout the build process, where an extension can +register a hook and run specialized code. + +An extension is simply a Python module. When an extension is loaded, Sphinx +imports this module and executes its ``setup()`` function, which in turn +notifies Sphinx of everything the extension offers -- see the extension tutorial +for examples. + +The configuration file itself can be treated as an extension if it contains a +``setup()`` function. All other extensions to load must be listed in the +:confval:`extensions` configuration value. + +.. toctree:: + + tutorial + appapi + envapi + builderapi + markupapi + domainapi + nodes diff --git a/doc/extdev/markupapi.rst b/doc/extdev/markupapi.rst new file mode 100644 index 000000000..8f6c29b1f --- /dev/null +++ b/doc/extdev/markupapi.rst @@ -0,0 +1,11 @@ +Docutils markup API +=================== + +Roles +----- + + +Directives +---------- + +TODO. diff --git a/doc/extdev/nodes.rst b/doc/extdev/nodes.rst new file mode 100644 index 000000000..e67fa3da6 --- /dev/null +++ b/doc/extdev/nodes.rst @@ -0,0 +1,57 @@ +.. _nodes: + +Doctree node classes added by Sphinx +==================================== + +.. module:: sphinx.addnodes + +Nodes for domain-specific object descriptions +--------------------------------------------- + +.. autoclass:: desc +.. autoclass:: desc_signature +.. autoclass:: desc_addname +.. autoclass:: desc_type +.. autoclass:: desc_returns +.. autoclass:: desc_name +.. autoclass:: desc_parameterlist +.. autoclass:: desc_parameter +.. autoclass:: desc_optional +.. autoclass:: desc_annotation +.. autoclass:: desc_content + +New admonition-like constructs +------------------------------ + +.. autoclass:: versionmodified +.. autoclass:: seealso + +Other paragraph-level nodes +------------------------------- + +.. autoclass:: compact_paragraph + +New inline nodes +---------------- + +.. autoclass:: index +.. autoclass:: pending_xref +.. autoclass:: literal_emphasis +.. autoclass:: abbreviation +.. autoclass:: download_reference + +Special nodes +------------- + +.. autoclass:: only +.. autoclass:: meta +.. autoclass:: highlightlang + +You should not need to generate the nodes below in extensions. + +.. autoclass:: glossary +.. autoclass:: toctree +.. autoclass:: start_of_file +.. autoclass:: productionlist +.. autoclass:: production +.. autoclass:: termsep diff --git a/doc/extdev/tutorial.rst b/doc/extdev/tutorial.rst new file mode 100644 index 000000000..382641904 --- /dev/null +++ b/doc/extdev/tutorial.rst @@ -0,0 +1,397 @@ +.. _exttut: + +Tutorial: Writing a simple extension +==================================== + +This section is intended as a walkthrough for the creation of custom extensions. +It covers the basics of writing and activating an extensions, as well as +commonly used features of extensions. + +As an example, we will cover a "todo" extension that adds capabilities to +include todo entries in the documentation, and collecting these in a central +place. (A similar "todo" extension is distributed with Sphinx.) + + +Important objects +----------------- + +There are several key objects whose API you will use while writing an +extension. These are: + +**Application** + The application object (usually called ``app``) is an instance of + :class:`.Sphinx`. It controls the most high-level functionality, such as the + setup of extensions, event dispatching and producing output (logging). + + If you have the environment object, the application is available as + ``env.app``. + +**Environment** + The build environment object (usually called ``env``) is an instance of + :class:`.BuildEnvironment`. It is responsible for parsing the source + documents stores all metadata about the document collection and is serialized + after each build. + + Its API provides methods to do with access to metadata, resolving references, + etc. It can also be used by extensions to cache information that should + persist for incremental rebuilds. + + If you have the application or builder object, the environment is available + as ``app.env`` or ``builder.env``. + +**Builder** + The builder object (usually called ``builder``) is an instance of a specific + subclass of :class:`.Builder`. Each builder class knows how to convert the + parsed documents into an output format, or otherwise process them (e.g. check + external links). + + If you have the application object, the environment is available as + ``app.builder``. + +**Config** + The config object (usually called ``config``) provides the values of + configuration values set in :file:`conf.py` as attributes. It is an instance + of :class:`.Config`. + + The config is available as ``app.config`` or ``env.config``. + + +Build Phases +------------ + +One thing that is vital in order to understand extension mechanisms is the way +in which a Sphinx project is built: this works in several phases. + +**Phase 0: Initialization** + + In this phase, almost nothing interesting for us happens. The source + directory is searched for source files, and extensions are initialized. + Should a stored build environment exist, it is loaded, otherwise a new one is + created. + +**Phase 1: Reading** + + In Phase 1, all source files (and on subsequent builds, those that are new or + changed) are read and parsed. This is the phase where directives and roles + are encountered by the docutils, and the corresponding code is executed. The + output of this phase is a *doctree* for each source files, that is a tree of + docutils nodes. For document elements that aren't fully known until all + existing files are read, temporary nodes are created. + + There are nodes provided by docutils, which are documented `in the docutils + documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__. + Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`. + + During reading, the build environment is updated with all meta- and cross + reference data of the read documents, such as labels, the names of headings, + described Python objects and index entries. This will later be used to + replace the temporary nodes. + + The parsed doctrees are stored on the disk, because it is not possible to + hold all of them in memory. + +**Phase 2: Consistency checks** + + Some checking is done to ensure no surprises in the built documents. + +**Phase 3: Resolving** + + Now that the metadata and cross-reference data of all existing documents is + known, all temporary nodes are replaced by nodes that can be converted into + output. For example, links are created for object references that exist, and + simple literal nodes are created for those that don't. + +**Phase 4: Writing** + + This phase converts the resolved doctrees to the desired output format, such + as HTML or LaTeX. This happens via a so-called docutils writer that visits + the individual nodes of each doctree and produces some output in the process. + +.. note:: + + Some builders deviate from this general build plan, for example, the builder + that checks external links does not need anything more than the parsed + doctrees and therefore does not have phases 2--4. + + +Extension Design +---------------- + +We want the extension to add the following to Sphinx: + +* A "todo" directive, containing some content that is marked with "TODO", and + only shown in the output if a new config value is set. (Todo entries should + not be in the output by default.) + +* A "todolist" directive that creates a list of all todo entries throughout the + documentation. + +For that, we will need to add the following elements to Sphinx: + +* New directives, called ``todo`` and ``todolist``. +* New document tree nodes to represent these directives, conventionally also + called ``todo`` and ``todolist``. We wouldn't need new nodes if the new + directives only produced some content representable by existing nodes. +* A new config value ``todo_include_todos`` (config value names should start + with the extension name, in order to stay unique) that controls whether todo + entries make it into the output. +* New event handlers: one for the :event:`doctree-resolved` event, to replace + the todo and todolist nodes, and one for :event:`env-purge-doc` (the reason + for that will be covered later). + + +The Setup Function +------------------ + +.. currentmodule:: sphinx.application + +The new elements are added in the extension's setup function. Let us create a +new Python module called :file:`todo.py` and add the setup function:: + + def setup(app): + app.add_config_value('todo_include_todos', False, False) + + app.add_node(todolist) + app.add_node(todo, + html=(visit_todo_node, depart_todo_node), + latex=(visit_todo_node, depart_todo_node), + text=(visit_todo_node, depart_todo_node)) + + app.add_directive('todo', TodoDirective) + app.add_directive('todolist', TodolistDirective) + app.connect('doctree-resolved', process_todo_nodes) + app.connect('env-purge-doc', purge_todos) + +The calls in this function refer to classes and functions not yet written. What +the individual calls do is the following: + +* :meth:`~Sphinx.add_config_value` lets Sphinx know that it should recognize the + new *config value* ``todo_include_todos``, whose default value should be + ``False`` (this also tells Sphinx that it is a boolean value). + + If the third argument was ``True``, all documents would be re-read if the + config value changed its value. This is needed for config values that + influence reading (build phase 1). + +* :meth:`~Sphinx.add_node` adds a new *node class* to the build system. It also + can specify visitor functions for each supported output format. These visitor + functions are needed when the new nodes stay until phase 4 -- since the + ``todolist`` node is always replaced in phase 3, it doesn't need any. + + We need to create the two node classes ``todo`` and ``todolist`` later. + +* :meth:`~Sphinx.add_directive` adds a new *directive*, given by name and class. + + The handler functions are created later. + +* Finally, :meth:`~Sphinx.connect` adds an *event handler* to the event whose + name is given by the first argument. The event handler function is called + with several arguments which are documented with the event. + + +The Node Classes +---------------- + +Let's start with the node classes:: + + from docutils import nodes + + class todo(nodes.Admonition, nodes.Element): + pass + + class todolist(nodes.General, nodes.Element): + pass + + def visit_todo_node(self, node): + self.visit_admonition(node) + + def depart_todo_node(self, node): + self.depart_admonition(node) + +Node classes usually don't have to do anything except inherit from the standard +docutils classes defined in :mod:`docutils.nodes`. ``todo`` inherits from +``Admonition`` because it should be handled like a note or warning, ``todolist`` +is just a "general" node. + +.. note:: + + Many extensions will not have to create their own node classes and work fine + with the nodes already provided by `docutils + <http://docutils.sourceforge.net/docs/ref/doctree.html>`__ and :ref:`Sphinx + <nodes>`. + + +The Directive Classes +--------------------- + +A directive class is a class deriving usually from +``docutils.parsers.rst.Directive``. The directive interface is covered in +detail in the `docutils documentation`_; the important thing is that the class +has attributes that configure the allowed markup and a method ``run`` that +returns a list of nodes. + +The ``todolist`` directive is quite simple:: + + from docutils.parsers.rst import Directive + + class TodolistDirective(Directive): + + def run(self): + return [todolist('')] + +An instance of our ``todolist`` node class is created and returned. The +todolist directive has neither content nor arguments that need to be handled. + +The ``todo`` directive function looks like this:: + + from sphinx.util.compat import make_admonition + + class TodoDirective(Directive): + + # this enables content in the directive + has_content = True + + def run(self): + env = self.state.document.settings.env + + targetid = "todo-%d" % env.new_serialno('todo') + targetnode = nodes.target('', '', ids=[targetid]) + + ad = make_admonition(todo, self.name, [_('Todo')], self.options, + self.content, self.lineno, self.content_offset, + self.block_text, self.state, self.state_machine) + + if not hasattr(env, 'todo_all_todos'): + env.todo_all_todos = [] + env.todo_all_todos.append({ + 'docname': env.docname, + 'lineno': self.lineno, + 'todo': ad[0].deepcopy(), + 'target': targetnode, + }) + + return [targetnode] + ad + +Several important things are covered here. First, as you can see, you can refer +to the build environment instance using ``self.state.document.settings.env``. + +Then, to act as a link target (from the todolist), the todo directive needs to +return a target node in addition to the todo node. The target ID (in HTML, this +will be the anchor name) is generated by using ``env.new_serialno`` which +returns a new unique integer on each call and therefore leads to unique target +names. The target node is instantiated without any text (the first two +arguments). + +An admonition is created using a standard docutils function (wrapped in Sphinx +for docutils cross-version compatibility). The first argument gives the node +class, in our case ``todo``. The third argument gives the admonition title (use +``arguments`` here to let the user specify the title). A list of nodes is +returned from ``make_admonition``. + +Then, the todo node is added to the environment. This is needed to be able to +create a list of all todo entries throughout the documentation, in the place +where the author puts a ``todolist`` directive. For this case, the environment +attribute ``todo_all_todos`` is used (again, the name should be unique, so it is +prefixed by the extension name). It does not exist when a new environment is +created, so the directive must check and create it if necessary. Various +information about the todo entry's location are stored along with a copy of the +node. + +In the last line, the nodes that should be put into the doctree are returned: +the target node and the admonition node. + +The node structure that the directive returns looks like this:: + + +--------------------+ + | target node | + +--------------------+ + +--------------------+ + | todo node | + +--------------------+ + \__+--------------------+ + | admonition title | + +--------------------+ + | paragraph | + +--------------------+ + | ... | + +--------------------+ + + +The Event Handlers +------------------ + +Finally, let's look at the event handlers. First, the one for the +:event:`env-purge-doc` event:: + + def purge_todos(app, env, docname): + if not hasattr(env, 'todo_all_todos'): + return + env.todo_all_todos = [todo for todo in env.todo_all_todos + if todo['docname'] != docname] + +Since we store information from source files in the environment, which is +persistent, it may become out of date when the source file changes. Therefore, +before each source file is read, the environment's records of it are cleared, +and the :event:`env-purge-doc` event gives extensions a chance to do the same. +Here we clear out all todos whose docname matches the given one from the +``todo_all_todos`` list. If there are todos left in the document, they will be +added again during parsing. + +The other handler belongs to the :event:`doctree-resolved` event. This event is +emitted at the end of phase 3 and allows custom resolving to be done:: + + def process_todo_nodes(app, doctree, fromdocname): + if not app.config.todo_include_todos: + for node in doctree.traverse(todo): + node.parent.remove(node) + + # Replace all todolist nodes with a list of the collected todos. + # Augment each todo with a backlink to the original location. + env = app.builder.env + + for node in doctree.traverse(todolist): + if not app.config.todo_include_todos: + node.replace_self([]) + continue + + content = [] + + for todo_info in env.todo_all_todos: + para = nodes.paragraph() + filename = env.doc2path(todo_info['docname'], base=None) + description = ( + _('(The original entry is located in %s, line %d and can be found ') % + (filename, todo_info['lineno'])) + para += nodes.Text(description, description) + + # Create a reference + newnode = nodes.reference('', '') + innernode = nodes.emphasis(_('here'), _('here')) + newnode['refdocname'] = todo_info['docname'] + newnode['refuri'] = app.builder.get_relative_uri( + fromdocname, todo_info['docname']) + newnode['refuri'] += '#' + todo_info['target']['refid'] + newnode.append(innernode) + para += newnode + para += nodes.Text('.)', '.)') + + # Insert into the todolist + content.append(todo_info['todo']) + content.append(para) + + node.replace_self(content) + +It is a bit more involved. If our new "todo_include_todos" config value is +false, all todo and todolist nodes are removed from the documents. + +If not, todo nodes just stay where and how they are. Todolist nodes are +replaced by a list of todo entries, complete with backlinks to the location +where they come from. The list items are composed of the nodes from the todo +entry and docutils nodes created on the fly: a paragraph for each entry, +containing text that gives the location, and a link (reference node containing +an italic node) with the backreference. The reference URI is built by +``app.builder.get_relative_uri`` which creates a suitable URI depending on the +used builder, and appending the todo node's (the target's) ID as the anchor +name. + +.. _docutils documentation: http://docutils.sourceforge.net/docs/ref/rst/directives.html |