diff options
257 files changed, 8495 insertions, 7221 deletions
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 4fbf6ff2e..a863f2888 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,5 +1,13 @@ Subject: <what happen when you do on which document project> +<!-- + Important: This is a list of issues for Sphinx, not a forum. + If you'd like to post a question, please move to sphinx-users group. + https://groups.google.com/forum/#!forum/sphinx-users + + Thanks, +--> + ### Problem - <Detail of problem> @@ -40,6 +40,7 @@ Other contributors, listed alphabetically, are: * Timotheus Kampik - JS theme & search enhancements * Dave Kuhlman -- original LaTeX writer * Blaise Laflamme -- pyramid theme +* Chris Lamb -- reproducibility fixes * Thomas Lamb -- linkcheck builder * Łukasz Langa -- partial support for autodoc * Ian Lee -- quickstart improvements @@ -49,6 +50,7 @@ Other contributors, listed alphabetically, are: * Will Maier -- directory HTML builder * Jacob Mason -- websupport library (GSOC project) * Glenn Matthews -- python domain signature improvements +* Kurt McKee -- documentation updates * Roland Meister -- epub builder * Ezio Melotti -- collapsible sidebar JavaScript * Bruce Mitchener -- Minor epub improvement @@ -1,3 +1,127 @@ +Release 1.8.0 (in development) +============================== + +Dependencies +------------ + +Incompatible changes +-------------------- + +* #4460: extensions which stores any data to environment should return the + version of its env data structure as metadata. In detail, please see + :ref:`ext-metadata`. +* Sphinx expects source parser modules to have supported file formats as + ``Parser.supported`` attribute +* The default value of :confval:`epub_author` and :confval:`epub_publisher` are + changed from ``'unknown'`` to the value of :confval:`author`. This is same as + a ``conf.py`` file sphinx-build generates. +* The ``gettext_compact`` attribute is removed from ``document.settings`` + object. Please use ``config.gettext_compact`` instead. +* The processing order on reading phase is changed. smart_quotes, sphinx + domains, :event:`doctree-read` event and versioning doctrees are invoked + earlier than so far. For more details, please read a description of + :py:meth:`.Sphinx.add_transform()` +* #4827: All ``substitution_definition`` nodes are removed from doctree on + reading phase +* ``docutils.conf`` on ``$HOME`` and ``/etc`` directories are ignored. Only + ``docutils.conf`` on confdir is refered. +* #789: ``:samp:`` role supports to escape curly braces with backslash +* #4811: The files under :confval:`html_static_path` are excluded from source + files. +* latex: Use ``\sphinxcite`` for citation references instead ``\hyperref`` +* The config value ``viewcode_import`` is renamed to + :confval:`viewcode_follow_imported_members` (refs: #4035) + +Deprecated +---------- + +* :confval:`source_parsers` is deprecated +* ``Application.import_object()`` is deprecated +* Drop function based directive support. For now, Sphinx only supports class + based directives. +* ``Sphinx.add_source_parser()`` has changed; the *suffix* argument has + been deprecated +* ``sphinx.util.docutils.directive_helper()`` is deprecated +* ``sphinx.cmdline`` is deprecated +* All ``env.update()``, ``env._read_serial()`` and ``env._read_parallel()`` are + deprecated +* ``sphinx.locale.l_()`` is deprecated +* #2157: helper function ``warn()`` for HTML themes is deprecated +* ``env._nitpick_ignore`` is deprecated +* ``app.override_domain()`` is deprecated +* ``app.add_stylesheet()`` is deprecated +* ``sphinx.versioning.prepare()`` is deprecated +* ``Config.__init__()`` has changed; the *dirname*, *filename* and *tags* + argument has been deprecated +* ``Config.check_types()`` is deprecated +* ``Config.check_unicode()`` is deprecated +* ``sphinx.application.CONFIG_FILENAME`` is deprecated +* ``highlightlang`` directive is deprecated +* ``env.read_doc()`` is deprecated +* ``sphinx.writers.latex.Table.caption_footnotetexts`` is deprecated +* ``sphinx.writers.latex.Table.header_footnotetexts`` is deprecated +* ``sphinx.writers.latex.LaTeXWriter.footnotestack`` is deprecated +* ``sphinx.writers.latex.LaTeXWriter.restrict_footnote()`` is deprecated +* ``sphinx.writers.latex.LaTeXWriter.unrestrict_footnote()`` is deprecated +* ``LaTeXWriter.bibitems`` is deprecated +* ``BuildEnvironment.load()`` is deprecated +* ``BuildEnvironment.loads()`` is deprecated +* ``BuildEnvironment.frompickle()`` is deprecated +* ``BuildEnvironment.dump()`` is deprecated +* ``BuildEnvironment.dumps()`` is deprecated +* ``BuildEnvironment.topickle()`` is deprecated + +For more details, see `deprecation APIs list +<http://www.sphinx-doc.org/en/master/extdev/index.html#deprecated-apis>`_ + +Features added +-------------- + +* Add :event:`config-inited` event +* Add ``sphinx.config.Any`` to represent the config value accepts any type of + value +* :confval:`source_suffix` allows a mapping fileext to file types +* Add :confval:`author` as a configuration value +* #2852: imgconverter: Support to convert GIF to PNG +* ``sphinx-build`` command supports i18n console output +* Add ``app.add_message_catalog()`` and ``sphinx.locale.get_translations()`` to + support translation for 3rd party extensions +* helper function ``warning()`` for HTML themes is added +* Add ``Domain.enumerable_nodes`` to manage own enumerable nodes for domains + (experimental) +* Add a new keyword argument ``override`` to Application APIs +* LaTeX: new key ``'fvset'`` for :confval:`latex_elements`. For + XeLaTeX/LuaLaTeX its default sets ``fanvyvrb`` to use normal, not small, + fontsize in code-blocks (refs: #4793) +* Add :confval:`html_css_files` and :confval:`epub_css_files` for adding CSS + files from configuration +* #4834: Ensure set object descriptions are reproducible. +* #4828: Allow to override :confval:`numfig_format` partially. Full definition + is not needed. +* Improve warning messages during including (refs: #4818) +* LaTeX: separate customizability of :rst:role:`guilabel` and + :rst:role:`menuselection` (refs: #4830) +* Add ``Config.read()`` classmethod to create a new config object from + configuration file +* #4866: Wrap graphviz diagrams in ``<div>`` tag +* Add :event:`viewcode-find-source` event to viewcode extension. + +Bugs fixed +---------- + +* i18n: message catalogs were reset on each initialization +* #4850: latex: footnote inside footnote was not rendered +* #4945: i18n: fix lang_COUNTRY not fallback correctly for IndexBuilder. Thanks + to Shengjing Zhu. + +Testing +-------- + +Features removed +---------------- + +* ``sphinx.ext.pngmath`` extension + Release 1.7.5 (in development) ============================== @@ -245,6 +369,8 @@ Features added * #4271: sphinx-build supports an option called ``-j auto`` to adjust numbers of processes automatically. +* Napoleon: added option to specify custom section tags. + Features removed ---------------- @@ -317,7 +317,7 @@ Documentation using a custom theme or integrated in a website * PSI4: http://www.psicode.org/psi4manual/master/index.html * PyMOTW: https://pymotw.com/2/ * python-aspectlib: https://python-aspectlib.readthedocs.io/ - (`sphinx_py3doc_enhanced_theme <https://pypi.python.org/pypi/sphinx_py3doc_enhanced_theme>`__) + (`sphinx_py3doc_enhanced_theme <https://pypi.org/project/sphinx_py3doc_enhanced_theme/>`__) * QGIS: https://qgis.org/en/docs/index.html * qooxdoo: http://www.qooxdoo.org/current/ * Roundup: http://www.roundup-tracker.org/ @@ -366,6 +366,7 @@ Books produced using Sphinx https://www.packtpub.com/application-development/expert-python-programming * "Expert Python Programming" (Japanese translation): https://www.amazon.co.jp/dp/4048686291/ +* "The Hitchhiker's Guide to Python": http://docs.python-guide.org/en/latest/ * "LassoGuide": http://www.lassosoft.com/Lasso-Documentation * "Learning Sphinx" (in Japanese): https://www.oreilly.co.jp/books/9784873116488/ @@ -104,36 +104,6 @@ smartypants.py license:: such damage. ---------------------------------------------------------------------- -The ElementTree package, included in this distribution in -test/etree13, is available under the following license: - ----------------------------------------------------------------------- -The ElementTree toolkit is - -Copyright (c) 1999-2007 by Fredrik Lundh - -By obtaining, using, and/or copying this software and/or its -associated documentation, you agree that you have read, understood, -and will comply with the following terms and conditions: - -Permission to use, copy, modify, and distribute this software and its -associated documentation for any purpose and without fee is hereby -granted, provided that the above copyright notice appears in all -copies, and that both that copyright notice and this permission notice -appear in supporting documentation, and that the name of Secret Labs -AB or the author not be used in advertising or publicity pertaining to -distribution of the software without specific, written prior -permission. - -SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO -THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- ABILITY -AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE -FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES -WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN -ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT -OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ----------------------------------------------------------------------- - The included JQuery JavaScript library is available under the MIT license: @@ -3,14 +3,6 @@ PYTHON ?= python .PHONY: all all: clean-pyc clean-backupfiles style-check type-check test -.PHONY: style-check -style-check: - @flake8 - -.PHONY: type-check -type-check: - mypy sphinx/ - .PHONY: clean clean: clean-pyc clean-pycache clean-patchfiles clean-backupfiles clean-generated clean-testfiles clean-buildfiles clean-mypyfiles @@ -59,22 +51,22 @@ clean-buildfiles: clean-mypyfiles: rm -rf .mypy_cache/ +.PHONY: style-check +style-check: + @flake8 + +.PHONY: type-check +type-check: + mypy sphinx/ + .PHONY: pylint pylint: @pylint --rcfile utils/pylintrc sphinx -.PHONY: reindent -reindent: - @echo "This target no longer does anything and will be removed imminently" - .PHONY: test test: @$(PYTHON) -m pytest -v $(TEST) -.PHONY: test-async -test-async: - @echo "This target no longer does anything and will be removed imminently" - .PHONY: covertest covertest: @$(PYTHON) -m pytest -v --cov=sphinx --junitxml=.junit.xml $(TEST) @@ -86,6 +78,6 @@ build: .PHONY: docs docs: ifndef target - $(info You need to give a provide a target variable, e.g. `make docs target=html`.) + $(info You need to give a provide a target variable, e.g. `make docs target=html`.) endif - $(MAKE) -C doc $(target) + $(MAKE) -C doc $(target) diff --git a/README.rst b/README.rst index 82f61710b..9d9ffba6e 100644 --- a/README.rst +++ b/README.rst @@ -3,10 +3,10 @@ ======== .. image:: https://img.shields.io/pypi/v/sphinx.svg - :target: https://pypi.python.org/pypi/Sphinx + :target: https://pypi.org/project/Sphinx/ :alt: Package on PyPi -.. image:: https://readthedocs.org/projects/sphinx/badge/ +.. image:: https://readthedocs.org/projects/sphinx/badge/?version=master :target: http://www.sphinx-doc.org/ :alt: Documentation Status @@ -71,7 +71,7 @@ We also publish beta releases:: If you wish to install `Sphinx` for development purposes, refer to `the contributors guide`__. -__ https://pypi.python.org/pypi/Sphinx +__ https://pypi.org/project/Sphinx/ __ http://www.sphinx-doc.org/en/master/devguide.html Documentation @@ -81,6 +81,15 @@ Documentation is available from `sphinx-doc.org`__. __ http://www.sphinx-doc.org/ +Get in touch +============ + +- Report bugs, suggest features or view the source code `on GitHub`_. +- For less well defined questions or ideas, use the `mailing list`_. + +.. _on GitHub: https://github.com/sphinx-doc/sphinx +.. _mailing list: https://groups.google.com/forum/#!forum/sphinx-users + Testing ======= diff --git a/doc/more.png b/doc/_static/more.png Binary files differindex 97553a8b7..97553a8b7 100644 --- a/doc/more.png +++ b/doc/_static/more.png diff --git a/doc/_templates/index.html b/doc/_templates/index.html index 5a8a2f025..032636d94 100644 --- a/doc/_templates/index.html +++ b/doc/_templates/index.html @@ -49,7 +49,7 @@ <table class="contentstable"> <tr> <td> - <p class="biglink"><a class="biglink" href="{{ pathto("tutorial") }}">{%trans%}First steps with Sphinx{%endtrans%}</a><br/> + <p class="biglink"><a class="biglink" href="{{ pathto("usage/quickstart") }}">{%trans%}First steps with Sphinx{%endtrans%}</a><br/> <span class="linkdescr">{%trans%}overview of basic tasks{%endtrans%}</span></p> </td><td> {%- if hasdoc('search') %}<p class="biglink"><a class="biglink" href="{{ pathto("search") }}">{%trans%}Search page{%endtrans%}</a><br/> @@ -75,8 +75,8 @@ <p>{%trans%} You can also download PDF/EPUB versions of the Sphinx documentation: a <a href="https://media.readthedocs.org/pdf/sphinx/stable/sphinx.pdf">PDF version</a> generated from - the LaTeX Sphinx produces, and - a <a href="https://media.readthedocs.org/epub/sphinx/stable/sphinx.epub">EPUB version</a>. + the LaTeX Sphinx producer, and + an <a href="https://media.readthedocs.org/epub/sphinx/stable/sphinx.epub">EPUB version</a>. {%endtrans%} </p> @@ -98,7 +98,7 @@ <p>{%trans%}There is a <a href="http://docs.sphinx-users.jp/">Japanese translation</a> of this documentation, thanks to the Japanese Sphinx user group.{%endtrans%}</p> <p>{%trans%}A Japanese book about Sphinx has been published by O'Reilly: - <a href="http://www.oreilly.co.jp/books/9784873116488/">Sphinxをはじめよう / + <a href="https://www.oreilly.co.jp/books/9784873116488/">Sphinxをはじめよう / Learning Sphinx</a>.{%endtrans%}</p> <!-- <p><img src="{{ pathto("_static/bookcover.png", 1) }}"/></p> --> diff --git a/doc/_templates/indexsidebar.html b/doc/_templates/indexsidebar.html index b07ef2033..e1747c762 100644 --- a/doc/_templates/indexsidebar.html +++ b/doc/_templates/indexsidebar.html @@ -1,4 +1,4 @@ -<p class="logo">A <a href="http://pocoo.org/"> +<p class="logo">A <a href="https://www.pocoo.org/"> <img src="{{ pathto("_static/pocoo.png", 1) }}" alt="Pocoo" /></a> {%trans%}project{%endtrans%}</p> @@ -8,20 +8,20 @@ not released yet.{%endtrans%}</p> <p>{%trans%}You can use it from the <a href="https://github.com/sphinx-doc/sphinx/">Git repo</a> or look for - released versions in the <a href="https://pypi.python.org/pypi/Sphinx">Python + released versions in the <a href="https://pypi.org/project/Sphinx/">Python Package Index</a>.{%endtrans%}</p> {% else %} <p>{%trans%}Current version: <b><a href="changes.html">{{ version }}</a></b>{%endtrans%}</p> -<p>{%trans%}Get Sphinx from the <a href="https://pypi.python.org/pypi/Sphinx">Python Package +<p>{%trans%}Get Sphinx from the <a href="https://pypi.org/project/Sphinx/">Python Package Index</a>, or install it with:{%endtrans%}</p> <pre>pip install -U Sphinx</pre> {% endif %} <h3>{%trans%}Questions? Suggestions?{%endtrans%}</h3> -<p>{%trans%}Join the <a href="http://groups.google.com/group/sphinx-users">sphinx-users</a> mailing list on Google Groups:{%endtrans%}</p> +<p>{%trans%}Join the <a href="https://groups.google.com/group/sphinx-users">sphinx-users</a> mailing list on Google Groups:{%endtrans%}</p> <div class="subscribeformwrapper"> -<form action="http://groups.google.com/group/sphinx-users/boxsubscribe" +<form action="https://groups.google.com/group/sphinx-users/boxsubscribe" class="subscribeform"> <input type="text" name="email" value="your@email" onfocus="$(this).val('');" /> diff --git a/doc/_themes/sphinx13/layout.html b/doc/_themes/sphinx13/layout.html index 597b6261f..8967d8265 100644 --- a/doc/_themes/sphinx13/layout.html +++ b/doc/_themes/sphinx13/layout.html @@ -70,7 +70,7 @@ <div class="pageheader"> <ul> <li><a href="{{ pathto('index') }}">Home</a></li> - <li><a href="{{ pathto('install') }}">Get it</a></li> + <li><a href="{{ pathto('usage/installation') }}">Get it</a></li> <li><a href="{{ pathto('contents') }}">Docs</a></li> <li><a href="{{ pathto('develop') }}">Extend/Develop</a></li> </ul> diff --git a/doc/_themes/sphinx13/static/sphinx13.css b/doc/_themes/sphinx13/static/sphinx13.css index 24a33fba7..eff18df3c 100644 --- a/doc/_themes/sphinx13/static/sphinx13.css +++ b/doc/_themes/sphinx13/static/sphinx13.css @@ -302,6 +302,10 @@ cite, code, tt { letter-spacing: -0.02em; } +table.deprecated code.literal { + word-break: break-all; +} + tt { background-color: #f2f2f2; border: 1px solid #ddd; diff --git a/doc/builders.rst b/doc/builders.rst index 1a6c24243..d968f209f 100644 --- a/doc/builders.rst +++ b/doc/builders.rst @@ -210,16 +210,10 @@ The builder's "name" must be given to the **-b** command-line option of .. autoattribute:: supported_image_types Note that a direct PDF builder is being provided by `rinohtype`_. The builder's -name is ``rinoh``. Refer to the `rinohtype manual`_ for details. There is also -PDF builder using ReportLab in `rst2pdf`_ version 0.12 or greater. However, -rst2pdf is no longer being actively maintained and suffers from some problems -when used with recent Sphinx versions. See the `rst2pdf manual`_ for usage -instructions. +name is ``rinoh``. Refer to the `rinohtype manual`_ for details. .. _rinohtype: https://github.com/brechtm/rinohtype .. _rinohtype manual: http://www.mos6581.org/rinohtype/quickstart.html#sphinx-builder -.. _rst2pdf: https://github.com/rst2pdf/rst2pdf -.. _rst2pdf manual: http://ralsina.me/static/manual.pdf .. module:: sphinx.builders.text .. class:: TextBuilder @@ -294,7 +288,7 @@ instructions. globalcontext_filename = 'globalcontext.phpdump' searchindex_filename = 'searchindex.phpdump' - .. _PHP serialization: https://pypi.python.org/pypi/phpserialize + .. _PHP serialization: https://pypi.org/project/phpserialize/ .. attribute:: implementation diff --git a/doc/conf.py b/doc/conf.py index 43b0d72b3..18d82b223 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -39,7 +39,7 @@ epub_uid = 'web-site' epub_scheme = 'url' epub_identifier = epub_publisher epub_pre_files = [('index.xhtml', 'Welcome')] -epub_post_files = [('install.xhtml', 'Installing Sphinx'), +epub_post_files = [('usage/installation.xhtml', 'Installing Sphinx'), ('develop.xhtml', 'Sphinx development')] epub_exclude_files = ['_static/opensearch.xml', '_static/doctools.js', '_static/jquery.js', '_static/searchtools.js', @@ -62,7 +62,8 @@ latex_elements = { \usepackage{courier} ''', 'passoptionstopackages': '\\PassOptionsToPackage{svgnames}{xcolor}', - 'preamble': '\\fvset{fontsize=auto}', + 'preamble': '\\DeclareUnicodeCharacter{229E}{\\ensuremath{\\boxplus}}', + 'fvset': '\\fvset{fontsize=auto}', # fix missing index entry due to RTD doing only once pdflatex after makeindex 'printindex': r''' \IfFileExists{\jobname.ind} diff --git a/doc/contents.rst b/doc/contents.rst index ab8d09eb0..77dcf6c42 100644 --- a/doc/contents.rst +++ b/doc/contents.rst @@ -7,20 +7,20 @@ Sphinx documentation contents .. toctree:: :maxdepth: 2 + usage/installation + usage/quickstart + usage/restructuredtext/index + usage/markdown + usage/configuration + intro - tutorial man/index - rest - markup/index - domains builders - config intl theming setuptools templating latex - markdown extensions extdev/index websupport @@ -33,6 +33,7 @@ Sphinx documentation contents authors + Indices and tables ================== diff --git a/doc/develop.rst b/doc/develop.rst index 19ca81ef9..d2a51b8e2 100644 --- a/doc/develop.rst +++ b/doc/develop.rst @@ -12,7 +12,7 @@ Sphinx is a maintained by a group of volunteers. We value every contribution! * The mailing list for development is at `Google Groups <https://groups.google.com/forum/#!forum/sphinx-dev>`_. * There is also the #sphinx-doc IRC channel on `freenode - <http://freenode.net/>`_. + <https://freenode.net/>`_. For more about our development process and methods, see the :doc:`devguide`. @@ -106,40 +106,40 @@ own extensions. .. _aafigure: https://launchpad.net/aafigure .. _gnuplot: http://www.gnuplot.info/ -.. _paver: http://paver.github.io/paver/ -.. _Sword: http://www.crosswire.org/sword/ +.. _paver: https://paver.readthedocs.io/en/latest/ +.. _Sword: https://www.crosswire.org/sword/ .. _Lilypond: http://lilypond.org/ .. _sdedit: http://sdedit.sourceforge.net/ -.. _Trac: http://trac.edgewall.org -.. _TracLinks: http://trac.edgewall.org/wiki/TracLinks -.. _OmegaT: http://www.omegat.org/ +.. _Trac: https://trac.edgewall.org/ +.. _TracLinks: https://trac.edgewall.org/wiki/TracLinks +.. _OmegaT: https://omegat.org/ .. _PlantUML: http://plantuml.com/ -.. _PyEnchant: http://www.rfk.id.au/software/pyenchant/ +.. _PyEnchant: https://pythonhosted.org/pyenchant/ .. _sadisplay: https://bitbucket.org/estin/sadisplay/wiki/Home .. _blockdiag: http://blockdiag.com/en/ .. _seqdiag: http://blockdiag.com/en/ .. _actdiag: http://blockdiag.com/en/ .. _nwdiag: http://blockdiag.com/en/ -.. _Google Analytics: http://www.google.com/analytics/ +.. _Google Analytics: https://www.google.com/analytics/ .. _Google Chart: https://developers.google.com/chart/ .. _Google Maps: https://www.google.com/maps .. _Google style: https://google.github.io/styleguide/pyguide.html .. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt .. _hyphenator: https://github.com/mnater/hyphenator -.. _exceltable: http://pythonhosted.org/sphinxcontrib-exceltable/ +.. _exceltable: https://pythonhosted.org/sphinxcontrib-exceltable/ .. _YouTube: http://www.youtube.com/ -.. _ClearQuest: http://www-03.ibm.com/software/products/en/clearquest -.. _Zope interfaces: http://docs.zope.org/zope.interface/README.html -.. _slideshare: http://www.slideshare.net/ +.. _ClearQuest: https://www.ibm.com/us-en/marketplace/rational-clearquest +.. _Zope interfaces: https://zopeinterface.readthedocs.io/en/latest/README.html +.. _slideshare: https://www.slideshare.net/ .. _TikZ/PGF LaTeX package: https://sourceforge.net/projects/pgf/ -.. _MATLAB: http://www.mathworks.com/products/matlab/ +.. _MATLAB: https://www.mathworks.com/products/matlab.html .. _swf: https://bitbucket.org/klorenz/sphinxcontrib-swf .. _findanything: https://bitbucket.org/klorenz/sphinxcontrib-findanything .. _cmakedomain: https://bitbucket.org/klorenz/sphinxcontrib-cmakedomain -.. _GNU Make: http://www.gnu.org/software/make/ +.. _GNU Make: https://www.gnu.org/software/make/ .. _makedomain: https://bitbucket.org/klorenz/sphinxcontrib-makedomain .. _inlinesyntaxhighlight: https://sphinxcontrib-inlinesyntaxhighlight.readthedocs.io/ .. _CMake: https://cmake.org .. _domaintools: https://bitbucket.org/klorenz/sphinxcontrib-domaintools -.. _restbuilder: https://pypi.python.org/pypi/sphinxcontrib-restbuilder +.. _restbuilder: https://pypi.org/project/sphinxcontrib-restbuilder/ .. _Lasso: http://www.lassosoft.com/ diff --git a/doc/ext/autodoc.rst b/doc/ext/autodoc.rst index 09098f39c..0bddba5c5 100644 --- a/doc/ext/autodoc.rst +++ b/doc/ext/autodoc.rst @@ -398,7 +398,7 @@ There are also new config values that you can set: .. confval:: autodoc_inherit_docstrings This value controls the docstrings inheritance. - If set to True the cocstring for classes or methods, if not explicitly set, + If set to True the docstring for classes or methods, if not explicitly set, is inherited form parents. The default is ``True``. diff --git a/doc/ext/example_google.py b/doc/ext/example_google.py index 5b4fa58df..4f6abacdf 100644 --- a/doc/ext/example_google.py +++ b/doc/ext/example_google.py @@ -29,7 +29,7 @@ Todo: * You have to also use ``sphinx.ext.todo`` extension .. _Google Python Style Guide: - http://google.github.io/styleguide/pyguide.html + https://google.github.io/styleguide/pyguide.html """ diff --git a/doc/ext/graphviz.rst b/doc/ext/graphviz.rst index ef0483da7..8b9fff90b 100644 --- a/doc/ext/graphviz.rst +++ b/doc/ext/graphviz.rst @@ -8,7 +8,7 @@ .. versionadded:: 0.6 -This extension allows you to embed `Graphviz <http://graphviz.org/>`_ graphs in +This extension allows you to embed `Graphviz <https://graphviz.org/>`_ graphs in your documents. It adds these directives: diff --git a/doc/ext/imgconverter.rst b/doc/ext/imgconverter.rst index 4c43bdf8e..322a9b5f3 100644 --- a/doc/ext/imgconverter.rst +++ b/doc/ext/imgconverter.rst @@ -1,7 +1,9 @@ .. highlight:: rest -:mod:`sphinx.ext.imgconverter` -- Convert images to appropriate format for builders -=================================================================================== +.. _sphinx.ext.imgconverter: + +:mod:`sphinx.ext.imgconverter` -- A reference implementation for image converter using Imagemagick +================================================================================================== .. module:: sphinx.ext.imgconverter :synopsis: Convert images to appropriate format for builders @@ -16,6 +18,12 @@ Internally, this extension uses Imagemagick_ to convert images. .. _Imagemagick: https://www.imagemagick.org/script/index.php +.. note:: Imagemagick rasterizes a SVG image on conversion. As a result, the image + becomes not scalable. To avoid that, please use other image converters + like sphinxcontrib-svg2pdfconverter_ (which uses Inkscape or rsvg-convert). + +.. _sphinxcontrib-svg2pdfconverter: https://github.com/missinglinkelectronics/sphinxcontrib-svg2pdfconverter + Configuration ------------- diff --git a/doc/ext/intersphinx.rst b/doc/ext/intersphinx.rst index fb0114fc4..b8ae104b3 100644 --- a/doc/ext/intersphinx.rst +++ b/doc/ext/intersphinx.rst @@ -38,7 +38,10 @@ Behind the scenes, this works as follows: specified individually, e.g. if the docs should be buildable without Internet access. -To use intersphinx linking, add ``'sphinx.ext.intersphinx'`` to your +Configuring Intersphinx +----------------------- + +To use Intersphinx linking, add ``'sphinx.ext.intersphinx'`` to your :confval:`extensions` config value, and use these new config values to activate linking: @@ -84,7 +87,7 @@ linking: To add links to modules and objects in the Python standard library documentation, use:: - intersphinx_mapping = {'python': ('https://docs.python.org/3.4', None)} + intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} This will download the corresponding :file:`objects.inv` file from the Internet and generate links to the pages under the given URI. The downloaded @@ -94,12 +97,12 @@ linking: A second example, showing the meaning of a non-``None`` value of the second tuple item:: - intersphinx_mapping = {'python': ('https://docs.python.org/3.4', + intersphinx_mapping = {'python': ('https://docs.python.org/3', 'python-inv.txt')} This will read the inventory from :file:`python-inv.txt` in the source directory, but still generate links to the pages under - ``https://docs.python.org/3.4``. It is up to you to update the inventory file + ``https://docs.python.org/3``. It is up to you to update the inventory file as new objects are added to the Python documentation. **Multiple target for the inventory** @@ -113,7 +116,7 @@ linking: this to specify mirror sites for server downtime of the primary inventory:: - intersphinx_mapping = {'python': ('https://docs.python.org/3.4', + intersphinx_mapping = {'python': ('https://docs.python.org/3', (None, 'python-inv.txt'))} .. confval:: intersphinx_cache_limit @@ -132,3 +135,13 @@ linking: timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds. + +Showing all links of an Intersphinx mapping file +------------------------------------------------ + +To show all Intersphinx links and their targets of an Intersphinx mapping file, run +``python -msphinx.ext.intersphinx url-or-path``. This is helpful when searching for the root cause of a broken +Intersphinx link in a documentation project. The following example prints the Intersphinx mapping of the Python 3 +documentation:: + + $ python -msphinx.ext.intersphinx https://docs.python.org/3/objects.inv diff --git a/doc/ext/linkcode.rst b/doc/ext/linkcode.rst index 05d2cc6db..f00345fca 100644 --- a/doc/ext/linkcode.rst +++ b/doc/ext/linkcode.rst @@ -44,4 +44,4 @@ function that returns an URL based on the object. if not info['module']: return None filename = info['module'].replace('.', '/') - return "http://somesite/sourcerepo/%s.py" % filename + return "https://somesite/sourcerepo/%s.py" % filename diff --git a/doc/ext/math.rst b/doc/ext/math.rst index 4097bb29e..3e4f94c6c 100644 --- a/doc/ext/math.rst +++ b/doc/ext/math.rst @@ -248,7 +248,7 @@ Sphinx. __ https://cdjns.com - __ http://docs.mathjax.org/en/latest/start.html + __ https://docs.mathjax.org/en/latest/start.html The path can be absolute or relative; if it is relative, it is relative to the ``_static`` directory of the built docs. @@ -258,7 +258,7 @@ Sphinx. documentation set on one server, it is advisable to install MathJax in a shared location. - You can also give a full ``http://`` URL different from the CDN URL. + You can also give a full ``https://`` URL different from the CDN URL. :mod:`sphinx.ext.jsmath` -- Render math via JavaScript @@ -284,9 +284,9 @@ package jsMath_. It provides this config value: a shared location. -.. _dvipng: http://savannah.nongnu.org/projects/dvipng/ +.. _dvipng: https://savannah.nongnu.org/projects/dvipng/ .. _dvisvgm: http://dvisvgm.bplaced.net/ .. _MathJax: https://www.mathjax.org/ .. _jsMath: http://www.math.union.edu/~dpvc/jsmath/ -.. _preview-latex package: http://www.gnu.org/software/auctex/preview-latex.html -.. _AmSMath LaTeX package: http://www.ams.org/publications/authors/tex/amslatex +.. _preview-latex package: https://www.gnu.org/software/auctex/preview-latex.html +.. _AmSMath LaTeX package: https://www.ams.org/publications/authors/tex/amslatex diff --git a/doc/ext/napoleon.rst b/doc/ext/napoleon.rst index f7e9081f7..6c5f0d61a 100644 --- a/doc/ext/napoleon.rst +++ b/doc/ext/napoleon.rst @@ -52,9 +52,9 @@ source code files. .. _ReStructuredText: http://docutils.sourceforge.net/rst.html .. _docstrings: https://www.python.org/dev/peps/pep-0287/ .. _Google Python Style Guide: - http://google.github.io/styleguide/pyguide.html + https://google.github.io/styleguide/pyguide.html .. _Google: - http://google.github.io/styleguide/pyguide.html#Comments + https://google.github.io/styleguide/pyguide.html#Comments .. _NumPy: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt .. _Khan Academy: @@ -63,8 +63,8 @@ source code files. Getting Started --------------- -1. After :doc:`setting up Sphinx <../tutorial>` to build your docs, enable - napoleon in the Sphinx `conf.py` file:: +1. After :doc:`setting up Sphinx </usage/quickstart>` to build your docs, + enable napoleon in the Sphinx `conf.py` file:: # conf.py @@ -267,7 +267,7 @@ sure that "sphinx.ext.napoleon" is enabled in `conf.py`:: napoleon_use_rtype = True .. _Google style: - http://google-styleguide.googlecode.com/svn/trunk/pyguide.html + https://google.github.io/styleguide/pyguide.html .. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt diff --git a/doc/ext/thirdparty.rst b/doc/ext/thirdparty.rst index 40c24246a..20dda9ff1 100644 --- a/doc/ext/thirdparty.rst +++ b/doc/ext/thirdparty.rst @@ -6,8 +6,9 @@ repository. It is open for anyone who wants to maintain an extension publicly; just send a short message asking for write permissions. There are also several extensions hosted elsewhere. The `Sphinx extension -survey <https://sphinxext-survey.readthedocs.io/>`__ contains a -comprehensive list. +survey <https://sphinxext-survey.readthedocs.io/>`__ and `awesome-sphinxdoc +<https://github.com/yoloseem/awesome-sphinxdoc>`__ contains a comprehensive +list. If you write an extension that you think others will find useful or you think should be included as a part of Sphinx, please write to the project mailing diff --git a/doc/ext/viewcode.rst b/doc/ext/viewcode.rst index 5823090f6..87071144f 100644 --- a/doc/ext/viewcode.rst +++ b/doc/ext/viewcode.rst @@ -15,6 +15,19 @@ a highlighted version of the source code, and a link will be added to all object descriptions that leads to the source code of the described object. A link back from the source to the description will also be inserted. +.. warning:: + + Basically, ``viewcode`` extension will import the modules being linked to. + If any modules have side effects on import, these will be executed when + ``sphinx-build`` is run. + + If you document scripts (as opposed to library modules), make sure their + main routine is protected by a ``if __name__ == '__main__'`` condition. + + In addition, if you don't want to import the modules by ``viewcode``, + you can tell the location of the location of source code to ``viewcode`` + using :event:`viewcode-find-source` event. + This extension works only on HTML related builders like ``html``, ``applehelp``, ``devhelp``, ``htmlhelp``, ``qthelp`` and so on except ``singlehtml``. By default ``epub`` builder doesn't @@ -22,24 +35,18 @@ support this extension (see :confval:`viewcode_enable_epub`). There is an additional config value: -.. confval:: viewcode_import +.. confval:: viewcode_follow_imported_members If this is ``True``, viewcode extension will follow alias objects that imported from another module such as functions, classes and attributes. As side effects, this option else they produce nothing. The default is ``True``. - .. warning:: - - :confval:`viewcode_import` **imports** the modules to be followed real - location. If any modules have side effects on import, these will be - executed by ``viewcode`` when ``sphinx-build`` is run. - - If you document scripts (as opposed to library modules), make sure their - main routine is protected by a ``if __name__ == '__main__'`` condition. - .. versionadded:: 1.3 + .. versionchanged:: 1.8 + Renamed from ``viewcode_import`` to ``viewcode_follow_imported_members``. + .. confval:: viewcode_enable_epub If this is ``True``, viewcode extension is also enabled even if you use @@ -62,3 +69,17 @@ There is an additional config value: Some reader's rendering result are corrupted and `epubcheck <https://github.com/IDPF/epubcheck>`_'s score becomes worse even if the reader supports. + +.. event:: viewcode-find-source (app, modname) + + .. versionadded:: 1.8 + + Find the source code for a module. + An event handler for this event should return + a tuple of the source code itself and a dictionary of tags. + The dictionary maps the name of a class, function, attribute, etc + to a tuple of its type, the start line number, and the end line number. + The type should be one of "class", "def", or "other". + + :param app: The Sphinx application object. + :param modname: The name of the module to find source code for. diff --git a/doc/extdev/appapi.rst b/doc/extdev/appapi.rst index ce856d031..584cf375b 100644 --- a/doc/extdev/appapi.rst +++ b/doc/extdev/appapi.rst @@ -7,9 +7,9 @@ Application API :synopsis: Application class and extensibility interface. -Each Sphinx extension is a Python module with at least a :func:`setup` function. -This function is called at initialization time with one argument, the -application object representing the Sphinx process. +Each Sphinx extension is a Python module with at least a :func:`setup` +function. This function is called at initialization time with one argument, +the application object representing the Sphinx process. .. class:: Sphinx @@ -23,387 +23,79 @@ These methods are usually called in an extension's ``setup()`` function. Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext` package. -.. method:: Sphinx.setup_extension(name) - - Load the extension given by the module *name*. Use this if your extension - needs the features provided by another extension. - -.. method:: Sphinx.add_builder(builder) - - Register a new builder. *builder* must be a class that inherits from - :class:`~sphinx.builders.Builder`. - -.. method:: Sphinx.add_config_value(name, default, rebuild) - - Register a configuration value. This is necessary for Sphinx to recognize - new values and set default values accordingly. The *name* should be prefixed - with the extension name, to avoid clashes. The *default* value can be any - Python object. The string value *rebuild* must be one of those values: - - * ``'env'`` if a change in the setting only takes effect when a document is - parsed -- this means that the whole environment must be rebuilt. - * ``'html'`` if a change in the setting needs a full rebuild of HTML - documents. - * ``''`` if a change in the setting will not need any special rebuild. - - .. versionchanged:: 0.4 - If the *default* value is a callable, it will be called with the config - object as its argument in order to get the default value. This can be - used to implement config values whose default depends on other values. - - .. versionchanged:: 0.6 - Changed *rebuild* from a simple boolean (equivalent to ``''`` or - ``'env'``) to a string. However, booleans are still accepted and - converted internally. - -.. method:: Sphinx.add_domain(domain) - - Make the given *domain* (which must be a class; more precisely, a subclass of - :class:`~sphinx.domains.Domain`) known to Sphinx. - - .. versionadded:: 1.0 - -.. method:: Sphinx.override_domain(domain) - - Make the given *domain* class known to Sphinx, assuming that there is already - a domain with its ``.name``. The new domain must be a subclass of the - existing one. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_index_to_domain(domain, index) - - Add a custom *index* class to the domain named *domain*. *index* must be a - subclass of :class:`~sphinx.domains.Index`. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_event(name) - - Register an event called *name*. This is needed to be able to emit it. - -.. method:: Sphinx.set_translator(name, translator_class) - - Register or override a Docutils translator class. This is used to register - a custom output translator or to replace a builtin translator. - This allows extensions to use custom translator and define custom - nodes for the translator (see :meth:`add_node`). - - .. versionadded:: 1.3 - -.. method:: Sphinx.add_node(node, **kwds) - - Register a Docutils node class. This is necessary for Docutils internals. - It may also be used in the future to validate nodes in the parsed documents. - - Node visitor functions for the Sphinx HTML, LaTeX, text and manpage writers - can be given as keyword arguments: the keyword should be one or more of - ``'html'``, ``'latex'``, ``'text'``, ``'man'``, ``'texinfo'`` or any other - supported translators, the value a 2-tuple of ``(visit, depart)`` methods. - ``depart`` can be ``None`` if the ``visit`` function raises - :exc:`docutils.nodes.SkipNode`. Example: - - .. code-block:: python - - class math(docutils.nodes.Element): pass - - def visit_math_html(self, node): - self.body.append(self.starttag(node, 'math')) - def depart_math_html(self, node): - self.body.append('</math>') - - app.add_node(math, html=(visit_math_html, depart_math_html)) - - Obviously, translators for which you don't specify visitor methods will choke - on the node when encountered in a document to translate. - - .. versionchanged:: 0.5 - Added the support for keyword arguments giving visit functions. - -.. method:: Sphinx.add_enumerable_node(node, figtype, title_getter=None, **kwds) - - Register a Docutils node class as a numfig target. Sphinx numbers the node - automatically. And then the users can refer it using :rst:role:`numref`. - - *figtype* is a type of enumerable nodes. Each figtypes have individual - numbering sequences. As a system figtypes, ``figure``, ``table`` and - ``code-block`` are defined. It is able to add custom nodes to these - default figtypes. It is also able to define new custom figtype if new - figtype is given. - - *title_getter* is a getter function to obtain the title of node. It takes - an instance of the enumerable node, and it must return its title as string. - The title is used to the default title of references for :rst:role:`ref`. - By default, Sphinx searches ``docutils.nodes.caption`` or - ``docutils.nodes.title`` from the node as a title. - - Other keyword arguments are used for node visitor functions. See the - :meth:`Sphinx.add_node` for details. - - .. versionadded:: 1.4 - -.. method:: Sphinx.add_directive(name, func, content, arguments, **options) - Sphinx.add_directive(name, directiveclass) - - Register a Docutils directive. *name* must be the prospective directive - name. There are two possible ways to write a directive: - - * In the docutils 0.4 style, *obj* is the directive function. *content*, - *arguments* and *options* are set as attributes on the function and - determine whether the directive has content, arguments and options, - respectively. **This style is deprecated.** - - * In the docutils 0.5 style, *directiveclass* is the directive class. It - must already have attributes named *has_content*, *required_arguments*, - *optional_arguments*, *final_argument_whitespace* and *option_spec* that - correspond to the options for the function way. See `the Docutils docs - <http://docutils.sourceforge.net/docs/howto/rst-directives.html>`_ for - details. - - The directive class must inherit from the class - ``docutils.parsers.rst.Directive``. - - For example, the (already existing) :rst:dir:`literalinclude` directive would - be added like this: - - .. code-block:: python - - from docutils.parsers.rst import directives - add_directive('literalinclude', literalinclude_directive, - content = 0, arguments = (1, 0, 0), - linenos = directives.flag, - language = directives.unchanged, - encoding = directives.encoding) - - .. versionchanged:: 0.6 - Docutils 0.5-style directive classes are now supported. - -.. method:: Sphinx.add_directive_to_domain(domain, name, func, content, arguments, **options) - Sphinx.add_directive_to_domain(domain, name, directiveclass) - - Like :meth:`add_directive`, but the directive is added to the domain named - *domain*. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_role(name, role) - - Register a Docutils role. *name* must be the role name that occurs in the - source, *role* the role function (see the `Docutils documentation - <http://docutils.sourceforge.net/docs/howto/rst-roles.html>`_ on details). - -.. method:: Sphinx.add_role_to_domain(domain, name, role) - - Like :meth:`add_role`, but the role is added to the domain named *domain*. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_generic_role(name, nodeclass) - - Register a Docutils role that does nothing but wrap its contents in the - node given by *nodeclass*. - - .. versionadded:: 0.6 - -.. method:: Sphinx.add_object_type(directivename, rolename, indextemplate='', parse_node=None, \ - ref_nodeclass=None, objname='', doc_field_types=[]) - - This method is a very convenient way to add a new :term:`object` type that - can be cross-referenced. It will do this: - - * Create a new directive (called *directivename*) for documenting an object. - It will automatically add index entries if *indextemplate* is nonempty; if - given, it must contain exactly one instance of ``%s``. See the example - below for how the template will be interpreted. - * Create a new role (called *rolename*) to cross-reference to these - object descriptions. - * If you provide *parse_node*, it must be a function that takes a string and - a docutils node, and it must populate the node with children parsed from - the string. It must then return the name of the item to be used in - cross-referencing and index entries. See the :file:`conf.py` file in the - source for this documentation for an example. - * The *objname* (if not given, will default to *directivename*) names the - type of object. It is used when listing objects, e.g. in search results. - - For example, if you have this call in a custom Sphinx extension:: - - app.add_object_type('directive', 'dir', 'pair: %s; directive') - - you can use this markup in your documents:: - - .. rst:directive:: function - - Document a function. - - <...> - - See also the :rst:dir:`function` directive. - - For the directive, an index entry will be generated as if you had prepended :: - - .. index:: pair: function; directive - - The reference node will be of class ``literal`` (so it will be rendered in a - proportional font, as appropriate for code) unless you give the - *ref_nodeclass* argument, which must be a docutils node class. Most useful - are ``docutils.nodes.emphasis`` or ``docutils.nodes.strong`` -- you can also - use ``docutils.nodes.generated`` if you want no further text decoration. If - the text should be treated as literal (e.g. no smart quote replacement), but - not have typewriter styling, use ``sphinx.addnodes.literal_emphasis`` or - ``sphinx.addnodes.literal_strong``. - - For the role content, you have the same syntactical possibilities as for - standard Sphinx roles (see :ref:`xref-syntax`). - - This method is also available under the deprecated alias - ``add_description_unit``. - -.. method:: Sphinx.add_crossref_type(directivename, rolename, indextemplate='', ref_nodeclass=None, objname='') - - This method is very similar to :meth:`add_object_type` except that the - directive it generates must be empty, and will produce no output. - - That means that you can add semantic targets to your sources, and refer to - them using custom roles instead of generic ones (like :rst:role:`ref`). - Example call:: - - app.add_crossref_type('topic', 'topic', 'single: %s', docutils.nodes.emphasis) - - Example usage:: - - .. topic:: application API - - The application API - ------------------- - - <...> - - See also :topic:`this section <application API>`. - - (Of course, the element following the ``topic`` directive needn't be a - section.) - -.. method:: Sphinx.add_transform(transform) - - Add the standard docutils :class:`Transform` subclass *transform* to the list - of transforms that are applied after Sphinx parses a reST document. - -.. method:: Sphinx.add_post_transform(transform) - - Add the standard docutils :class:`Transform` subclass *transform* to the list - of transforms that are applied before Sphinx writes a document. - -.. method:: Sphinx.add_javascript(filename) - - Add *filename* to the list of JavaScript files that the default HTML template - will include. The filename must be relative to the HTML static path, see - :confval:`the docs for the config value <html_static_path>`. A full URI with - scheme, like ``http://example.org/foo.js``, is also supported. - - .. versionadded:: 0.5 - -.. method:: Sphinx.add_stylesheet(filename, alternate=None, title=None) - - Add *filename* to the list of CSS files that the default HTML template will - include. Like for :meth:`add_javascript`, the filename must be relative to - the HTML static path, or a full URI with scheme. - - .. versionadded:: 1.0 - - .. versionchanged:: 1.6 - Optional ``alternate`` and/or ``title`` attributes can be supplied with - the *alternate* (of boolean type) and *title* (a string) arguments. The - default is no title and *alternate* = ``False`` (see `this explanation - <https://developer.mozilla.org/en-US/docs/Web/CSS/Alternative_style_sheets>`_). - -.. method:: Sphinx.add_latex_package(packagename, options=None) +.. currentmodule:: sphinx.application - Add *packagename* to the list of packages that LaTeX source code will include. - If you provide *options*, it will be taken to `\usepackage` declaration. +.. automethod:: Sphinx.setup_extension(name) - .. code-block:: python +.. automethod:: Sphinx.require_sphinx(version) - app.add_latex_package('mypackage') # => \usepackage{mypackage} - app.add_latex_package('mypackage', 'foo,bar') # => \usepackage[foo,bar]{mypackage} +.. automethod:: Sphinx.connect(event, callback) - .. versionadded:: 1.3 +.. automethod:: Sphinx.disconnect(listener_id) -.. method:: Sphinx.add_lexer(alias, lexer) +.. automethod:: Sphinx.add_builder(builder) - Use *lexer*, which must be an instance of a Pygments lexer class, to - highlight code blocks with the given language *alias*. +.. automethod:: Sphinx.add_config_value(name, default, rebuild) - .. versionadded:: 0.6 +.. automethod:: Sphinx.add_event(name) -.. method:: Sphinx.add_autodocumenter(cls) +.. automethod:: Sphinx.set_translator(name, translator_class) - Add *cls* as a new documenter class for the :mod:`sphinx.ext.autodoc` - extension. It must be a subclass of :class:`sphinx.ext.autodoc.Documenter`. - This allows to auto-document new types of objects. See the source of the - autodoc module for examples on how to subclass :class:`Documenter`. +.. automethod:: Sphinx.add_node(node, \*\*kwds) - .. XXX add real docs for Documenter and subclassing +.. automethod:: Sphinx.add_enumerable_node(node, figtype, title_getter=None, \*\*kwds) - .. versionadded:: 0.6 +.. method:: Sphinx.add_directive(name, func, content, arguments, \*\*options) +.. automethod:: Sphinx.add_directive(name, directiveclass) -.. method:: Sphinx.add_autodoc_attrgetter(type, getter) +.. automethod:: Sphinx.add_role(name, role) - Add *getter*, which must be a function with an interface compatible to the - :func:`getattr` builtin, as the autodoc attribute getter for objects that are - instances of *type*. All cases where autodoc needs to get an attribute of a - type are then handled by this function instead of :func:`getattr`. +.. automethod:: Sphinx.add_generic_role(name, nodeclass) - .. versionadded:: 0.6 +.. automethod:: Sphinx.add_domain(domain) -.. method:: Sphinx.add_search_language(cls) +.. automethod:: Sphinx.override_domain(domain) - Add *cls*, which must be a subclass of :class:`sphinx.search.SearchLanguage`, - as a support language for building the HTML full-text search index. The - class must have a *lang* attribute that indicates the language it should be - used for. See :confval:`html_search_language`. +.. method:: Sphinx.add_directive_to_domain(domain, name, func, content, arguments, \*\*options) +.. automethod:: Sphinx.add_directive_to_domain(domain, name, directiveclass) - .. versionadded:: 1.1 +.. automethod:: Sphinx.add_role_to_domain(domain, name, role) -.. method:: Sphinx.add_source_parser(suffix, parser) +.. automethod:: Sphinx.add_index_to_domain(domain, index) - Register a parser class for specified *suffix*. +.. automethod:: Sphinx.add_object_type(directivename, rolename, indextemplate='', parse_node=None, ref_nodeclass=None, objname='', doc_field_types=[]) - .. versionadded:: 1.4 +.. automethod:: Sphinx.add_crossref_type(directivename, rolename, indextemplate='', ref_nodeclass=None, objname='') -.. method:: Sphinx.add_html_theme(name, theme_path) +.. automethod:: Sphinx.add_transform(transform) - Register a HTML Theme. The *name* is a name of theme, and *path* is a - full path to the theme (refs: :ref:`distribute-your-theme`). +.. automethod:: Sphinx.add_post_transform(transform) - .. versionadded:: 1.6 +.. automethod:: Sphinx.add_javascript(filename) -.. method:: Sphinx.add_env_collector(collector) +.. automethod:: Sphinx.add_stylesheet(filename, alternate=None, title=None) - Register an environment collector class (refs: :ref:`collector-api`) +.. automethod:: Sphinx.add_latex_package(packagename, options=None) - .. versionadded:: 1.6 +.. automethod:: Sphinx.add_lexer(alias, lexer) -.. method:: Sphinx.require_sphinx(version) +.. automethod:: Sphinx.add_autodocumenter(cls) - Compare *version* (which must be a ``major.minor`` version string, - e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build - when it is too old. +.. automethod:: Sphinx.add_autodoc_attrgetter(type, getter) - .. versionadded:: 1.0 +.. automethod:: Sphinx.add_search_language(cls) -.. method:: Sphinx.connect(event, callback) +.. automethod:: Sphinx.add_source_suffix(suffix, filetype) - Register *callback* to be called when *event* is emitted. For details on - available core events and the arguments of callback functions, please see - :ref:`events`. +.. automethod:: Sphinx.add_source_parser(parser) - The method returns a "listener ID" that can be used as an argument to - :meth:`disconnect`. +.. automethod:: Sphinx.add_env_collector(collector) -.. method:: Sphinx.disconnect(listener_id) +.. automethod:: Sphinx.add_html_theme(name, theme_path) - Unregister callback *listener_id*. +.. automethod:: Sphinx.add_message_catalog(catalog, locale_dir) +.. automethod:: Sphinx.is_parallel_allowed(typ) .. exception:: ExtensionError @@ -414,18 +106,11 @@ package. Emitting events --------------- -.. method:: Sphinx.emit(event, *arguments) - - Emit *event* and pass *arguments* to the callback functions. Return the - return values of all callbacks as a list. Do not emit core Sphinx events - in extensions! - -.. method:: Sphinx.emit_firstresult(event, *arguments) +.. class:: Sphinx - Emit *event* and pass *arguments* to the callback functions. Return the - result of the first callback that doesn't return ``None``. + .. automethod:: emit(event, \*arguments) - .. versionadded:: 0.5 + .. automethod:: emit_firstresult(event, \*arguments) Producing messages / logging @@ -500,6 +185,12 @@ handlers to the events. Example: Emitted when the builder object has been created. It is available as ``app.builder``. +.. event:: config-inited (app, config) + + Emitted when the config object has been initialized. + + .. versionadded:: 1.8 + .. event:: env-get-outdated (app, env, added, changed, removed) Emitted when the environment determines which source files have changed and @@ -614,7 +305,7 @@ handlers to the events. Example: .. event:: env-check-consistency (env) - Emmited when Consistency checks phase. You can check consistency of + Emitted when Consistency checks phase. You can check consistency of metadata for whole of documents. .. versionadded:: 1.6 @@ -674,28 +365,15 @@ Checking the Sphinx version Use this to adapt your extension to API changes in Sphinx. -.. data:: version_info - - A tuple of five elements; for Sphinx version 1.2.1 beta 3 this would be - ``(1, 2, 1, 'beta', 3)``. - - .. versionadded:: 1.2 - Before version 1.2, check the string ``sphinx.__version__``. +.. autodata:: version_info The Config object ----------------- -.. module:: sphinx.config +.. currentmodule:: sphinx.config -.. class:: Config - - The config object makes the values of all config values available as - attributes. - - It is available as the ``config`` attribute on the application and - environment objects. For example, to get the value of :confval:`language`, - use either ``app.config.language`` or ``env.config.language``. +.. autoclass:: Config .. _template-bridge: @@ -716,38 +394,12 @@ Exceptions .. module:: sphinx.errors -.. exception:: SphinxError - - This is the base class for "nice" exceptions. When such an exception is - raised, Sphinx will abort the build and present the exception category and - message to the user. - - Extensions are encouraged to derive from this exception for their custom - errors. - - Exceptions *not* derived from :exc:`SphinxError` are treated as unexpected - and shown to the user with a part of the traceback (and the full traceback - saved in a temporary file). - - .. attribute:: category - - Description of the exception "category", used in converting the exception - to a string ("category: message"). Should be set accordingly in - subclasses. - -.. exception:: ConfigError - - Used for erroneous values or nonsensical combinations of configuration - values. - -.. exception:: ExtensionError - - Used for errors in setting up extensions. +.. autoexception:: SphinxError -.. exception:: ThemeError +.. autoexception:: ConfigError - Used for errors to do with themes. +.. autoexception:: ExtensionError -.. exception:: VersionRequirementError +.. autoexception:: ThemeError - Raised when the docs require a higher Sphinx version than the current one. +.. autoexception:: VersionRequirementError diff --git a/doc/extdev/builderapi.rst b/doc/extdev/builderapi.rst index b8ff0595b..2c2cf12e3 100644 --- a/doc/extdev/builderapi.rst +++ b/doc/extdev/builderapi.rst @@ -17,6 +17,9 @@ Builder API .. autoattribute:: format .. autoattribute:: epilog .. autoattribute:: supported_image_types + .. autoattribute:: supported_remote_images + .. autoattribute:: supported_data_uri_images + .. autoattribute:: default_translator_class These methods are predefined and will be called from the application: diff --git a/doc/extdev/i18n.rst b/doc/extdev/i18n.rst new file mode 100644 index 000000000..c8c54da36 --- /dev/null +++ b/doc/extdev/i18n.rst @@ -0,0 +1,17 @@ +.. _i18n-api: + +i18n API +======== + +.. currentmodule:: sphinx.locale + +.. autofunction:: init + +.. autofunction:: init_console + +.. autofunction:: get_translation + +.. autofunction:: _ + +.. autofunction:: __ + diff --git a/doc/extdev/index.rst b/doc/extdev/index.rst index ace43fd71..ba1e5cbd1 100644 --- a/doc/extdev/index.rst +++ b/doc/extdev/index.rst @@ -52,6 +52,8 @@ Note that it is still necessary to register the builder using .. _entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins +.. _ext-metadata: + Extension metadata ------------------ @@ -63,6 +65,11 @@ as metadata of the extension. Metadata keys currently recognized are: * ``'version'``: a string that identifies the extension version. It is used for extension version requirement checking (see :confval:`needs_extensions`) and informational purposes. If not given, ``"unknown version"`` is substituted. +* ``'env_version'``: an integer that identifies the version of env data + structure if the extension stores any data to environment. It is used to + detect the data structure has been changed from last build. The extensions + have to increment the version when data structure has changed. If not given, + Sphinx considers the extension does not stores any data to environment. * ``'parallel_read_safe'``: a boolean that specifies if parallel reading of source files can be used when the extension is loaded. It defaults to ``False``, i.e. you have to explicitly specify your extension to be @@ -86,6 +93,8 @@ APIs used for writing extensions parserapi nodes logging + i18n + utils Deprecated APIs --------------- @@ -99,12 +108,206 @@ The following is a list of deprecated interface. .. list-table:: deprecated APIs :header-rows: 1 + :class: deprecated + :widths: 40, 10, 10, 40 * - Target - Deprecated - (will be) Removed - Alternatives + * - :rst:dir:`highlightlang` + - 1.8 + - 4.0 + - :rst:dir:`highlight` + + * - :meth:`~sphinx.application.Sphinx.add_stylesheet()` + - 1.8 + - 4.0 + - :meth:`~sphinx.application.Sphinx.add_css_file()` + + * - ``viewcode_import`` (config value) + - 1.8 + - 3.0 + - :confval:`viewcode_follow_imported_members` + + * - ``sphinx.writers.latex.Table.caption_footnotetexts`` + - 1.8 + - 3.0 + - - + + * - ``sphinx.writers.latex.Table.header_footnotetexts`` + - 1.8 + - 3.0 + - - + + * - ``sphinx.writers.latex.LaTeXWriter.footnotestack`` + - 1.8 + - 3.0 + - - + + * - ``sphinx.writers.latex.LaTeXWriter.restrict_footnote()`` + - 1.8 + - 3.0 + - - + + * - ``sphinx.writers.latex.LaTeXWriter.unrestrict_footnote()`` + - 1.8 + - 3.0 + - - + + * - ``sphinx.writers.latex.LaTeXWriter.bibitems`` + - 1.8 + - 3.0 + - Nothing + + * - ``sphinx.application.CONFIG_FILENAME`` + - 1.8 + - 3.0 + - ``sphinx.config.CONFIG_FILENAME`` + + * - ``Config.check_unicode()`` + - 1.8 + - 3.0 + - ``sphinx.config.check_unicode()`` + + * - ``Config.check_types()`` + - 1.8 + - 3.0 + - ``sphinx.config.check_confval_types()`` + + * - ``dirname``, ``filename`` and ``tags`` arguments of + ``Config.__init__()`` + - 1.8 + - 3.0 + - ``Config.read()`` + + * - The value of :confval:`html_search_options` + - 1.8 + - 3.0 + - see :confval:`html_search_options` + + * - ``sphinx.versioning.prepare()`` + - 1.8 + - 3.0 + - ``sphinx.versioning.UIDTransform`` + + * - ``sphinx.application.Sphinx.override_domain()`` + - 1.8 + - 3.0 + - :meth:`~sphinx.application.Sphinx.add_domain()` + + * - ``BuildEnvironment.load()`` + - 1.8 + - 3.0 + - ``pickle.load()`` + + * - ``BuildEnvironment.loads()`` + - 1.8 + - 3.0 + - ``pickle.loads()`` + + * - ``BuildEnvironment.frompickle()`` + - 1.8 + - 3.0 + - ``pickle.load()`` + + * - ``BuildEnvironment.dump()`` + - 1.8 + - 3.0 + - ``pickle.dump()`` + + * - ``BuildEnvironment.dumps()`` + - 1.8 + - 3.0 + - ``pickle.dumps()`` + + * - ``BuildEnvironment.topickle()`` + - 1.8 + - 3.0 + - ``pickle.dump()`` + + * - ``BuildEnvironment._nitpick_ignore`` + - 1.8 + - 3.0 + - :confval:`nitpick_ignore` + + * - ``warn()`` (template helper function) + - 1.8 + - 3.0 + - ``warning()`` + + * - :confval:`source_parsers` + - 1.8 + - 3.0 + - :meth:`~sphinx.application.Sphinx.add_source_parser()` + + * - ``Sphinx.import_object()`` + - 1.8 + - 3.0 + - ``sphinx.util.import_object()`` + + * - ``suffix`` argument of + :meth:`~sphinx.application.Sphinx.add_source_parser()` + - 1.8 + - 3.0 + - :meth:`~sphinx.application.Sphinx.add_source_suffix()` + + * - ``sphinx.util.docutils.directive_helper()`` + - 1.8 + - 3.0 + - ``Directive`` class of docutils + + * - ``sphinx.cmdline`` + - 1.8 + - 3.0 + - ``sphinx.cmd.build`` + + * - ``BuildEnvironment.update()`` + - 1.8 + - 3.0 + - ``Builder.read()`` + + * - ``BuildEnvironment.read_doc()`` + - 1.8 + - 3.0 + - ``Builder.read_doc()`` + + * - ``BuildEnvironment._read_serial()`` + - 1.8 + - 3.0 + - ``Builder.read()`` + + * - ``BuildEnvironment._read_parallel()`` + - 1.8 + - 3.0 + - ``Builder.read()`` + + * - ``BuildEnvironment.write_doctree()`` + - 1.8 + - 3.0 + - ``Builder.write_doctree()`` + + * - ``sphinx.locale.l_()`` + - 1.8 + - 3.0 + - :func:`sphinx.locale._()` + + * - ``sphinx.locale.lazy_gettext()`` + - 1.8 + - 3.0 + - :func:`sphinx.locale._()` + + * - ``sphinx.locale.mygettext()`` + - 1.8 + - 3.0 + - :func:`sphinx.locale._()` + + * - ``sphinx.util.copy_static_entry()`` + - 1.5 + - 3.0 + - ``sphinx.util.fileutil.copy_asset()`` + * - ``sphinx.build_main()`` - 1.7 - 2.0 @@ -178,13 +381,18 @@ The following is a list of deprecated interface. * - ``sphinx.websupport`` - 1.6 - 2.0 - - `sphinxcontrib-websupport <https://pypi.python.org/pypi/sphinxcontrib-websupport>`_ + - `sphinxcontrib-websupport <https://pypi.org/project/sphinxcontrib-websupport/>`_ * - ``StandaloneHTMLBuilder.css_files`` - 1.6 - 2.0 - :meth:`~sphinx.application.Sphinx.add_stylesheet()` + * - ``document.settings.gettext_compact`` + - 1.8 + - 1.8 + - :confval:`gettext_compact` + * - ``Sphinx.status_iterator()`` - 1.6 - 1.7 diff --git a/doc/extdev/logging.rst b/doc/extdev/logging.rst index 50110c2a4..2e735e286 100644 --- a/doc/extdev/logging.rst +++ b/doc/extdev/logging.rst @@ -3,18 +3,11 @@ Logging API =========== -.. function:: sphinx.util.logging.getLogger(name) +.. currentmodule:: sphinx.util.logging - Returns a logger wrapped by :class:`SphinxLoggerAdapter` with the specified *name*. +.. autofunction:: getLogger(name) - Example usage:: - - from sphinx.util import logging # Load on top of python's logging module - - logger = logging.getLogger(__name__) - logger.info('Hello, this is an extension!') - -.. class:: SphinxLoggerAdapter(logging.LoggerAdapter) +.. autoclass:: SphinxLoggerAdapter(logging.LoggerAdapter) .. method:: SphinxLoggerAdapter.error(level, msg, *args, **kwargs) .. method:: SphinxLoggerAdapter.critical(level, msg, *args, **kwargs) @@ -62,16 +55,6 @@ Logging API colored as ``"darkgray"``, and debug2 level ones are ``"lightgray"``. The others are not colored. -.. function:: pending_logging() - - Marks all logs as pending:: - - with pending_logging(): - logger.warning('Warning message!') # not flushed yet - some_long_process() - - # the warning is flushed here - -.. function:: pending_warnings() +.. autofunction:: pending_logging() - Marks warning logs as pending. Similar to :func:`pending_logging`. +.. autofunction:: pending_warnings() diff --git a/doc/extdev/parserapi.rst b/doc/extdev/parserapi.rst index 7008a15a8..e71661b2e 100644 --- a/doc/extdev/parserapi.rst +++ b/doc/extdev/parserapi.rst @@ -3,6 +3,35 @@ Parser API ========== +`The docutils documentation describes`__ parsers as follows: + + The Parser analyzes the input document and creates a node tree + representation. + +__ http://docutils.sourceforge.net/docs/dev/hacking.html#parsing-the-document + +In Sphinx, the parser modules works as same as docutils. The parsers are +registered to Sphinx by extensions using Application APIs; +:meth:`Sphinx.add_source_suffix()` and :meth:`Sphinx.add_source_parsers()`. + +The *source suffix* is a mapping from file suffix to file type. For example, +``.rst`` file is mapped to ``'restructuredtext'`` type. Sphinx uses the +file type to looking for parsers from registered list. On searching, +Sphinx refers to the ``Parser.supported`` attribute and picks up a parser +which contains the file type in the attribute. + +The users can override the source suffix mappings using +:confval:`source_suffix` like following:: + + # a mapping from file suffix to file types + source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', + } + +You should indicate file types your parser supports. This will allow users +to configure their settings appropriately. + .. module:: sphinx.parsers .. autoclass:: Parser diff --git a/doc/extdev/utils.rst b/doc/extdev/utils.rst new file mode 100644 index 000000000..ebf05629e --- /dev/null +++ b/doc/extdev/utils.rst @@ -0,0 +1,22 @@ +Utilities +========= + +Sphinx provides utility classes and functions to develop extensions. + +Base classes for components +--------------------------- + +These base classes are useful to allow your extensions to obtain Sphinx +components (e.g. :class:`.Config`, :class:`.BuildEnvironment` and so on) easily. + +.. note:: The subclasses of them might not work with bare docutils because they + are strongly coupled with Sphinx. + +.. autoclass:: sphinx.transforms.SphinxTransform + :members: + +.. autoclass:: sphinx.util.docutils.SphinxDirective + :members: + +.. autoclass:: sphinx.transforms.post_transforms.images.ImageConverter + :members: diff --git a/doc/faq.rst b/doc/faq.rst index fe3173749..b2d6cc9e6 100644 --- a/doc/faq.rst +++ b/doc/faq.rst @@ -11,12 +11,9 @@ How do I... ... create PDF files without LaTeX? `rinohtype`_ provides a PDF builder that can be used as a drop-in - replacement for the LaTeX builder. Alternatively, you can use `rst2pdf`_ - version 0.12 or greater which comes with built-in Sphinx integration. See - the :ref:`builders` section for details. + replacement for the LaTeX builder. .. _rinohtype: https://github.com/brechtm/rinohtype - .. _rst2pdf: https://github.com/rst2pdf/rst2pdf ... get section numbers? They are automatic in LaTeX output; for HTML, give a ``:numbered:`` option to @@ -26,7 +23,7 @@ How do I... Use themes, see :doc:`theming`. ... add global substitutions or includes? - Add them in the :confval:`rst_epilog` config value. + Add them in the :confval:`rst_prolog` or :confval:`rst_epilog` config value. ... display the whole TOC tree in the sidebar? Use the :data:`toctree` callable in a custom layout template, probably in the @@ -75,9 +72,9 @@ SCons PyPI Jannis Leidel wrote a `setuptools command - <https://pypi.python.org/pypi/Sphinx-PyPI-upload>`_ that automatically + <https://pypi.org/project/Sphinx-PyPI-upload/>`_ that automatically uploads Sphinx documentation to the PyPI package documentation area at - http://pythonhosted.org/. + https://pythonhosted.org/. GitHub Pages Directories starting with underscores are ignored by default which breaks @@ -106,7 +103,7 @@ Google Analytics {% block footer %} {{ super() }} - <div class="footer">This page uses <a href="http://analytics.google.com/"> + <div class="footer">This page uses <a href="https://analytics.google.com/"> Google Analytics</a> to collect statistics. You can disable it by blocking the JavaScript coming from www.google-analytics.com. <script type="text/javascript"> @@ -122,7 +119,7 @@ Google Analytics {% endblock %} -.. _api role: http://git.savannah.gnu.org/cgit/kenozooid.git/tree/doc/extapi.py +.. _api role: https://git.savannah.gnu.org/cgit/kenozooid.git/tree/doc/extapi.py .. _xhtml to reST: http://docutils.sourceforge.net/sandbox/xhtml2rest/xhtml2rest.py diff --git a/doc/glossary.rst b/doc/glossary.rst index a92e52b98..167edb8d4 100644 --- a/doc/glossary.rst +++ b/doc/glossary.rst @@ -32,7 +32,7 @@ Glossary Content of the directive. - See :ref:`directives` for more information. + See :ref:`rst-directives` for more information. document name Since reST source files can have different extensions (some people like @@ -56,8 +56,9 @@ Glossary Having domains means that there are no naming problems when one set of documentation wants to refer to e.g. C++ and Python classes. It also means that extensions that support the documentation of whole new - languages are much easier to write. For more information about domains, - see the chapter :ref:`domains`. + languages are much easier to write. + + For more information, refer to :doc:`/usage/restructuredtext/domains`. environment A structure where information about all documents under the root is saved, @@ -81,8 +82,12 @@ Glossary role A reStructuredText markup element that allows marking a piece of text. Like directives, roles are extensible. The basic syntax looks like this: - ``:rolename:`content```. See :ref:`inlinemarkup` for details. + ``:rolename:`content```. See :ref:`rst-inline-markup` for details. source directory The directory which, including its subdirectories, contains all source files for one Sphinx project. + + reStructuredText + An easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and + parser system. diff --git a/doc/install.rst b/doc/install.rst deleted file mode 100644 index 2eb44b809..000000000 --- a/doc/install.rst +++ /dev/null @@ -1,155 +0,0 @@ -:orphan: - -Installing Sphinx -================= - -Since Sphinx is written in the Python language, you need to install Python -(the required version is at least 2.7) and Sphinx. - -Sphinx packages are available on the `Python Package Index -<https://pypi.python.org/pypi/Sphinx>`_. - -You can also download a snapshot from the Git repository: - -* as a `.tar.gz <https://github.com/sphinx-doc/sphinx/archive/master.tar.gz>`__ - file or -* as a `.zip <https://github.com/sphinx-doc/sphinx/archive/master.zip>`_ file - -There are introductions for several environments: - -.. contents:: - :depth: 1 - :local: - :backlinks: none - - -Debian/Ubuntu: Install Sphinx using packaging system ----------------------------------------------------- - -You may install using this command if you use Debian/Ubuntu. - -.. code-block:: bash - - $ apt-get install python-sphinx - - -Other Linux distributions -------------------------- - -Most Linux distributions have Sphinx in their package repositories. Usually the -package is called "python-sphinx", "python-Sphinx" or "sphinx". Be aware that -there are two other packages with "sphinx" in their name: a speech recognition -toolkit (CMU Sphinx) and a full-text search database (Sphinx search). - - -Mac OS X: Install Sphinx using MacPorts ---------------------------------------- - -If you use Mac OS X `MacPorts <http://www.macports.org/>`_, use this command to -install all necessary software. - -.. code-block:: bash - - $ sudo port install py27-sphinx - -To set up the executable paths, use the ``port select`` command: - -.. code-block:: bash - - $ sudo port select --set python python27 - $ sudo port select --set sphinx py27-sphinx - -Type :command:`which sphinx-quickstart` to check if the installation was -successful. - - -Windows: Install Python and Sphinx ----------------------------------- - -Install Python -^^^^^^^^^^^^^^ - -Most Windows users do not have Python, so we begin with the installation of -Python itself. If you have already installed Python, please skip this section. - -Go to https://www.python.org/, the main download site for Python. Look at the left -sidebar and under "Quick Links", click "Windows Installer" to download. - -.. image:: pythonorg.png - -.. note:: - - Currently, Python offers two major versions, 2.x and 3.x. Sphinx 1.5 can run - under Python 2.7, 3.4, 3.5, 3.6, with the recommended version being 2.7. This - chapter assumes you have installed Python 2.7. - -Follow the Windows installer for Python. - -.. image:: installpython.jpg - -After installation, you better add the Python executable directories to the -environment variable ``PATH`` in order to run Python and package commands such -as ``sphinx-build`` easily from the Command Prompt. - -* Right-click the "My Computer" icon and choose "Properties" -* Click the "Environment Variables" button under the "Advanced" tab - -* If "Path" (or "PATH") is already an entry in the "System variables" list, edit - it. If it is not present, add a new variable called "PATH". - -* Add these paths, separating entries by ";": - - - ``C:\Python27`` -- this folder contains the main Python executable - - ``C:\Python27\Scripts`` -- this folder will contain executables added by - Python packages installed with pip (see below) - - This is for Python 2.7. If you use another version of - Python or installed to a non-default location, change the digits "27" - accordingly. - -* Now run the **Command Prompt**. After command prompt window appear, type - ``python`` and Enter. If the Python installation was successful, the - installed Python version is printed, and you are greeted by the prompt - ``>>>``. Type ``Ctrl+Z`` and Enter to quit. - - -Install the pip command -^^^^^^^^^^^^^^^^^^^^^^^ - -Python has a very useful :command:`pip` command which can download and install -3rd-party libraries with a single command. This is provided by the -Python Packaging Authority(PyPA): -https://groups.google.com/forum/#!forum/pypa-dev - -To install pip, download https://bootstrap.pypa.io/get-pip.py and -save it somewhere. After download, invoke the command prompt, go to the -directory with ``get-pip.py`` and run this command: - -.. code-block:: bat - - C:\> python get-pip.py - -Now :command:`pip` command is installed. From there we can go to the Sphinx -install. - -.. note:: - - ``pip`` has been contained in the Python official installation after version - of Python-3.4.0 or Python-2.7.9. - - -Installing Sphinx with pip --------------------------- - -If you finished the installation of pip, type this line in the command prompt: - -.. code-block:: bat - - C:\> pip install sphinx - -After installation, type :command:`sphinx-build -h` on the command prompt. If -everything worked fine, you will get a Sphinx version number and a list of -options for this command. - -That it. Installation is over. Head to :doc:`tutorial` to make a Sphinx -project. diff --git a/doc/installpython.jpg b/doc/installpython.jpg Binary files differdeleted file mode 100644 index fff4630ae..000000000 --- a/doc/installpython.jpg +++ /dev/null diff --git a/doc/intl.rst b/doc/intl.rst index c6bfb7bbf..43f620bd0 100644 --- a/doc/intl.rst +++ b/doc/intl.rst @@ -13,7 +13,7 @@ in itself. See the :ref:`intl-options` for details on configuration. :width: 100% Workflow visualization of translations in Sphinx. (The stick-figure is taken - from an `XKCD comic <http://xkcd.com/779/>`_.) + from an `XKCD comic <https://xkcd.com/779/>`_.) .. contents:: :local: @@ -317,13 +317,13 @@ There is `sphinx translation page`_ for Sphinx (master) documentation. .. rubric:: Footnotes .. [1] See the `GNU gettext utilities - <http://www.gnu.org/software/gettext/manual/gettext.html#Introduction>`_ + <https://www.gnu.org/software/gettext/manual/gettext.html#Introduction>`_ for details on that software suite. .. [2] Because nobody expects the Spanish Inquisition! -.. _`transifex-client`: https://pypi.python.org/pypi/transifex-client -.. _`sphinx-intl`: https://pypi.python.org/pypi/sphinx-intl +.. _`transifex-client`: https://pypi.org/project/transifex-client/ +.. _`sphinx-intl`: https://pypi.org/project/sphinx-intl/ .. _Transifex: https://www.transifex.com/ .. _`sphinx translation page`: https://www.transifex.com/sphinx-doc/sphinx-doc/ .. _`Transifex Client documentation`: http://docs.transifex.com/developer/client/ diff --git a/doc/intro.rst b/doc/intro.rst index a789145fe..d44003b71 100644 --- a/doc/intro.rst +++ b/doc/intro.rst @@ -8,7 +8,7 @@ you have a directory containing a bunch of reST-formatted documents (and possibly subdirectories of docs in there as well), Sphinx can generate a nicely-organized arrangement of HTML files (in some other directory) for easy browsing and navigation. But from the same source, it can also generate a PDF -file using LaTeX, `rinohtype`_ or `rst2pdf`_ (see :ref:`builders`). +file using LaTeX. The focus is on hand-written documentation, rather than auto-generated API docs. Though there is support for that kind of documentation as well (which is @@ -21,7 +21,6 @@ also `Write the docs <https://write-the-docs.readthedocs.io/>`_, written by Eric Holscher. .. _rinohtype: https://github.com/brechtm/rinohtype -.. _rst2pdf: https://github.com/rst2pdf/rst2pdf Conversion from other systems ----------------------------- @@ -30,20 +29,20 @@ This section is intended to collect helpful hints for those wanting to migrate to reStructuredText/Sphinx from other documentation systems. * Gerard Flanagan has written a script to convert pure HTML to reST; it can be - found at the `Python Package Index <https://pypi.python.org/pypi/html2rest>`_. + found at the `Python Package Index <https://pypi.org/project/html2rest/>`_. * For converting the old Python docs to Sphinx, a converter was written which can be found at `the Python SVN repository - <http://svn.python.org/projects/doctools/converter>`_. It contains generic + <https://svn.python.org/projects/doctools/converter/>`_. It contains generic code to convert Python-doc-style LaTeX markup to Sphinx reST. * Marcin Wojdyr has written a script to convert Docbook to reST with Sphinx markup; it is at `GitHub <https://github.com/wojdyr/db2rst>`_. * Christophe de Vienne wrote a tool to convert from Open/LibreOffice documents - to Sphinx: `odt2sphinx <https://pypi.python.org/pypi/odt2sphinx/>`_. + to Sphinx: `odt2sphinx <https://pypi.org/project/odt2sphinx/>`_. -* To convert different markups, `Pandoc <http://pandoc.org/>`_ is +* To convert different markups, `Pandoc <https://pandoc.org/>`_ is a very helpful tool. @@ -70,5 +69,5 @@ highlighting support, you must also install the Pygments_ library. Usage ----- -See :doc:`tutorial` for an introduction. It also contains links to more -advanced sections in this manual for the topics it discusses. +See :doc:`/usage/quickstart` for an introduction. It also contains links to +more advanced sections in this manual for the topics it discusses. diff --git a/doc/latex.rst b/doc/latex.rst index 79c0c5135..cfd6f7d8b 100644 --- a/doc/latex.rst +++ b/doc/latex.rst @@ -1,4 +1,4 @@ -.. highlightlang:: python +.. highlight:: python .. _latex: @@ -389,13 +389,6 @@ Macros multiple paragraphs in header cells of tables. .. versionadded:: 1.6.3 ``\sphinxstylecodecontinued`` and ``\sphinxstylecodecontinues``. -- by default the Sphinx style file ``sphinx.sty`` executes the command - ``\fvset{fontsize=\small}`` as part of its configuration of - ``fancyvrb.sty``. This may be overriden for example via - ``\fvset{fontsize=auto}`` which will let code listings use the ambient font - size. Refer to ``fancyvrb.sty``'s documentation for further keys. - - .. versionadded:: 1.5 - the table of contents is typeset via ``\sphinxtableofcontents`` which is a wrapper (whose definition can be found in :file:`sphinxhowto.cls` or in :file:`sphinxmanual.cls`) of standard ``\tableofcontents``. @@ -404,6 +397,8 @@ Macros formerly, the meaning of ``\tableofcontents`` was modified by Sphinx. - the ``\maketitle`` command is redefined by the class files :file:`sphinxmanual.cls` and :file:`sphinxhowto.cls`. +- the citation reference is typeset via ``\sphinxcite`` which is a wrapper + of standard ``\cite``. Environments ~~~~~~~~~~~~ diff --git a/doc/man/sphinx-apidoc.rst b/doc/man/sphinx-apidoc.rst index 9a13f1401..97135d98e 100644 --- a/doc/man/sphinx-apidoc.rst +++ b/doc/man/sphinx-apidoc.rst @@ -4,7 +4,7 @@ sphinx-apidoc Synopsis -------- -**sphinx-apidoc** [*options*] -o <*outputdir*> <*sourcedir*> [*pathnames* ...] +**sphinx-apidoc** [*OPTIONS*] -o <*OUTPUT_PATH*> <*MODULE_PATH*> [*EXCLUDE_PATTERN*, ...] Description ----------- @@ -13,9 +13,12 @@ Description that, using the :rst:dir:`autodoc` extension, document a whole package in the style of other automatic API documentation tools. -*sourcedir* is the path to a Python package to document, and *outputdir* is the -directory where the generated sources are placed. Any *pathnames* given are -paths to be excluded from the generation. +*MODULE_PATH* is the path to a Python package to document, and *OUTPUT_PATH* is +the directory where the generated sources are placed. Any *EXCLUDE_PATTERN*\s +given are `fnmatch-style`_ file and/or directory patterns that will be excluded +from generation. + +.. _fnmatch-style: https://docs.python.org/3/library/fnmatch.html .. warning:: @@ -31,7 +34,7 @@ Options .. program:: sphinx-apidoc -.. option:: -o <outputdir> +.. option:: -o <OUTPUT_PATH> Directory to place the output files. If it does not exist, it is created. @@ -51,7 +54,7 @@ Options Suffix for the source files generated. Defaults to ``rst``. -.. option:: -d <maxdepth> +.. option:: -d <MAXDEPTH> Maximum depth for the generated table of contents file. @@ -130,3 +133,5 @@ See also -------- :manpage:`sphinx-build(1)`, :manpage:`sphinx-autogen(1)` + +.. _fnmatch: https://docs.python.org/3/library/fnmatch.html diff --git a/doc/man/sphinx-quickstart.rst b/doc/man/sphinx-quickstart.rst index c4bbc531b..1d7a99add 100644 --- a/doc/man/sphinx-quickstart.rst +++ b/doc/man/sphinx-quickstart.rst @@ -20,7 +20,7 @@ Options .. option:: -q, --quiet - Quiet mode that will skips interactive wizard to specify options. + Quiet mode that will skip interactive wizard to specify options. This option requires `-p`, `-a` and `-v` options. .. option:: -h, --help, --version @@ -112,13 +112,17 @@ Options Enable `sphinx.ext.viewcode` extension. +.. option:: --ext-githubpages + + Enable `sphinx.ext.githubpages` extension. + .. option:: --extensions=EXTENSIONS - Enable arbitary extensions. + Enable arbitrary extensions. .. rubric:: Makefile and Batchfile Creation Options -.. option:: --use-make-mode, --no-use-make-mode +.. option:: --use-make-mode (-m), --no-use-make-mode (-M) :file:`Makefile/make.bat` uses (or doesn't use) :ref:`make-mode <make_mode>`. Default is ``use``, which generates a more concise :file:`Makefile/make.bat`. diff --git a/doc/markdown.rst b/doc/markdown.rst deleted file mode 100644 index 75a35b9d0..000000000 --- a/doc/markdown.rst +++ /dev/null @@ -1,45 +0,0 @@ -.. highlightlang:: python - -.. _markdown: - -Markdown support -================ - -`Markdown <https://daringfireball.net/projects/markdown/>`__ is a lightweight markup language with a simplistic plain -text formatting syntax. -It exists in many syntactically different *flavors*. -To support Markdown-based documentation, Sphinx can use -`recommonmark <http://recommonmark.readthedocs.io/en/latest/index.html>`__. -recommonmark is a Docutils bridge to `CommonMark-py <https://github.com/rtfd/CommonMark-py>`__, a -Python package for parsing the `CommonMark <http://commonmark.org/>`__ Markdown flavor. - - -Configuration -------------- - -To configure your Sphinx project for Markdown support, proceed as follows: - -#. Install recommonmark: - - :: - - pip install recommonmark - -#. Add the Markdown parser to the ``source_parsers`` configuration variable in your Sphinx configuration file: - - :: - - source_parsers = { - '.md': 'recommonmark.parser.CommonMarkParser', - } - - You can replace `.md` with a filename extension of your choice. - -#. Add the Markdown filename extension to the ``source_suffix`` configuration variable: - - :: - - source_suffix = ['.rst', '.md'] - -#. You can further configure recommonmark to allow custom syntax that standard CommonMark doesn't support. Read more in - the `recommonmark documentation <http://recommonmark.readthedocs.io/en/latest/auto_structify.html>`__. diff --git a/doc/markup/code.rst b/doc/markup/code.rst deleted file mode 100644 index 19ec0cc4c..000000000 --- a/doc/markup/code.rst +++ /dev/null @@ -1,270 +0,0 @@ -.. highlight:: rest - -.. _code-examples: - -Showing code examples ---------------------- - -.. index:: pair: code; examples - single: sourcecode - -Examples of Python source code or interactive sessions are represented using -standard reST literal blocks. They are started by a ``::`` at the end of the -preceding paragraph and delimited by indentation. - -Representing an interactive session requires including the prompts and output -along with the Python code. No special markup is required for interactive -sessions. After the last line of input or output presented, there should not be -an "unused" primary prompt; this is an example of what *not* to do:: - - >>> 1 + 1 - 2 - >>> - -Syntax highlighting is done with `Pygments <http://pygments.org>`_ and handled -in a smart way: - -* There is a "highlighting language" for each source file. Per default, this is - ``'python'`` as the majority of files will have to highlight Python snippets, - but the doc-wide default can be set with the :confval:`highlight_language` - config value. - -* Within Python highlighting mode, interactive sessions are recognized - automatically and highlighted appropriately. Normal Python code is only - highlighted if it is parseable (so you can use Python as the default, but - interspersed snippets of shell commands or other code blocks will not be - highlighted as Python). - -* The highlighting language can be changed using the ``highlight`` directive, - used as follows: - - .. rst:directive:: .. highlight:: language - - Example:: - - .. highlight:: c - - This language is used until the next ``highlight`` directive is encountered. - -* For documents that have to show snippets in different languages, there's also - a :rst:dir:`code-block` directive that is given the highlighting language - directly: - - .. rst:directive:: .. code-block:: language - - Use it like this:: - - .. code-block:: ruby - - Some Ruby code. - - The directive's alias name :rst:dir:`sourcecode` works as well. - -* The valid values for the highlighting language are: - - * ``none`` (no highlighting) - * ``python`` (the default when :confval:`highlight_language` isn't set) - * ``guess`` (let Pygments guess the lexer based on contents, only works with - certain well-recognizable languages) - * ``rest`` - * ``c`` - * ... and any other `lexer alias that Pygments supports - <http://pygments.org/docs/lexers/>`_. - -* If highlighting with the selected language fails (i.e. Pygments emits an - "Error" token), the block is not highlighted in any way. - -Line numbers -^^^^^^^^^^^^ - -Pygments can generate line numbers for code blocks. For -automatically-highlighted blocks (those started by ``::``), line numbers must be -switched on in a :rst:dir:`highlight` directive, with the ``linenothreshold`` -option:: - - .. highlight:: python - :linenothreshold: 5 - -This will produce line numbers for all code blocks longer than five lines. - -For :rst:dir:`code-block` blocks, a ``linenos`` flag option can be given to -switch on line numbers for the individual block:: - - .. code-block:: ruby - :linenos: - - Some more Ruby code. - -The first line number can be selected with the ``lineno-start`` option. If -present, ``linenos`` flag is automatically activated:: - - .. code-block:: ruby - :lineno-start: 10 - - Some more Ruby code, with line numbering starting at 10. - -Additionally, an ``emphasize-lines`` option can be given to have Pygments -emphasize particular lines:: - - .. code-block:: python - :emphasize-lines: 3,5 - - def some_function(): - interesting = False - print 'This line is highlighted.' - print 'This one is not...' - print '...but this one is.' - -.. versionchanged:: 1.1 - ``emphasize-lines`` has been added. - -.. versionchanged:: 1.3 - ``lineno-start`` has been added. - -.. versionchanged:: 1.6.6 - LaTeX supports the ``emphasize-lines`` option. - -Includes -^^^^^^^^ - -.. rst:directive:: .. literalinclude:: filename - - Longer displays of verbatim text may be included by storing the example text - in an external file containing only plain text. The file may be included - using the ``literalinclude`` directive. [1]_ For example, to include the - Python source file :file:`example.py`, use:: - - .. literalinclude:: example.py - - The file name is usually relative to the current file's path. However, if it - is absolute (starting with ``/``), it is relative to the top source - directory. - - Tabs in the input are expanded if you give a ``tab-width`` option with the - desired tab width. - - Like :rst:dir:`code-block`, the directive supports the ``linenos`` flag - option to switch on line numbers, the ``lineno-start`` option to select the - first line number, the ``emphasize-lines`` option to emphasize particular - lines, and a ``language`` option to select a language different from the - current file's standard language. Example with options:: - - .. literalinclude:: example.rb - :language: ruby - :emphasize-lines: 12,15-18 - :linenos: - - Include files are assumed to be encoded in the :confval:`source_encoding`. - If the file has a different encoding, you can specify it with the - ``encoding`` option:: - - .. literalinclude:: example.py - :encoding: latin-1 - - The directive also supports including only parts of the file. If it is a - Python module, you can select a class, function or method to include using - the ``pyobject`` option:: - - .. literalinclude:: example.py - :pyobject: Timer.start - - This would only include the code lines belonging to the ``start()`` method in - the ``Timer`` class within the file. - - Alternately, you can specify exactly which lines to include by giving a - ``lines`` option:: - - .. literalinclude:: example.py - :lines: 1,3,5-10,20- - - This includes the lines 1, 3, 5 to 10 and lines 20 to the last line. - - Another way to control which part of the file is included is to use the - ``start-after`` and ``end-before`` options (or only one of them). If - ``start-after`` is given as a string option, only lines that follow the first - line containing that string are included. If ``end-before`` is given as a - string option, only lines that precede the first lines containing that string - are included. - - With lines selected using ``start-after`` it is still possible to use - ``lines``, the first allowed line having by convention the line number ``1``. - - When lines have been selected in any of the ways described above, the - line numbers in ``emphasize-lines`` refer to those selected lines, counted - consecutively starting at ``1``. - - When specifying particular parts of a file to display, it can be useful to - display the original line numbers. This can be done using the - ``lineno-match`` option, which is however allowed only when the selection - consists of contiguous lines. - - You can prepend and/or append a line to the included code, using the - ``prepend`` and ``append`` option, respectively. This is useful e.g. for - highlighting PHP code that doesn't include the ``<?php``/``?>`` markers. - - - If you want to show the diff of the code, you can specify the old - file by giving a ``diff`` option:: - - .. literalinclude:: example.py - :diff: example.py.orig - - This shows the diff between example.py and example.py.orig with unified diff - format. - - .. versionadded:: 0.4.3 - The ``encoding`` option. - .. versionadded:: 0.6 - The ``pyobject``, ``lines``, ``start-after`` and ``end-before`` options, - as well as support for absolute filenames. - .. versionadded:: 1.0 - The ``prepend`` and ``append`` options, as well as ``tab-width``. - .. versionadded:: 1.3 - The ``diff`` option. - The ``lineno-match`` option. - .. versionchanged:: 1.6 - With both ``start-after`` and ``lines`` in use, the first line as per - ``start-after`` is considered to be with line number ``1`` for ``lines``. - -Caption and name -^^^^^^^^^^^^^^^^ - -.. versionadded:: 1.3 - -A ``caption`` option can be given to show that name before the code block. -A ``name`` option can be provided implicit target name that can be referenced -by using :rst:role:`ref`. -For example:: - - .. code-block:: python - :caption: this.py - :name: this-py - - print 'Explicit is better than implicit.' - - -:rst:dir:`literalinclude` also supports the ``caption`` and ``name`` option. -``caption`` has an additional feature that if you leave the value empty, the shown -filename will be exactly the one given as an argument. - - -Dedent -^^^^^^ - -.. versionadded:: 1.3 - -A ``dedent`` option can be given to strip indentation characters from the code -block. For example:: - - .. literalinclude:: example.rb - :language: ruby - :dedent: 4 - :lines: 10-15 - -:rst:dir:`code-block` also supports the ``dedent`` option. - - -.. rubric:: Footnotes - -.. [1] There is a standard ``.. include`` directive, but it raises errors if the - file is not found. This one only emits a warning. diff --git a/doc/markup/index.rst b/doc/markup/index.rst deleted file mode 100644 index ca70b9581..000000000 --- a/doc/markup/index.rst +++ /dev/null @@ -1,19 +0,0 @@ -.. _sphinxmarkup: - -Sphinx Markup Constructs -======================== - -Sphinx adds a lot of new directives and interpreted text roles to `standard reST -markup`_. This section contains the reference material for these facilities. - -.. toctree:: - - toctree - para - code - inline - misc - -More markup is added by :ref:`domains`. - -.. _standard reST markup: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html diff --git a/doc/markup/misc.rst b/doc/markup/misc.rst deleted file mode 100644 index 51e3a405c..000000000 --- a/doc/markup/misc.rst +++ /dev/null @@ -1,344 +0,0 @@ -.. highlight:: rest - -Miscellaneous markup -==================== - -.. _metadata: - -File-wide metadata ------------------- - -reST has the concept of "field lists"; these are a sequence of fields marked up -like this:: - - :fieldname: Field content - -A field list near the top of a file is parsed by docutils as the "docinfo" -which is normally used to record the author, date of publication and other -metadata. *In Sphinx*, a field list preceding any other markup is moved from -the docinfo to the Sphinx environment as document metadata and is not displayed -in the output; a field list appearing after the document title will be part of -the docinfo as normal and will be displayed in the output. - -At the moment, these metadata fields are recognized: - -``tocdepth`` - The maximum depth for a table of contents of this file. - - .. versionadded:: 0.4 - -``nocomments`` - If set, the web application won't display a comment form for a page generated - from this source file. - -``orphan`` - If set, warnings about this file not being included in any toctree will be - suppressed. - - .. versionadded:: 1.0 - - -Meta-information markup ------------------------ - -.. rst:directive:: .. sectionauthor:: name <email> - - Identifies the author of the current section. The argument should include - the author's name such that it can be used for presentation and email - address. The domain name portion of the address should be lower case. - Example:: - - .. sectionauthor:: Guido van Rossum <guido@python.org> - - By default, this markup isn't reflected in the output in any way (it helps - keep track of contributions), but you can set the configuration value - :confval:`show_authors` to ``True`` to make them produce a paragraph in the - output. - - -.. rst:directive:: .. codeauthor:: name <email> - - The :rst:dir:`codeauthor` directive, which can appear multiple times, names - the authors of the described code, just like :rst:dir:`sectionauthor` names - the author(s) of a piece of documentation. It too only produces output if - the :confval:`show_authors` configuration value is ``True``. - - -Index-generating markup ------------------------ - -Sphinx automatically creates index entries from all object descriptions (like -functions, classes or attributes) like discussed in :ref:`domains`. - -However, there is also explicit markup available, to make the index more -comprehensive and enable index entries in documents where information is not -mainly contained in information units, such as the language reference. - -.. rst:directive:: .. index:: <entries> - - This directive contains one or more index entries. Each entry consists of a - type and a value, separated by a colon. - - For example:: - - .. index:: - single: execution; context - module: __main__ - module: sys - triple: module; search; path - - The execution context - --------------------- - - ... - - This directive contains five entries, which will be converted to entries in - the generated index which link to the exact location of the index statement - (or, in case of offline media, the corresponding page number). - - Since index directives generate cross-reference targets at their location in - the source, it makes sense to put them *before* the thing they refer to -- - e.g. a heading, as in the example above. - - The possible entry types are: - - single - Creates a single index entry. Can be made a subentry by separating the - subentry text with a semicolon (this notation is also used below to - describe what entries are created). - pair - ``pair: loop; statement`` is a shortcut that creates two index entries, - namely ``loop; statement`` and ``statement; loop``. - triple - Likewise, ``triple: module; search; path`` is a shortcut that creates - three index entries, which are ``module; search path``, ``search; path, - module`` and ``path; module search``. - see - ``see: entry; other`` creates an index entry that refers from ``entry`` to - ``other``. - seealso - Like ``see``, but inserts "see also" instead of "see". - module, keyword, operator, object, exception, statement, builtin - These all create two index entries. For example, ``module: hashlib`` - creates the entries ``module; hashlib`` and ``hashlib; module``. (These - are Python-specific and therefore deprecated.) - - You can mark up "main" index entries by prefixing them with an exclamation - mark. The references to "main" entries are emphasized in the generated - index. For example, if two pages contain :: - - .. index:: Python - - and one page contains :: - - .. index:: ! Python - - then the backlink to the latter page is emphasized among the three backlinks. - - For index directives containing only "single" entries, there is a shorthand - notation:: - - .. index:: BNF, grammar, syntax, notation - - This creates four index entries. - - .. versionchanged:: 1.1 - Added ``see`` and ``seealso`` types, as well as marking main entries. - -.. rst:role:: index - - While the :rst:dir:`index` directive is a block-level markup and links to the - beginning of the next paragraph, there is also a corresponding role that sets - the link target directly where it is used. - - The content of the role can be a simple phrase, which is then kept in the - text and used as an index entry. It can also be a combination of text and - index entry, styled like with explicit targets of cross-references. In that - case, the "target" part can be a full entry as described for the directive - above. For example:: - - This is a normal reST :index:`paragraph` that contains several - :index:`index entries <pair: index; entry>`. - - .. versionadded:: 1.1 - - -.. _tags: - -Including content based on tags -------------------------------- - -.. rst:directive:: .. only:: <expression> - - Include the content of the directive only if the *expression* is true. The - expression should consist of tags, like this:: - - .. only:: html and draft - - Undefined tags are false, defined tags (via the ``-t`` command-line option or - within :file:`conf.py`, see :ref:`here <conf-tags>`) are true. Boolean - expressions, also using parentheses (like ``html and (latex or draft)``) are - supported. - - The *format* and the *name* of the current builder (``html``, ``latex`` or - ``text``) are always set as a tag [#]_. To make the distinction between - format and name explicit, they are also added with the prefix ``format_`` and - ``builder_``, e.g. the epub builder defines the tags ``html``, ``epub``, - ``format_html`` and ``builder_epub``. - - These standard tags are set *after* the configuration file is read, so they - are not available there. - - All tags must follow the standard Python identifier syntax as set out in - the `Identifiers and keywords - <https://docs.python.org/2/reference/lexical_analysis.html#identifiers>`_ - documentation. That is, a tag expression may only consist of tags that - conform to the syntax of Python variables. In ASCII, this consists of the - uppercase and lowercase letters ``A`` through ``Z``, the underscore ``_`` - and, except for the first character, the digits ``0`` through ``9``. - - .. versionadded:: 0.6 - .. versionchanged:: 1.2 - Added the name of the builder and the prefixes. - - .. warning:: - - This directive is designed to control only content of document. It could - not control sections, labels and so on. - - -Tables ------- - -Use :ref:`reStructuredText tables <rst-tables>`, i.e. either - -- grid table syntax (:duref:`ref <grid-tables>`), -- simple table syntax (:duref:`ref <simple-tables>`), -- :dudir:`csv-table` syntax, -- or :dudir:`list-table` syntax. - -The :dudir:`table` directive serves as optional wrapper of the *grid* and -*simple* syntaxes. - -They work fine in -HTML output, however there are some gotchas when using tables in LaTeX: the -column width is hard to determine correctly automatically. For this reason, the -following directive exists: - -.. rst:directive:: .. tabularcolumns:: column spec - - This directive gives a "column spec" for the next table occurring in the - source file. The spec is the second argument to the LaTeX ``tabulary`` - package's environment (which Sphinx uses to translate tables). It can have - values like :: - - |l|l|l| - - which means three left-adjusted, nonbreaking columns. For columns with - longer text that should automatically be broken, use either the standard - ``p{width}`` construct, or tabulary's automatic specifiers: - - +-----+------------------------------------------+ - |``L``| flush left column with automatic width | - +-----+------------------------------------------+ - |``R``| flush right column with automatic width | - +-----+------------------------------------------+ - |``C``| centered column with automatic width | - +-----+------------------------------------------+ - |``J``| justified column with automatic width | - +-----+------------------------------------------+ - - The automatic widths of the ``LRCJ`` columns are attributed by ``tabulary`` - in proportion to the observed shares in a first pass where the table cells - are rendered at their natural "horizontal" widths. - - By default, Sphinx uses a table layout with ``J`` for every column. - - .. versionadded:: 0.3 - - .. versionchanged:: 1.6 - Merged cells may now contain multiple paragraphs and are much better - handled, thanks to custom Sphinx LaTeX macros. This novel situation - motivated the switch to ``J`` specifier and not ``L`` by default. - - .. hint:: - - Sphinx actually uses ``T`` specifier having done ``\newcolumntype{T}{J}``. - To revert to previous default, insert ``\newcolumntype{T}{L}`` in the - LaTeX preamble (see :confval:`latex_elements`). - - A frequent issue with tabulary is that columns with little contents are - "squeezed". The minimal column width is a tabulary parameter called - ``\tymin``. You may set it globally in the LaTeX preamble via - ``\setlength{\tymin}{40pt}`` for example. - - Else, use the :rst:dir:`tabularcolumns` directive with an explicit - ``p{40pt}`` (for example) for that column. You may use also ``l`` - specifier but this makes the task of setting column widths more difficult - if some merged cell intersects that column. - - .. warning:: - - Tables with more than 30 rows are rendered using ``longtable``, not - ``tabulary``, in order to allow pagebreaks. The ``L``, ``R``, ... specifiers - do not work for these tables. - - Tables that contain list-like elements such as object descriptions, - blockquotes or any kind of lists cannot be set out of the box with - ``tabulary``. They are therefore set with the standard LaTeX ``tabular`` (or - ``longtable``) environment if you don't give a ``tabularcolumns`` directive. - If you do, the table will be set with ``tabulary`` but you must use the - ``p{width}`` construct (or Sphinx's ``\X`` and ``\Y`` specifiers described - below) for the columns containing these elements. - - Literal blocks do not work with ``tabulary`` at all, so tables containing - a literal block are always set with ``tabular``. The verbatim environment - used for literal blocks only works in ``p{width}`` (and ``\X`` or ``\Y``) - columns, hence Sphinx generates such column specs for tables containing - literal blocks. - - Since Sphinx 1.5, the ``\X{a}{b}`` specifier is used (there *is* a backslash - in the specifier letter). It is like ``p{width}`` with the width set to a - fraction ``a/b`` of the current line width. You can use it in the - :rst:dir:`tabularcolumns` (it is not a problem if some LaTeX macro is also - called ``\X``.) - - It is *not* needed for ``b`` to be the total number of columns, nor for the - sum of the fractions of the ``\X`` specifiers to add up to one. For example - ``|\X{2}{5}|\X{1}{5}|\X{1}{5}|`` is legitimate and the table will occupy - 80% of the line width, the first of its three columns having the same width - as the sum of the next two. - - This is used by the ``:widths:`` option of the :dudir:`table` directive. - - Since Sphinx 1.6, there is also the ``\Y{f}`` specifier which admits a - decimal argument, such has ``\Y{0.15}``: this would have the same effect as - ``\X{3}{20}``. - - .. versionchanged:: 1.6 - - Merged cells from complex grid tables (either multi-row, multi-column, or - both) now allow blockquotes, lists, literal blocks, ... as do regular cells. - - Sphinx's merged cells interact well with ``p{width}``, ``\X{a}{b}``, ``Y{f}`` - and tabulary's columns. - - .. note:: - - :rst:dir:`tabularcolumns` conflicts with ``:widths:`` option of table - directives. If both are specified, ``:widths:`` option will be ignored. - -Math ----- - -See :ref:`math-support`. - -.. rubric:: Footnotes - -.. [#] For most builders name and format are the same. At the moment only - builders derived from the html builder distinguish between the builder - format and the builder name. - - Note that the current builder tag is not available in ``conf.py``, it is - only available after the builder is initialized. - diff --git a/doc/markup/para.rst b/doc/markup/para.rst deleted file mode 100644 index ba2cc52f5..000000000 --- a/doc/markup/para.rst +++ /dev/null @@ -1,253 +0,0 @@ -.. highlight:: rest - -Paragraph-level markup ----------------------- - -.. index:: note, warning - pair: changes; in version - -These directives create short paragraphs and can be used inside information -units as well as normal text: - -.. rst:directive:: .. note:: - - An especially important bit of information about an API that a user should be - aware of when using whatever bit of API the note pertains to. The content of - the directive should be written in complete sentences and include all - appropriate punctuation. - - Example:: - - .. note:: - - This function is not suitable for sending spam e-mails. - -.. rst:directive:: .. warning:: - - An important bit of information about an API that a user should be very aware - of when using whatever bit of API the warning pertains to. The content of - the directive should be written in complete sentences and include all - appropriate punctuation. This differs from :rst:dir:`note` in that it is - recommended over :rst:dir:`note` for information regarding security. - -.. rst:directive:: .. versionadded:: version - - This directive documents the version of the project which added the described - feature to the library or C API. When this applies to an entire module, it - should be placed at the top of the module section before any prose. - - The first argument must be given and is the version in question; you can add - a second argument consisting of a *brief* explanation of the change. - - Example:: - - .. versionadded:: 2.5 - The *spam* parameter. - - Note that there must be no blank line between the directive head and the - explanation; this is to make these blocks visually continuous in the markup. - -.. rst:directive:: .. versionchanged:: version - - Similar to :rst:dir:`versionadded`, but describes when and what changed in - the named feature in some way (new parameters, changed side effects, etc.). - -.. rst:directive:: .. deprecated:: version - - Similar to :rst:dir:`versionchanged`, but describes when the feature was - deprecated. An explanation can also be given, for example to inform the - reader what should be used instead. Example:: - - .. deprecated:: 3.1 - Use :func:`spam` instead. - - --------------- - -.. rst:directive:: seealso - - Many sections include a list of references to module documentation or - external documents. These lists are created using the :rst:dir:`seealso` - directive. - - The :rst:dir:`seealso` directive is typically placed in a section just before - any subsections. For the HTML output, it is shown boxed off from the main - flow of the text. - - The content of the :rst:dir:`seealso` directive should be a reST definition - list. Example:: - - .. seealso:: - - Module :py:mod:`zipfile` - Documentation of the :py:mod:`zipfile` standard module. - - `GNU tar manual, Basic Tar Format <http://link>`_ - Documentation for tar archive files, including GNU tar extensions. - - There's also a "short form" allowed that looks like this:: - - .. seealso:: modules :py:mod:`zipfile`, :py:mod:`tarfile` - - .. versionadded:: 0.5 - The short form. - -.. rst:directive:: .. rubric:: title - - This directive creates a paragraph heading that is not used to create a - table of contents node. - - .. note:: - - If the *title* of the rubric is "Footnotes" (or the selected language's - equivalent), this rubric is ignored by the LaTeX writer, since it is - assumed to only contain footnote definitions and therefore would create an - empty heading. - - -.. rst:directive:: centered - - This directive creates a centered boldfaced line of text. Use it as - follows:: - - .. centered:: LICENSE AGREEMENT - - .. deprecated:: 1.1 - This presentation-only directive is a legacy from older versions. Use a - :rst:dir:`rst-class` directive instead and add an appropriate style. - - -.. rst:directive:: hlist - - This directive must contain a bullet list. It will transform it into a more - compact list by either distributing more than one item horizontally, or - reducing spacing between items, depending on the builder. - - For builders that support the horizontal distribution, there is a ``columns`` - option that specifies the number of columns; it defaults to 2. Example:: - - .. hlist:: - :columns: 3 - - * A list of - * short items - * that should be - * displayed - * horizontally - - .. versionadded:: 0.6 - - -Table-of-contents markup ------------------------- - -The :rst:dir:`toctree` directive, which generates tables of contents of -subdocuments, is described in :ref:`toctree-directive`. - -For local tables of contents, use the standard reST :dudir:`contents directive -<table-of-contents>`. - - -.. _glossary-directive: - -Glossary --------- - -.. rst:directive:: .. glossary:: - - This directive must contain a reST definition-list-like markup with terms and - definitions. The definitions will then be referencable with the - :rst:role:`term` role. Example:: - - .. glossary:: - - environment - A structure where information about all documents under the root is - saved, and used for cross-referencing. The environment is pickled - after the parsing stage, so that successive runs only need to read - and parse new and changed documents. - - source directory - The directory which, including its subdirectories, contains all - source files for one Sphinx project. - - In contrast to regular definition lists, *multiple* terms per entry are - allowed, and inline markup is allowed in terms. You can link to all of the - terms. For example:: - - .. glossary:: - - term 1 - term 2 - Definition of both terms. - - (When the glossary is sorted, the first term determines the sort order.) - - If you want to specify "grouping key" for general index entries, you can put a "key" - as "term : key". For example:: - - .. glossary:: - - term 1 : A - term 2 : B - Definition of both terms. - - Note that "key" is used for grouping key as is. - The "key" isn't normalized; key "A" and "a" become different groups. - The whole characters in "key" is used instead of a first character; it is used for - "Combining Character Sequence" and "Surrogate Pairs" grouping key. - - In i18n situation, you can specify "localized term : key" even if original text only - have "term" part. In this case, translated "localized term" will be categorized in - "key" group. - - .. versionadded:: 0.6 - You can now give the glossary directive a ``:sorted:`` flag that will - automatically sort the entries alphabetically. - - .. versionchanged:: 1.1 - Now supports multiple terms and inline markup in terms. - - .. versionchanged:: 1.4 - Index key for glossary term should be considered *experimental*. - -Grammar production displays ---------------------------- - -Special markup is available for displaying the productions of a formal grammar. -The markup is simple and does not attempt to model all aspects of BNF (or any -derived forms), but provides enough to allow context-free grammars to be -displayed in a way that causes uses of a symbol to be rendered as hyperlinks to -the definition of the symbol. There is this directive: - -.. rst:directive:: .. productionlist:: [name] - - This directive is used to enclose a group of productions. Each production is - given on a single line and consists of a name, separated by a colon from the - following definition. If the definition spans multiple lines, each - continuation line must begin with a colon placed at the same column as in the - first line. - - The argument to :rst:dir:`productionlist` serves to distinguish different - sets of production lists that belong to different grammars. - - Blank lines are not allowed within ``productionlist`` directive arguments. - - The definition can contain token names which are marked as interpreted text - (e.g. ``sum ::= `integer` "+" `integer```) -- this generates cross-references - to the productions of these tokens. Outside of the production list, you can - reference to token productions using :rst:role:`token`. - - Note that no further reST parsing is done in the production, so that you - don't have to escape ``*`` or ``|`` characters. - -The following is an example taken from the Python Reference Manual:: - - .. productionlist:: - try_stmt: try1_stmt | try2_stmt - try1_stmt: "try" ":" `suite` - : ("except" [`expression` ["," `target`]] ":" `suite`)+ - : ["else" ":" `suite`] - : ["finally" ":" `suite`] - try2_stmt: "try" ":" `suite` - : "finally" ":" `suite` diff --git a/doc/markup/toctree.rst b/doc/markup/toctree.rst deleted file mode 100644 index 78a72c1b4..000000000 --- a/doc/markup/toctree.rst +++ /dev/null @@ -1,242 +0,0 @@ -.. highlight:: rst -.. _toctree-directive: - -The TOC tree -============ - -.. index:: pair: table of; contents - -Since reST does not have facilities to interconnect several documents, or split -documents into multiple output files, Sphinx uses a custom directive to add -relations between the single files the documentation is made of, as well as -tables of contents. The ``toctree`` directive is the central element. - -.. note:: - - Simple "inclusion" of one file in another can be done with the - :dudir:`include` directive. - -.. rst:directive:: toctree - - This directive inserts a "TOC tree" at the current location, using the - individual TOCs (including "sub-TOC trees") of the documents given in the - directive body. Relative document names (not beginning with a slash) are - relative to the document the directive occurs in, absolute names are relative - to the source directory. A numeric ``maxdepth`` option may be given to - indicate the depth of the tree; by default, all levels are included. [#]_ - - Consider this example (taken from the Python docs' library reference index):: - - .. toctree:: - :maxdepth: 2 - - intro - strings - datatypes - numeric - (many more documents listed here) - - This accomplishes two things: - - * Tables of contents from all those documents are inserted, with a maximum - depth of two, that means one nested heading. ``toctree`` directives in - those documents are also taken into account. - * Sphinx knows the relative order of the documents ``intro``, - ``strings`` and so forth, and it knows that they are children of the shown - document, the library index. From this information it generates "next - chapter", "previous chapter" and "parent chapter" links. - - **Entries** - - Document titles in the :rst:dir:`toctree` will be automatically read from the - title of the referenced document. If that isn't what you want, you can - specify an explicit title and target using a similar syntax to reST - hyperlinks (and Sphinx's :ref:`cross-referencing syntax <xref-syntax>`). This - looks like:: - - .. toctree:: - - intro - All about strings <strings> - datatypes - - The second line above will link to the ``strings`` document, but will use the - title "All about strings" instead of the title of the ``strings`` document. - - You can also add external links, by giving an HTTP URL instead of a document - name. - - **Section numbering** - - If you want to have section numbers even in HTML output, give the - **toplevel** toctree a ``numbered`` option. For example:: - - .. toctree:: - :numbered: - - foo - bar - - Numbering then starts at the heading of ``foo``. Sub-toctrees are - automatically numbered (don't give the ``numbered`` flag to those). - - Numbering up to a specific depth is also possible, by giving the depth as a - numeric argument to ``numbered``. - - **Additional options** - - You can use ``caption`` option to provide a toctree caption and you can use - ``name`` option to provide implicit target name that can be referenced by - using :rst:role:`ref`:: - - .. toctree:: - :caption: Table of Contents - :name: mastertoc - - foo - - If you want only the titles of documents in the tree to show up, not other - headings of the same level, you can use the ``titlesonly`` option:: - - .. toctree:: - :titlesonly: - - foo - bar - - You can use "globbing" in toctree directives, by giving the ``glob`` flag - option. All entries are then matched against the list of available - documents, and matches are inserted into the list alphabetically. Example:: - - .. toctree:: - :glob: - - intro* - recipe/* - * - - This includes first all documents whose names start with ``intro``, then all - documents in the ``recipe`` folder, then all remaining documents (except the - one containing the directive, of course.) [#]_ - - The special entry name ``self`` stands for the document containing the - toctree directive. This is useful if you want to generate a "sitemap" from - the toctree. - - You can use the ``reversed`` flag option to reverse the order of the entries - in the list. This can be useful when using the ``glob`` flag option to - reverse the ordering of the files. Example:: - - .. toctree:: - :glob: - :reversed: - - recipe/* - - You can also give a "hidden" option to the directive, like this:: - - .. toctree:: - :hidden: - - doc_1 - doc_2 - - This will still notify Sphinx of the document hierarchy, but not insert links - into the document at the location of the directive -- this makes sense if you - intend to insert these links yourself, in a different style, or in the HTML - sidebar. - - In cases where you want to have only one top-level toctree and hide all other - lower level toctrees you can add the "includehidden" option to the top-level - toctree entry:: - - .. toctree:: - :includehidden: - - doc_1 - doc_2 - - All other toctree entries can then be eliminated by the "hidden" option. - - In the end, all documents in the :term:`source directory` (or subdirectories) - must occur in some ``toctree`` directive; Sphinx will emit a warning if it - finds a file that is not included, because that means that this file will not - be reachable through standard navigation. - - Use :confval:`exclude_patterns` to explicitly exclude documents or - directories from building completely. Use :ref:`the "orphan" metadata - <metadata>` to let a document be built, but notify Sphinx that it is not - reachable via a toctree. - - The "master document" (selected by :confval:`master_doc`) is the "root" of - the TOC tree hierarchy. It can be used as the documentation's main page, or - as a "full table of contents" if you don't give a ``maxdepth`` option. - - .. versionchanged:: 0.3 - Added "globbing" option. - - .. versionchanged:: 0.6 - Added "numbered" and "hidden" options as well as external links and - support for "self" references. - - .. versionchanged:: 1.0 - Added "titlesonly" option. - - .. versionchanged:: 1.1 - Added numeric argument to "numbered". - - .. versionchanged:: 1.2 - Added "includehidden" option. - - .. versionchanged:: 1.3 - Added "caption" and "name" option. - -Special names -------------- - -Sphinx reserves some document names for its own use; you should not try to -create documents with these names -- it will cause problems. - -The special document names (and pages generated for them) are: - -* ``genindex``, ``modindex``, ``search`` - - These are used for the general index, the Python module index, and the search - page, respectively. - - The general index is populated with entries from modules, all index-generating - :ref:`object descriptions <basic-domain-markup>`, and from :rst:dir:`index` - directives. - - The Python module index contains one entry per :rst:dir:`py:module` directive. - - The search page contains a form that uses the generated JSON search index and - JavaScript to full-text search the generated documents for search words; it - should work on every major browser that supports modern JavaScript. - -* every name beginning with ``_`` - - Though only few such names are currently used by Sphinx, you should not create - documents or document-containing directories with such names. (Using ``_`` as - a prefix for a custom template directory is fine.) - -.. warning:: - - Be careful with unusual characters in filenames. Some formats may interpret - these characters in unexpected ways: - - * Do not use the colon ``:`` for HTML based formats. Links to other parts - may not work. - - * Do not use the plus ``+`` for the ePub format. Some resources may not be - found. - -.. rubric:: Footnotes - -.. [#] The LaTeX writer only refers the ``maxdepth`` option of first toctree - directive in the document. - -.. [#] A note on available globbing syntax: you can use the standard shell - constructs ``*``, ``?``, ``[...]`` and ``[!...]`` with the feature that - these all don't match slashes. A double star ``**`` can be used to match - any sequence of characters *including* slashes. diff --git a/doc/pythonorg.png b/doc/pythonorg.png Binary files differdeleted file mode 100644 index cf9ccbbdb..000000000 --- a/doc/pythonorg.png +++ /dev/null diff --git a/doc/setuptools.rst b/doc/setuptools.rst index 8d759f985..10f732559 100644 --- a/doc/setuptools.rst +++ b/doc/setuptools.rst @@ -62,7 +62,7 @@ Options for setuptools integration A boolean that determines whether the saved environment should be discarded on build. Default is false. - This can also be set by passing the `-E` flag to ``setup.py``. + This can also be set by passing the `-E` flag to ``setup.py``: .. code-block:: bash diff --git a/doc/templating.rst b/doc/templating.rst index 66f72716b..e8e4cc415 100644 --- a/doc/templating.rst +++ b/doc/templating.rst @@ -62,7 +62,7 @@ following contents:: {% extends "!layout.html" %} {% block rootrellink %} - <li><a href="http://project.invalid/">Project Homepage</a> »</li> + <li><a href="https://project.invalid/">Project Homepage</a> »</li> {{ super() }} {% endblock %} @@ -70,8 +70,8 @@ By prefixing the name of the overridden template with an exclamation mark, Sphinx will load the layout template from the underlying HTML theme. **Important**: If you override a block, call ``{{ super() }}`` somewhere to -render the block's content in the extended template -- unless you don't want -that content to show up. +render the block's original content in the extended template -- unless you +don't want that content to show up. Working with the builtin templates @@ -228,6 +228,9 @@ them to generate links or output multiply used elements. Return the rendered relation bar. +.. function:: warning(message) + + Emit a warning message. Global Variables ~~~~~~~~~~~~~~~~ diff --git a/doc/theming.rst b/doc/theming.rst index 32aef20fa..fccc2362e 100644 --- a/doc/theming.rst +++ b/doc/theming.rst @@ -1,4 +1,4 @@ -.. highlightlang:: python +.. highlight:: python HTML theming support ==================== @@ -134,8 +134,8 @@ These themes are: Check out at its `installation page`_ how to set up properly :confval:`html_sidebars` for its use. - .. _Alabaster theme: https://pypi.python.org/pypi/alabaster - .. _installation page: http://alabaster.readthedocs.io/en/latest/installation.html + .. _Alabaster theme: https://pypi.org/project/alabaster/ + .. _installation page: https://alabaster.readthedocs.io/en/latest/installation.html * **classic** -- This is the classic theme, which looks like `the Python 2 documentation <https://docs.python.org/2/>`_. It can be customized via @@ -418,7 +418,15 @@ Third Party Themes View a working demo over on readthedocs.org. You can get install and options information at `Read the Docs Sphinx Theme`_ page. - .. _Read the Docs Sphinx Theme: https://pypi.python.org/pypi/sphinx_rtd_theme + .. _Read the Docs Sphinx Theme: https://pypi.org/project/sphinx_rtd_theme/ .. versionchanged:: 1.4 **sphinx_rtd_theme** has become optional. + + +Besides this, there are a lot of third party themes. You can find them on +PyPI__, GitHub__, sphinx-themes.org__ and so on. + +.. __: https://pypi.org/search/?q=&o=&c=Framework+%3A%3A+Sphinx+%3A%3A+Theme +.. __: https://github.com/search?utf8=%E2%9C%93&q=sphinx+theme&type= +.. __: https://sphinx-themes.org/ diff --git a/doc/config.rst b/doc/usage/configuration.rst index d5c32ca4a..10556b063 100644 --- a/doc/config.rst +++ b/doc/usage/configuration.rst @@ -1,9 +1,10 @@ -.. highlightlang:: python +.. highlight:: python .. _build-config: -The build configuration file -============================ +============= +Configuration +============= .. module:: conf :synopsis: Build configuration file. @@ -14,11 +15,10 @@ and contains (almost) all configuration needed to customize Sphinx input and output behavior. An optional file `docutils.conf`_ can be added to the configuration - directory to adjust `Docutils`_ configuration if not otherwise overriden or + directory to adjust `Docutils`_ configuration if not otherwise overridden or set by Sphinx. .. _`docutils`: http://docutils.sourceforge.net/ - .. _`docutils.conf`: http://docutils.sourceforge.net/docs/user/config.html The configuration file is executed as Python code at build time (using @@ -36,8 +36,8 @@ Important points to note: ``"sphinx.builders.Builder"`` means the ``Builder`` class in the ``sphinx.builders`` module. -* Remember that document names use ``/`` as the path separator and don't contain - the file name extension. +* Remember that document names use ``/`` as the path separator and don't + contain the file name extension. * Since :file:`conf.py` is read as a Python file, the usual rules apply for encodings and Unicode support: declare the encoding using an encoding cookie @@ -59,9 +59,6 @@ Important points to note: Note that the current builder tag is not available in ``conf.py``, as it is created *after* the builder is initialized. -.. seealso:: Additional configurations, such as adding stylesheets, - javascripts, builders, etc. can be made through the :doc:`/extdev/appapi`. - General configuration --------------------- @@ -85,17 +82,37 @@ General configuration That way, you can load an extension called ``extname`` from the subdirectory ``sphinxext``. - The configuration file itself can be an extension; for that, you only need to - provide a :func:`setup` function in it. + The configuration file itself can be an extension; for that, you only need + to provide a :func:`setup` function in it. .. confval:: source_suffix - The file name extension, or list of extensions, of source files. Only files - with this suffix will be read as sources. Default is ``'.rst'``. + The file extensions of source files. Sphinx considers the files with this + suffix as sources. This value can be a dictionary mapping file extensions + to file types. For example:: + + source_suffix = { + '.rst': 'restructuredtext', + '.txt': 'restructuredtext', + '.md': 'markdown', + } + + By default, Sphinx only supports ``'restrcturedtext'`` file type. You can + add a new file type using source parser extensions. Please read a document + of the extension to know what file type the extension supports. + + This also allows a list of file extensions. In that case, Sphinx conciders + that all they are ``'restructuredtext'``. Default is + ``{'.rst': 'restructuredtext'}``. + + .. note:: file extensions have to start with dot (like ``.rst``). .. versionchanged:: 1.3 Can now be a list of extensions. + .. versionchanged:: 1.8 + Support file type mapping + .. confval:: source_encoding The encoding of all reST source files. The recommended encoding, and the @@ -119,10 +136,15 @@ General configuration .. note:: - Read more about how to use Markdown with Sphinx at :ref:`markdown`. + Refer to :doc:`/usage/markdown` for more information on using Markdown + with Sphinx. .. versionadded:: 1.3 + .. deprecated:: 1.8 + Now Sphinx provides an API :meth:`Sphinx.add_source_parser` to register + a source parser. Please use it instead. + .. confval:: master_doc The document name of the "master" document, that is, the document that @@ -130,9 +152,10 @@ General configuration .. confval:: exclude_patterns - A list of glob-style patterns that should be excluded when looking for source - files. [1]_ They are matched against the source file names relative to the - source directory, using slashes as directory separators on all platforms. + A list of glob-style patterns that should be excluded when looking for + source files. [1]_ They are matched against the source file names relative + to the source directory, using slashes as directory separators on all + platforms. Example patterns: @@ -160,10 +183,10 @@ General configuration .. confval:: template_bridge - A string with the fully-qualified name of a callable (or simply a class) that - returns an instance of :class:`~sphinx.application.TemplateBridge`. This - instance is then used to render HTML documents, and possibly the output of - other builders (currently the changes builder). (Note that the template + A string with the fully-qualified name of a callable (or simply a class) + that returns an instance of :class:`~sphinx.application.TemplateBridge`. + This instance is then used to render HTML documents, and possibly the output + of other builders (currently the changes builder). (Note that the template bridge must be made theme-aware if HTML themes are to be used.) .. confval:: rst_epilog @@ -171,8 +194,9 @@ General configuration .. index:: pair: global; substitutions A string of reStructuredText that will be included at the end of every source - file that is read. This is the right place to add substitutions that should - be available in every file. An example:: + file that is read. This is a possible place to add substitutions that should + be available in every file (another being :confval:`rst_prolog`). An + example:: rst_epilog = """ .. |psf| replace:: Python Software Foundation @@ -182,8 +206,16 @@ General configuration .. confval:: rst_prolog + .. index:: pair: global; substitutions + A string of reStructuredText that will be included at the beginning of every - source file that is read. + source file that is read. This is a possible place to add substitutions that + should be available in every file (another being :confval:`rst_epilog`). An + example:: + + rst_prolog = """ + .. |psf| replace:: Python Software Foundation + """ .. versionadded:: 1.0 @@ -192,12 +224,12 @@ General configuration .. index:: default; domain primary; domain - The name of the default :ref:`domain <domains>`. Can also be ``None`` to - disable a default domain. The default is ``'py'``. Those objects in other - domains (whether the domain name is given explicitly, or selected by a - :rst:dir:`default-domain` directive) will have the domain name explicitly - prepended when named (e.g., when the default domain is C, Python functions - will be named "Python function", not just "function"). + The name of the default :doc:`domain </usage/restructuredtext/domains>`. + Can also be ``None`` to disable a default domain. The default is ``'py'``. + Those objects in other domains (whether the domain name is given explicitly, + or selected by a :rst:dir:`default-domain` directive) will have the domain + name explicitly prepended when named (e.g., when the default domain is C, + Python functions will be named "Python function", not just "function"). .. versionadded:: 1.0 @@ -217,9 +249,9 @@ General configuration .. confval:: keep_warnings - If true, keep warnings as "system message" paragraphs in the built documents. - Regardless of this setting, warnings are always written to the standard error - stream when ``sphinx-build`` is run. + If true, keep warnings as "system message" paragraphs in the built + documents. Regardless of this setting, warnings are always written to the + standard error stream when ``sphinx-build`` is run. The default is ``False``, the pre-0.5 behavior was to always keep them. @@ -272,8 +304,8 @@ General configuration .. confval:: needs_sphinx If set to a ``major.minor`` version string like ``'1.1'``, Sphinx will - compare it with its version and refuse to build if it is too old. Default is - no requirement. + compare it with its version and refuse to build if it is too old. Default + is no requirement. .. versionadded:: 1.0 @@ -282,8 +314,8 @@ General configuration .. confval:: needs_extensions - This value can be a dictionary specifying version requirements for extensions - in :confval:`extensions`, e.g. ``needs_extensions = + This value can be a dictionary specifying version requirements for + extensions in :confval:`extensions`, e.g. ``needs_extensions = {'sphinxcontrib.something': '1.5'}``. The version strings should be in the form ``major.minor``. Requirements do not have to be specified for all extensions, only for those you want to check. @@ -297,7 +329,7 @@ General configuration A URL to cross-reference :rst:role:`manpage` directives. If this is defined to ``https://manpages.debian.org/{path}``, the - :literal:`:manpage:`man(1)`` role will like to + :literal:`:manpage:`man(1)`` role will link to <https://manpages.debian.org/man(1)>. The patterns available are: * ``page`` - the manual page (``man``) @@ -321,10 +353,10 @@ General configuration .. confval:: nitpick_ignore - A list of ``(type, target)`` tuples (by default empty) that should be ignored - when generating warnings in "nitpicky mode". Note that ``type`` should - include the domain name if present. Example entries would be ``('py:func', - 'int')`` or ``('envvar', 'LD_LIBRARY_PATH')``. + A list of ``(type, target)`` tuples (by default empty) that should be + ignored when generating warnings in "nitpicky mode". Note that ``type`` + should include the domain name if present. Example entries would be + ``('py:func', 'int')`` or ``('envvar', 'LD_LIBRARY_PATH')``. .. versionadded:: 1.1 @@ -336,8 +368,8 @@ General configuration .. note:: - The LaTeX builder always assigns numbers whether this option is enabled or - not. + The LaTeX builder always assigns numbers whether this option is enabled + or not. .. versionadded:: 1.3 @@ -345,7 +377,7 @@ General configuration A dictionary mapping ``'figure'``, ``'table'``, ``'code-block'`` and ``'section'`` to strings that are used for format of figure numbers. - As a special character, `%s` will be replaced to figure number. + As a special character, ``%s`` will be replaced to figure number. Default is to use ``'Fig. %s'`` for ``'figure'``, ``'Table %s'`` for ``'table'``, ``'Listing %s'`` for ``'code-block'`` and ``'Section'`` for @@ -444,6 +476,7 @@ General configuration .. versionadded:: 1.5 + Project information ------------------- @@ -451,6 +484,10 @@ Project information The documented project's name. +.. confval:: author + + The author name(s) of the document. The default value is ``'unknown'``. + .. confval:: copyright A copyright statement in the style ``'2008, Author Name'``. @@ -479,7 +516,7 @@ Project information * Otherwise, the current time is formatted using :func:`time.strftime` and the format given in :confval:`today_fmt`. - The default is no :confval:`today` and a :confval:`today_fmt` of ``'%B %d, + The default is now :confval:`today` and a :confval:`today_fmt` of ``'%B %d, %Y'`` (or, if translation is enabled with :confval:`language`, an equivalent format for the selected locale). @@ -493,7 +530,7 @@ Project information .. versionchanged:: 1.4 The default is now ``'default'``. It is similar to ``'python3'``; - it is mostly a superset of ``'python'``. but it fallbacks to + it is mostly a superset of ``'python'`` but it fallbacks to ``'none'`` without warning if failed. ``'python3'`` and other languages will emit warning if failed. If you prefer Python 2 only highlighting, you can set it back to ``'python'``. @@ -510,7 +547,8 @@ Project information .. confval:: pygments_style The style name to use for Pygments highlighting of source code. If not set, - either the theme's default style or ``'sphinx'`` is selected for HTML output. + either the theme's default style or ``'sphinx'`` is selected for HTML + output. .. versionchanged:: 0.3 If the value is a fully-qualified name of a custom Pygments style class, @@ -545,8 +583,8 @@ Project information .. confval:: trim_footnote_reference_space - Trim spaces before footnote references that are necessary for the reST parser - to recognize the footnote, but do not look too nice in the output. + Trim spaces before footnote references that are necessary for the reST + parser to recognize the footnote, but do not look too nice in the output. .. versionadded:: 0.6 @@ -733,6 +771,7 @@ documentation on :ref:`intl` for details. .. versionchanged:: 1.5 Added ``{path}`` and ``{basename}`` tokens. + .. _html-options: Options for HTML output @@ -744,7 +783,7 @@ that use Sphinx's HTMLWriter class. .. confval:: html_theme The "theme" that the HTML output should use. See the :doc:`section about - theming <theming>`. The default is ``'alabaster'``. + theming </theming>`. The default is ``'alabaster'``. .. versionadded:: 0.6 @@ -766,11 +805,12 @@ that use Sphinx's HTMLWriter class. .. confval:: html_style - The style sheet to use for HTML pages. A file of that name must exist either - in Sphinx's :file:`static/` path, or in one of the custom paths given in - :confval:`html_static_path`. Default is the stylesheet given by the selected - theme. If you only want to add or override a few things compared to the - theme's stylesheet, use CSS ``@import`` to import the theme's stylesheet. + The style sheet to use for HTML pages. A file of that name must exist + either in Sphinx's :file:`static/` path, or in one of the custom paths given + in :confval:`html_static_path`. Default is the stylesheet given by the + selected theme. If you only want to add or override a few things compared + to the theme's stylesheet, use CSS ``@import`` to import the theme's + stylesheet. .. confval:: html_title @@ -781,8 +821,8 @@ that use Sphinx's HTMLWriter class. .. confval:: html_short_title - A shorter "title" for the HTML docs. This is used in for links in the header - and in the HTML Help docs. If not given, it defaults to the value of + A shorter "title" for the HTML docs. This is used in for links in the + header and in the HTML Help docs. If not given, it defaults to the value of :confval:`html_title`. .. versionadded:: 0.4 @@ -818,6 +858,22 @@ that use Sphinx's HTMLWriter class. The image file will be copied to the ``_static`` directory of the output HTML, but only if the file does not already exist there. +.. confval:: html_css_files + + A list of CSS files. The entry must be a *filename* string or a tuple + containing the *filename* string and the *attributes* dictionary. The + *filename* must be relative to the :confval:`html_static_path`, or a full URI + with scheme like ``http://example.org/style.css``. The *attributes* is used + for attributes of ``<link>`` tag. It defaults to an empty list. + + Example:: + + html_css_files = ['custom.css' + 'https://example.com/css/custom.css', + ('print.css', {'media': 'print'})] + + .. versionadded:: 1.8 + .. confval:: html_static_path A list of paths that contain custom static files (such as style @@ -827,12 +883,31 @@ that use Sphinx's HTMLWriter class. named :file:`default.css` will overwrite the theme's :file:`default.css`. + As these files are not meant to be built, they are automatically excluded + from source files. + + .. note:: + + For security reasons, dotfiles under ``html_static_path`` will + not be copied. If you would like to copy them intentionally, please + add each filepath to this setting:: + + html_static_path = ['_static', '_static/.htaccess'] + + Another way to do that, you can also use + :confval:`html_extra_path`. It allows to copy dotfiles under + the directories. + .. versionchanged:: 0.4 The paths in :confval:`html_static_path` can now contain subdirectories. .. versionchanged:: 1.0 The entries in :confval:`html_static_path` can now be single files. + .. versionchanged:: 1.8 + The files under :confval:`html_static_path` are excluded from source + files. + .. confval:: html_extra_path A list of paths that contain extra files not directly related to @@ -841,15 +916,15 @@ that use Sphinx's HTMLWriter class. directory. They are copied to the output directory. They will overwrite any existing file of the same name. - As these files are not meant to be built, they are automatically added to - :confval:`exclude_patterns`. + As these files are not meant to be built, they are automatically excluded + from source files. .. versionadded:: 1.2 .. versionchanged:: 1.4 - The dotfiles in the extra directory will be copied to the output directory. - And it refers :confval:`exclude_patterns` on copying extra files and - directories, and ignores if path matches to patterns. + The dotfiles in the extra directory will be copied to the output + directory. And it refers :confval:`exclude_patterns` on copying extra + files and directories, and ignores if path matches to patterns. .. confval:: html_last_updated_fmt @@ -1004,12 +1079,12 @@ that use Sphinx's HTMLWriter class. .. confval:: html_use_opensearch - If nonempty, an `OpenSearch <http://www.opensearch.org/Home>`_ description file will be - output, and all pages will contain a ``<link>`` tag referring to it. Since - OpenSearch doesn't support relative URLs for its search page location, the - value of this option must be the base URL from which these documents are - served (without trailing slash), e.g. ``"https://docs.python.org"``. The - default is ``''``. + If nonempty, an `OpenSearch <http://www.opensearch.org/Home>`_ description + file will be output, and all pages will contain a ``<link>`` tag referring + to it. Since OpenSearch doesn't support relative URLs for its search page + location, the value of this option must be the base URL from which these + documents are served (without trailing slash), e.g. + ``"https://docs.python.org"``. The default is ``''``. .. confval:: html_file_suffix @@ -1097,8 +1172,8 @@ that use Sphinx's HTMLWriter class. Sphinx uses a Python implementation by default. You can use a C implementation to accelerate building the index file. - * `PorterStemmer <https://pypi.python.org/pypi/PorterStemmer>`_ (``en``) - * `PyStemmer <https://pypi.python.org/pypi/PyStemmer>`_ (all languages) + * `PorterStemmer <https://pypi.org/project/PorterStemmer/>`_ (``en``) + * `PyStemmer <https://pypi.org/project/PyStemmer/>`_ (all languages) .. versionadded:: 1.1 With support for ``en`` and ``ja``. @@ -1116,25 +1191,27 @@ that use Sphinx's HTMLWriter class. The Japanese support has these options: :type: - _`type` is dotted module path string to specify Splitter implementation which - should be derived from :class:`sphinx.search.ja.BaseSplitter`. - If not specified or None is specified, ``'sphinx.search.ja.DefaultSplitter'`` will - be used. + _`type` is dotted module path string to specify Splitter implementation + which should be derived from :class:`sphinx.search.ja.BaseSplitter`. If + not specified or None is specified, + ``'sphinx.search.ja.DefaultSplitter'`` will be used. You can choose from these modules: :'sphinx.search.ja.DefaultSplitter': TinySegmenter algorithm. This is default splitter. :'sphinx.search.ja.MeCabSplitter': - MeCab binding. To use this splitter, 'mecab' python binding or dynamic link - library ('libmecab.so' for linux, 'libmecab.dll' for windows) is required. + MeCab binding. To use this splitter, 'mecab' python binding or dynamic + link library ('libmecab.so' for linux, 'libmecab.dll' for windows) is + required. :'sphinx.search.ja.JanomeSplitter': Janome binding. To use this splitter, - `Janome <https://pypi.python.org/pypi/Janome>`_ is required. - - To keep compatibility, ``'mecab'``, ``'janome'`` and ``'default'`` are also - acceptable. However it will be deprecated in Sphinx-1.6. + `Janome <https://pypi.org/project/Janome/>`_ is required. + .. deprecated:: 1.6 + ``'mecab'``, ``'janome'`` and ``'default'`` is deprecated. + To keep compatibility, ``'mecab'``, ``'janome'`` and ``'default'`` are + also acceptable. Other option values depend on splitter value which you choose. @@ -1144,8 +1221,8 @@ that use Sphinx's HTMLWriter class. :dict: _`dict option` is the dictionary to use for the MeCab algorithm. :lib: - _`lib option` is the library name for finding the MeCab library via ctypes if - the Python binding is not installed. + _`lib option` is the library name for finding the MeCab library via + ctypes if the Python binding is not installed. For example:: @@ -1157,17 +1234,17 @@ that use Sphinx's HTMLWriter class. } Options for ``'janome'``: - :user_dic: _`user_dic option` is the user dictionary file path for Janome. + :user_dic: + _`user_dic option` is the user dictionary file path for Janome. :user_dic_enc: - _`user_dic_enc option` is the encoding for the user dictionary file specified by - ``user_dic`` option. Default is 'utf8'. + _`user_dic_enc option` is the encoding for the user dictionary file + specified by ``user_dic`` option. Default is 'utf8'. .. versionadded:: 1.1 .. versionchanged:: 1.4 - html_search_options for Japanese is re-organized and any custom splitter can be - used by `type`_ settings. - + html_search_options for Japanese is re-organized and any custom splitter + can be used by `type`_ settings. The Chinese support has these options: @@ -1191,16 +1268,24 @@ that use Sphinx's HTMLWriter class. .. versionadded:: 1.3 -.. confval:: htmlhelp_basename - - Output file base name for HTML help builder. Default is ``'pydoc'``. - .. confval:: html_experimental_html5_writer - Output is processed with HTML5 writer. This feature needs docutils 0.13 or newer. Default is ``False``. + Output is processed with HTML5 writer. This feature needs docutils 0.13 or + newer. Default is ``False``. .. versionadded:: 1.6 + +.. _htmlhelp-options: + +Options for HTML help output +----------------------------- + +.. confval:: htmlhelp_basename + + Output file base name for HTML help builder. Default is ``'pydoc'``. + + .. _applehelp-options: Options for Apple Help output @@ -1246,8 +1331,8 @@ HTML builder, so the HTML options also apply where appropriate. .. confval:: applehelp_icon - The help bundle icon file, or ``None`` for no icon. According to Apple’s - documentation, this should be a 16-by-16 pixel version of the application’s + The help bundle icon file, or ``None`` for no icon. According to Apple's + documentation, this should be a 16-by-16 pixel version of the application's icon with a transparent background, saved as a PNG file. .. confval:: applehelp_kb_product @@ -1261,14 +1346,14 @@ HTML builder, so the HTML options also apply where appropriate. e.g. ``https://example.com/kbsearch.py?p='product'&q='query'&l='lang'``. Help Viewer will replace the values ``'product'``, ``'query'`` and ``'lang'`` at runtime with the contents of :confval:`applehelp_kb_product`, - the text entered by the user in the search box and the user’s system + the text entered by the user in the search box and the user's system language respectively. Defaults to ``None`` for no remote search. .. confval:: applehelp_remote_url - The URL for remote content. You can place a copy of your Help Book’s + The URL for remote content. You can place a copy of your Help Book's ``Resources`` folder at this location and Help Viewer will attempt to use it to fetch updated content. @@ -1383,8 +1468,8 @@ the `Dublin Core metadata <http://dublincore.org/>`_. The HTML theme for the epub output. Since the default themes are not optimized for small screen space, using the same theme for HTML and epub - output is usually not wise. This defaults to ``'epub'``, a theme designed to - save visual space. + output is usually not wise. This defaults to ``'epub'``, a theme designed + to save visual space. .. confval:: epub_theme_options @@ -1410,8 +1495,8 @@ the `Dublin Core metadata <http://dublincore.org/>`_. .. confval:: epub_author - The author of the document. This is put in the Dublin Core metadata. The - default value is ``'unknown'``. + The author of the document. This is put in the Dublin Core metadata. It + defaults to the :confval:`author` option. .. confval:: epub_contributor @@ -1431,9 +1516,9 @@ the `Dublin Core metadata <http://dublincore.org/>`_. .. confval:: epub_publisher - The publisher of the document. This is put in the Dublin Core metadata. You - may use any sensible string, e.g. the project homepage. The default value is - ``'unknown'``. + The publisher of the document. This is put in the Dublin Core metadata. + You may use any sensible string, e.g. the project homepage. The defaults to + the :confval:`author` option. .. confval:: epub_copyright @@ -1458,9 +1543,9 @@ the `Dublin Core metadata <http://dublincore.org/>`_. A unique identifier for the document. This is put in the Dublin Core metadata. You may use a - `XML's Name format <https://www.w3.org/TR/REC-xml/#NT-NameStartChar>`_ string. - You can't use hyphen, period, numbers as a first character. - The default value is ``'unknown'``. + `XML's Name format <https://www.w3.org/TR/REC-xml/#NT-NameStartChar>`_ + string. You can't use hyphen, period, numbers as a first character. The + default value is ``'unknown'``. .. confval:: epub_cover @@ -1478,6 +1563,14 @@ the `Dublin Core metadata <http://dublincore.org/>`_. .. versionadded:: 1.1 +.. confval:: epub_css_files + + A list of CSS files. The entry must be a *filename* string or a tuple + containing the *filename* string and the *attributes* dictionary. For more + information, see :confval:`html_css_files`. + + .. versionadded:: 1.8 + .. confval:: epub_guide Meta data for the guide element of :file:`content.opf`. This is a @@ -1606,12 +1699,14 @@ the `Dublin Core metadata <http://dublincore.org/>`_. .. [#] https://developer.mozilla.org/en-US/docs/Web/CSS/writing-mode + .. _latex-options: Options for LaTeX output ------------------------ -These options influence LaTeX output. See further :doc:`latex`. +These options influence LaTeX output. Refer to :doc:`/latex` for more +information. .. confval:: latex_engine @@ -1659,7 +1754,8 @@ These options influence LaTeX output. See further :doc:`latex`. they are to be inserted literally. * *author*: Author for the LaTeX document. The same LaTeX markup caveat as for *title* applies. Use ``\\and`` to separate multiple authors, as in: - ``'John \\and Sarah'`` (backslashes must be Python-escaped to reach LaTeX). + ``'John \\and Sarah'`` (backslashes must be Python-escaped to reach + LaTeX). * *documentclass*: Normally, one of ``'manual'`` or ``'howto'`` (provided by Sphinx and based on ``'report'``, resp. ``'article'``; Japanese documents use ``'jsbook'``, resp. ``'jreport'``.) "howto" (non-Japanese) @@ -1671,7 +1767,8 @@ These options influence LaTeX output. See further :doc:`latex`. * *toctree_only*: Must be ``True`` or ``False``. If true, the *startdoc* document itself is not included in the output, only the documents referenced by it via TOC trees. With this option, you can put extra stuff - in the master document that shows up in the HTML, but not the LaTeX output. + in the master document that shows up in the HTML, but not the LaTeX + output. .. versionadded:: 1.2 In the past including your own document class required you to prepend the @@ -1768,29 +1865,34 @@ These options influence LaTeX output. See further :doc:`latex`. ``'papersize'`` Paper size option of the document class (``'a4paper'`` or ``'letterpaper'``), default ``'letterpaper'``. + ``'pointsize'`` Point size option of the document class (``'10pt'``, ``'11pt'`` or ``'12pt'``), default ``'10pt'``. + ``'pxunit'`` the value of the ``px`` when used in image attributes ``width`` and ``height``. The default value is ``'0.75bp'`` which achieves ``96px=1in`` (in TeX ``1in = 72bp = 72.27pt``.) To obtain for example ``100px=1in`` use ``'0.01in'`` or ``'0.7227pt'`` (the latter leads to TeX computing a more precise value, due to the smaller unit - used in the specification); for ``72px=1in``, - simply use ``'1bp'``; for ``90px=1in``, use ``'0.8bp'`` or ``'0.803pt'``. + used in the specification); for ``72px=1in``, simply use ``'1bp'``; for + ``90px=1in``, use ``'0.8bp'`` or ``'0.803pt'``. .. versionadded:: 1.5 + ``'sphinxsetup'`` A comma separated list of ``key=value`` package options for the Sphinx - LaTeX style, default empty. See :doc:`latex`. + LaTeX style, default empty. See :doc:`/latex`. .. versionadded:: 1.5 + ``'passoptionstopackages'`` A string which will be positioned early in the preamble, designed to contain ``\\PassOptionsToPackage{options}{foo}`` commands. Default empty. .. versionadded:: 1.4 + ``'babel'`` "babel" package inclusion, default ``'\\usepackage{babel}'`` (the suitable document language string is passed as class option, and @@ -1802,6 +1904,7 @@ These options influence LaTeX output. See further :doc:`latex`. is ``'\\usepackage{polyglossia}\n\\setmainlanguage{<language>}'``. .. versionchanged:: 1.6 ``'lualatex'`` uses same default setting as ``'xelatex'`` + ``'fontpkg'`` Font package inclusion, default ``'\\usepackage{times}'`` (which uses Times for text, Helvetica for sans serif and Courier for code-blocks). @@ -1809,12 +1912,10 @@ These options influence LaTeX output. See further :doc:`latex`. .. hint:: Courier is much wider than Times, and Sphinx emits LaTeX command - ``\small`` in code-blocks to compensate. Since ``1.5`` this is not - hard-coded anymore, and can be modified via inclusion in - ``'preamble'`` key of ``\\fvset{fontsize=auto}``. This is - recommended if the fonts match better than Times and Courier. At - ``1.8`` a separate ``'fvset'`` key will permit such customization - without usage of ``'preamble'`` key. + ``\small`` in code-blocks to compensate. Since ``1.5`` this is not + hard-coded anymore: ``\fvset{fontsize=auto}`` can be added to + preamble to not change font size in code-blocks. Since ``1.8`` a + separate ``'fvset'`` key is provided for this. .. versionchanged:: 1.2 Defaults to ``''`` when the :confval:`language` uses the Cyrillic @@ -1823,6 +1924,8 @@ These options influence LaTeX output. See further :doc:`latex`. Defaults to ``''`` when :confval:`latex_engine` is ``'xelatex'``. .. versionchanged:: 1.6 Defaults to ``''`` also with ``'lualatex'``. + .. versionchanged:: 1.8 + ``'xelatex'`` and ``'lualatex'`` do ``\fvset{fontsize=auto}``. ``'fncychap'`` Inclusion of the "fncychap" package (which makes fancy chapter titles), default ``'\\usepackage[Bjarne]{fncychap}'`` for English documentation @@ -1831,12 +1934,15 @@ These options influence LaTeX output. See further :doc:`latex`. the "Bjarne" style uses numbers spelled out in English). Other "fncychap" styles you can try are "Lenny", "Glenn", "Conny", "Rejne" and "Bjornstrup". You can also set this to ``''`` to disable fncychap. + ``'preamble'`` - Additional preamble content, default empty. See :doc:`latex`. + Additional preamble content, default empty. See :doc:`/latex`. + ``'atendofbody'`` Additional document content (right before the indices), default empty. .. versionadded:: 1.5 + ``'figure_align'`` Latex figure float alignment, default 'htbp' (here, top, bottom, page). Whenever an image doesn't fit into the current page, it will be @@ -1845,6 +1951,7 @@ These options influence LaTeX output. See further :doc:`latex`. and position figures strictly in the order they appear in the source. .. versionadded:: 1.3 + ``'footer'`` Additional footer content (before the indices), default empty. @@ -1861,6 +1968,7 @@ These options influence LaTeX output. See further :doc:`latex`. .. versionadded:: 1.2 .. versionchanged:: 1.6 Added this documentation. + ``'maxlistdepth'`` LaTeX allows by default at most 6 levels for nesting list and quote-like environments, with at most 4 enumerated lists, and 4 bullet @@ -1878,6 +1986,7 @@ These options influence LaTeX output. See further :doc:`latex`. dedicated commands of this LaTeX package. .. versionadded:: 1.5 + ``'inputenc'`` "inputenc" package inclusion, defaults to ``'\\usepackage[utf8]{inputenc}'`` when using pdflatex. @@ -1886,10 +1995,12 @@ These options influence LaTeX output. See further :doc:`latex`. .. versionchanged:: 1.4.3 Previously ``'\\usepackage[utf8]{inputenc}'`` was used for all compilers. + ``'cmappkg'`` "cmap" package inclusion, default ``'\\usepackage{cmap}'``. .. versionadded:: 1.2 + ``'fontenc'`` "fontenc" package inclusion, default ``'\\usepackage[T1]{fontenc}'``. @@ -1898,6 +2009,7 @@ These options influence LaTeX output. See further :doc:`latex`. :confval:`latex_engine` is ``'xelatex'``. .. versionchanged:: 1.6 ``'lualatex'`` also uses ``fontspec`` per default. + ``'geometry'`` "geometry" package inclusion, the default definition is: @@ -1928,6 +2040,7 @@ These options influence LaTeX output. See further :doc:`latex`. width will be set to an integer multiple of the *zenkaku* width, and the text height to an integer multiple of the baseline. See the :ref:`hmargin <latexsphinxsetuphmargin>` documentation for more. + ``'hyperref'`` "hyperref" package inclusion; also loads package "hypcap" and issues ``\urlstyle{same}``. This is done after :file:`sphinx.sty` file is @@ -1939,15 +2052,17 @@ These options influence LaTeX output. See further :doc:`latex`. .. versionadded:: 1.5 Previously this was done from inside :file:`sphinx.sty`. + ``'maketitle'`` "maketitle" call, default ``'\\maketitle'`` (but it has been redefined by the Sphinx ``manual`` and ``howto`` classes.) Override - if you want to - generate a differently-styled title page. + if you want to generate a differently-styled title page. + ``'releasename'`` value that prefixes ``'release'`` element on title page, default ``'Release'``. As for *title* and *author* used in the tuples of :confval:`latex_documents`, it is inserted as LaTeX markup. + ``'tableofcontents'`` "tableofcontents" call, default ``'\\sphinxtableofcontents'`` (it is a wrapper of unmodified ``\tableofcontents``, which may itself be @@ -1960,6 +2075,7 @@ These options influence LaTeX output. See further :doc:`latex`. Previously the meaning of ``\tableofcontents`` itself was modified by Sphinx. This created an incompatibility with dedicated packages modifying it also such as "tocloft" or "etoc". + ``'transition'`` Commands used to display transitions, default ``'\n\n\\bigskip\\hrule\\bigskip\n\n'``. Override if you want to @@ -1968,6 +2084,7 @@ These options influence LaTeX output. See further :doc:`latex`. .. versionadded:: 1.2 .. versionchanged:: 1.6 Remove unneeded ``{}`` after ``\\hrule``. + ``'printindex'`` "printindex" call, the last thing in the file, default ``'\\printindex'``. Override if you want to generate the index @@ -1975,6 +2092,17 @@ These options influence LaTeX output. See further :doc:`latex`. ``'\\footnotesize\\raggedright\\printindex'`` is advisable when the index is full of long entries. + ``'fvset'`` + Customization of ``fancyvrb`` LaTeX package. Defaults to + ``'\\fvset{fontsize=\\small}'``, because default font (Courier) used in + code-blocks is wider and taller than default text font (Times). + + For ``'xelatex'`` and ``'lualatex'``, defaults to + ``'\\fvset{fontsize=auto}'``, because the default fonts are part of + one unified typeface family (Latin Modern OpenType). + + .. versionadded:: 1.8 + * Keys that are set by other options and therefore should not be overridden are: @@ -2003,19 +2131,20 @@ These options influence LaTeX output. See further :doc:`latex`. .. confval:: latex_additional_files - A list of file names, relative to the configuration directory, to copy to the - build directory when building LaTeX output. This is useful to copy files - that Sphinx doesn't copy automatically, e.g. if they are referenced in custom - LaTeX added in ``latex_elements``. Image files that are referenced in source - files (e.g. via ``.. image::``) are copied automatically. + A list of file names, relative to the configuration directory, to copy to + the build directory when building LaTeX output. This is useful to copy + files that Sphinx doesn't copy automatically, e.g. if they are referenced in + custom LaTeX added in ``latex_elements``. Image files that are referenced + in source files (e.g. via ``.. image::``) are copied automatically. - You have to make sure yourself that the filenames don't collide with those of - any automatically copied files. + You have to make sure yourself that the filenames don't collide with those + of any automatically copied files. .. versionadded:: 0.6 .. versionchanged:: 1.2 - This overrides the files which is provided from Sphinx such as sphinx.sty. + This overrides the files which is provided from Sphinx such as + ``sphinx.sty``. .. _text-options: @@ -2057,8 +2186,8 @@ These options influence text output. .. confval:: text_secnumber_suffix - Suffix for section numbers in text output. Default: ``". "``. Set to ``" "`` - to suppress the final dot on section numbers. + Suffix for section numbers in text output. Default: ``". "``. Set to + ``" "`` to suppress the final dot on section numbers. .. versionadded:: 1.7 @@ -2076,20 +2205,27 @@ These options influence manual page output. must be a list of tuples ``(startdocname, name, description, authors, section)``, where the items are: - * *startdocname*: document name that is the "root" of the manual page. All - documents referenced by it in TOC trees will be included in the manual file - too. (If you want one master manual page, use your :confval:`master_doc` - here.) - * *name*: name of the manual page. This should be a short string without - spaces or special characters. It is used to determine the file name as - well as the name of the manual page (in the NAME section). - * *description*: description of the manual page. This is used in the NAME - section. - * *authors*: A list of strings with authors, or a single string. Can be an - empty string or list if you do not want to automatically generate an - AUTHORS section in the manual page. - * *section*: The manual page section. Used for the output file name as well - as in the manual page header. + *startdocname* + Document name that is the "root" of the manual page. All documents + referenced by it in TOC trees will be included in the manual file too. + (If you want one master manual page, use your :confval:`master_doc` here.) + + *name* + Name of the manual page. This should be a short string without spaces or + special characters. It is used to determine the file name as well as the + name of the manual page (in the NAME section). + + *description* + Description of the manual page. This is used in the NAME section. + + *authors* + A list of strings with authors, or a single string. Can be an empty + string or list if you do not want to automatically generate an AUTHORS + section in the manual page. + + *section* + The manual page section. Used for the output file name as well as in the + manual page header. .. versionadded:: 1.0 @@ -2114,27 +2250,38 @@ These options influence Texinfo output. author, dir_entry, description, category, toctree_only)``, where the items are: - * *startdocname*: document name that is the "root" of the Texinfo file. All - documents referenced by it in TOC trees will be included in the Texinfo - file too. (If you want only one Texinfo file, use your - :confval:`master_doc` here.) - * *targetname*: file name (no extension) of the Texinfo file in the output - directory. - * *title*: Texinfo document title. Can be empty to use the title of the - *startdoc*. Inserted as Texinfo markup, so special characters like @ and - {} will need to be escaped to be inserted literally. - * *author*: Author for the Texinfo document. Inserted as Texinfo markup. - Use ``@*`` to separate multiple authors, as in: ``'John@*Sarah'``. - * *dir_entry*: The name that will appear in the top-level ``DIR`` menu file. - * *description*: Descriptive text to appear in the top-level ``DIR`` menu - file. - * *category*: Specifies the section which this entry will appear in the - top-level ``DIR`` menu file. - * *toctree_only*: Must be ``True`` or ``False``. If true, the *startdoc* - document itself is not included in the output, only the documents - referenced by it via TOC trees. With this option, you can put extra stuff - in the master document that shows up in the HTML, but not the Texinfo - output. + *startdocname* + Document name that is the "root" of the Texinfo file. All documents + referenced by it in TOC trees will be included in the Texinfo file too. + (If you want only one Texinfo file, use your :confval:`master_doc` here.) + + *targetname* + File name (no extension) of the Texinfo file in the output directory. + + *title* + Texinfo document title. Can be empty to use the title of the *startdoc*. + Inserted as Texinfo markup, so special characters like ``@`` and ``{}`` + will need to be escaped to be inserted literally. + + *author* + Author for the Texinfo document. Inserted as Texinfo markup. Use ``@*`` + to separate multiple authors, as in: ``'John@*Sarah'``. + + *dir_entry* + The name that will appear in the top-level ``DIR`` menu file. + + *description* + Descriptive text to appear in the top-level ``DIR`` menu file. + + *category* + Specifies the section which this entry will appear in the top-level + ``DIR`` menu file. + + *toctree_only* + Must be ``True`` or ``False``. If true, the *startdoc* document itself is + not included in the output, only the documents referenced by it via TOC + trees. With this option, you can put extra stuff in the master document + that shows up in the HTML, but not the Texinfo output. .. versionadded:: 1.1 @@ -2220,7 +2367,8 @@ builder, the HTML options also apply where appropriate. .. confval:: qthelp_basename - The basename for the qthelp file. It defaults to the :confval:`project` name. + The basename for the qthelp file. It defaults to the :confval:`project` + name. .. confval:: qthelp_namespace @@ -2285,7 +2433,7 @@ Options for the linkcheck builder A list of regular expressions that match URIs that should skip checking the validity of anchors in links. This allows skipping entire sites, where anchors are used to control dynamic pages, or just specific anchors within - a page, where javascript is used to add anchors dynamically, or use the + a page, where JavaScript is used to add anchors dynamically, or use the fragment as part of to trigger an internal REST request. Default is ``["/#!"]``. @@ -2306,8 +2454,8 @@ Options for the XML builder .. [1] A note on available globbing syntax: you can use the standard shell constructs ``*``, ``?``, ``[...]`` and ``[!...]`` with the feature that - these all don't match slashes. A double star ``**`` can be used to match - any sequence of characters *including* slashes. + these all don't match slashes. A double star ``**`` can be used to + match any sequence of characters *including* slashes. .. _cpp-config: @@ -2317,29 +2465,32 @@ Options for the C++ domain .. confval:: cpp_index_common_prefix - A list of prefixes that will be ignored when sorting C++ objects in the global index. - For example ``['awesome_lib::']``. + A list of prefixes that will be ignored when sorting C++ objects in the + global index. For example ``['awesome_lib::']``. .. versionadded:: 1.5 .. confval:: cpp_id_attributes A list of strings that the parser additionally should accept as attributes. - This can for example be used when attributes have been ``#define`` d for portability. + This can for example be used when attributes have been ``#define`` d for + portability. .. versionadded:: 1.5 .. confval:: cpp_paren_attributes - A list of strings that the parser additionally should accept as attributes with one argument. - That is, if ``my_align_as`` is in the list, then ``my_align_as(X)`` is parsed as an attribute - for all strings ``X`` that have balanced brances (``()``, ``[]``, and ``{}``). - This can for example be used when attributes have been ``#define`` d for portability. + A list of strings that the parser additionally should accept as attributes + with one argument. That is, if ``my_align_as`` is in the list, then + ``my_align_as(X)`` is parsed as an attribute for all strings ``X`` that have + balanced brances (``()``, ``[]``, and ``{}``). This can for example be used + when attributes have been ``#define`` d for portability. .. versionadded:: 1.5 + Example of configuration file ============================= -.. literalinclude:: _static/conf.py.txt +.. literalinclude:: /_static/conf.py.txt :language: python diff --git a/doc/usage/installation.rst b/doc/usage/installation.rst new file mode 100644 index 000000000..5c0d7de75 --- /dev/null +++ b/doc/usage/installation.rst @@ -0,0 +1,191 @@ +================= +Installing Sphinx +================= + +.. contents:: + :depth: 1 + :local: + :backlinks: none + +.. highlight:: console + +Overview +-------- + +Sphinx is written in `Python`__ and supports both Python 2.7 and Python 3.3+. +We recommend the latter. + +__ http://docs.python-guide.org/en/latest/ + + +Linux +----- + +Debian/Ubuntu +~~~~~~~~~~~~~ + +Install either ``python3-sphinx`` (Python 3) or ``python-sphinx`` (Python 2) +using :command:`apt-get`: + +:: + + $ apt-get install python3-sphinx + +If it not already present, this will install Python for you. + +RHEL, CentOS +~~~~~~~~~~~~ + +Install ``python-sphinx`` using :command:`yum`: + +:: + + $ yum install python-sphinx + +If it not already present, this will install Python for you. + +Other distributions +~~~~~~~~~~~~~~~~~~~ + +Most Linux distributions have Sphinx in their package repositories. Usually +the package is called ``python3-sphinx``, ``python-sphinx`` or ``sphinx``. Be +aware that there are at least two other packages with ``sphinx`` in their name: +a speech recognition toolkit (*CMU Sphinx*) and a full-text search database +(*Sphinx search*). + + +macOS +----- + +Sphinx can be installed using `Homebrew`__, `MacPorts`__, or as part of +a Python distribution such as `Anaconda`__. + +__ https://brew.sh/ +__ https://www.macports.org/ +__ https://www.anaconda.com/download/#macos + +Homebrew +~~~~~~~~ + +:: + + $ brew install sphinx-doc + +For more information, refer to the `package overview`__. + +__ http://formulae.brew.sh/formula/sphinx-doc + +MacPorts +~~~~~~~~ + +Install either ``python36-sphinx`` (Python 3) or ``python27-sphinx`` (Python 2) +using :command:`port`: + +:: + + $ sudo port install py36-sphinx + +To set up the executable paths, use the ``port select`` command: + +:: + + $ sudo port select --set python python36 + $ sudo port select --set sphinx py36-sphinx + +For more information, refer to the `package overview`__. + +__ https://www.macports.org/ports.php?by=library&substr=py36-sphinx + +Anaconda +~~~~~~~~ + +:: + + $ conda install sphinx + +Windows +------- + +.. todo:: Could we start packaging this? + +Most Windows users do not have Python installed by default, so we begin with +the installation of Python itself. If you are unsure, open the *Command +Prompt* (:kbd:`⊞Win-r` and type :command:`cmd`). Once the command prompt is +open, type :command:`python --version` and press Enter. If Python is +available, you will see the version of Python printed to the screen. If you do +not have Python installed, refer to the `Hitchhikers Guide to Python's`__ +Python on Windows installation guides. You can install either `Python 3`__ or +`Python 2.7`__. Python 3 is recommended. + +Once Python is installed, you can install Sphinx using :command:`pip`. Refer +to the :ref:`pip installation instructions <install-pypi>` below for more +information. + +__ http://docs.python-guide.org/en/latest/ +__ http://docs.python-guide.org/en/latest/starting/install3/win/ +__ http://docs.python-guide.org/en/latest/starting/install/win/ + + +.. _install-pypi: + +Installation from PyPI +---------------------- + +Sphinx packages are published on the `Python Package Index +<https://pypi.org/project/Sphinx/>`_. The preferred tool for installing +packages from *PyPI* is :command:`pip`. This tool is provided with all modern +versions of Python. + +On Linux or MacOS, you should open your terminal and run the following command. + +:: + + $ pip install -U sphinx + +On Windows, you should open *Command Prompt* (:kbd:`⊞Win-r` and type +:command:`cmd`) and run the same command. + +.. code-block:: doscon + + C:\> pip install -U sphinx + +After installation, type :command:`sphinx-build --version` on the command +prompt. If everything worked fine, you will see the version number for the +Sphinx package you just installed. + +Installation from *PyPI* also allows you to install the latest development +release. You will not generally need (or want) to do this, but it can be +useful if you see a possible bug in the latest stable release. To do this, use +the ``--pre`` flag. + +:: + + $ pip install -U --pre sphinx + + +Installation from source +------------------------ + +You can install Sphinx directly from a clone of the `Git repository`__. This +can be done either by cloning the repo and installing from the local clone, on +simply installing directly via :command:`git`. + +:: + + $ git clone https://github.com/sphinx-doc/sphinx + $ cd sphinx + $ pip install . + +:: + + $ pip install git+https://github.com/sphinx-doc/sphinx + +You can also download a snapshot of the Git repo in either `tar.gz`__ or +`zip`__ format. Once downloaded and extracted, these can be installed with +:command:`pip` as above. + +.. highlight:: default + +__ https://github.com/sphinx-doc/sphinx +__ https://github.com/sphinx-doc/sphinx/archive/master.tar.gz +__ https://github.com/sphinx-doc/sphinx/archive/master.zip diff --git a/doc/usage/markdown.rst b/doc/usage/markdown.rst new file mode 100644 index 000000000..f67b94cbd --- /dev/null +++ b/doc/usage/markdown.rst @@ -0,0 +1,47 @@ +.. highlight:: python + +.. _markdown: + +======== +Markdown +======== + +`Markdown`__ is a lightweight markup language with a simplistic plain text +formatting syntax. It exists in many syntactically different *flavors*. To +support Markdown-based documentation, Sphinx can use `recommonmark`__. +recommonmark is a Docutils bridge to `CommonMark-py`__, a Python package for +parsing the `CommonMark`__ Markdown flavor. + +__ https://daringfireball.net/projects/markdown/ +__ https://recommonmark.readthedocs.io/en/latest/index.html +__ https://github.com/rtfd/CommonMark-py +__ http://commonmark.org/ + +Configuration +------------- + +To configure your Sphinx project for Markdown support, proceed as follows: + +#. Install *recommonmark*:: + + pip install recommonmark + +#. Add the Markdown parser to the ``source_parsers`` configuration variable in + your Sphinx configuration file:: + + source_parsers = { + '.md': 'recommonmark.parser.CommonMarkParser', + } + + You can replace ``.md`` with a filename extension of your choice. + +#. Add the Markdown filename extension to the ``source_suffix`` configuration + variable:: + + source_suffix = ['.rst', '.md'] + +#. You can further configure *recommonmark* to allow custom syntax that + standard *CommonMark* doesn't support. Read more in the `recommonmark + documentation`__. + +__ https://recommonmark.readthedocs.io/en/latest/auto_structify.html diff --git a/doc/tutorial.rst b/doc/usage/quickstart.rst index 243d44d03..e90b993c6 100644 --- a/doc/tutorial.rst +++ b/doc/usage/quickstart.rst @@ -1,39 +1,33 @@ -.. highlight:: rst +=============== +Getting Started +=============== -First Steps with Sphinx -======================= - -This document is meant to give a tutorial-like overview of all common tasks -while using Sphinx. - -The green arrows designate "more info" links leading to advanced sections about -the described task. - - -Install Sphinx --------------- - -Install Sphinx, either from a distribution package or from -`PyPI <https://pypi.python.org/pypi/Sphinx>`_ with :: - - $ pip install Sphinx +Once Sphinx is :doc:`installed </usage/installation>`, you can proceed with +setting up your first Sphinx project. To ease the process of getting started, +Sphinx provides a tool, :program:`sphinx-quickstart`, which will generate a +documentation source directory and populate it with some defaults. We're going +to use the :program:`sphinx-quickstart` tool here, though it's use by no means +necessary. Setting up the documentation sources ------------------------------------ -The root directory of a Sphinx collection of reStructuredText document sources -is called the :term:`source directory`. This directory also contains the Sphinx -configuration file :file:`conf.py`, where you can configure all aspects of how -Sphinx reads your sources and builds your documentation. [#]_ +The root directory of a Sphinx collection of :term:`reStructuredText` document +sources is called the :term:`source directory`. This directory also contains +the Sphinx configuration file :file:`conf.py`, where you can configure all +aspects of how Sphinx reads your sources and builds your documentation. [#]_ Sphinx comes with a script called :program:`sphinx-quickstart` that sets up a source directory and creates a default :file:`conf.py` with the most useful -configuration values from a few questions it asks you. Just run :: +configuration values from a few questions it asks you. To use this, run: + +.. code-block:: shell $ sphinx-quickstart -and answer its questions. (Be sure to say yes to the "autodoc" extension.) +Answer each question asked. Be sure to say yes to the ``autodoc`` extension, as +we will use this later. There is also an automatic "API documentation" generator called :program:`sphinx-apidoc`; see :doc:`/man/sphinx-apidoc` for details. @@ -45,15 +39,15 @@ Defining document structure Let's assume you've run :program:`sphinx-quickstart`. It created a source directory with :file:`conf.py` and a master document, :file:`index.rst` (if you accepted the defaults). The main function of the :term:`master document` is to -serve as a welcome page, and to contain the root of the "table of contents tree" -(or *toctree*). This is one of the main things that Sphinx adds to +serve as a welcome page, and to contain the root of the "table of contents +tree" (or *toctree*). This is one of the main things that Sphinx adds to reStructuredText, a way to connect multiple files to a single hierarchy of documents. .. sidebar:: reStructuredText directives - ``toctree`` is a reStructuredText :dfn:`directive`, a very versatile piece of - markup. Directives can have arguments, options and content. + ``toctree`` is a reStructuredText :dfn:`directive`, a very versatile piece + of markup. Directives can have arguments, options and content. *Arguments* are given directly after the double colon following the directive's name. Each directive decides whether it can have arguments, and @@ -68,56 +62,63 @@ documents. A common gotcha with directives is that **the first line of the content must be indented to the same level as the options are**. +The ``toctree`` directive initially is empty, and looks like so: -The toctree directive initially is empty, and looks like this:: +.. code-block:: rest .. toctree:: :maxdepth: 2 -You add documents listing them in the *content* of the directive:: +You add documents listing them in the *content* of the directive: + +.. code-block:: rest .. toctree:: :maxdepth: 2 - intro - tutorial + usage/installation + usage/quickstart ... -This is exactly how the toctree for this documentation looks. The documents to -include are given as :term:`document name`\ s, which in short means that you -leave off the file name extension and use slashes as directory separators. +This is exactly how the ``toctree`` for this documentation looks. The +documents to include are given as :term:`document name`\ s, which in short +means that you leave off the file name extension and use forward slashes +(``/``) as directory separators. |more| Read more about :ref:`the toctree directive <toctree-directive>`. -You can now create the files you listed in the toctree and add content, and -their section titles will be inserted (up to the "maxdepth" level) at the place -where the toctree directive is placed. Also, Sphinx now knows about the order -and hierarchy of your documents. (They may contain ``toctree`` directives -themselves, which means you can create deeply nested hierarchies if necessary.) +You can now create the files you listed in the ``toctree`` and add content, and +their section titles will be inserted (up to the ``maxdepth`` level) at the +place where the ``toctree`` directive is placed. Also, Sphinx now knows about +the order and hierarchy of your documents. (They may contain ``toctree`` +directives themselves, which means you can create deeply nested hierarchies if +necessary.) Adding content -------------- -In Sphinx source files, you can use most features of standard reStructuredText. -There are also several features added by Sphinx. For example, you can add -cross-file references in a portable way (which works for all output types) using -the :rst:role:`ref` role. +In Sphinx source files, you can use most features of standard +:term:`reStructuredText`. There are also several features added by Sphinx. +For example, you can add cross-file references in a portable way (which works +for all output types) using the :rst:role:`ref` role. For an example, if you are viewing the HTML version you can look at the source for this document -- use the "Show Source" link in the sidebar. -|more| See :ref:`rst-primer` for a more in-depth introduction to -reStructuredText and :ref:`sphinxmarkup` for a full list of markup added by -Sphinx. +.. todo:: Update the below link when we add new guides on these. + +|more| See :doc:`/usage/restructuredtext/index` for a more in-depth +introduction to reStructuredText, including markup added by Sphinx. Running the build ----------------- Now that you have added some files and content, let's make a first build of the -docs. A build is started with the :program:`sphinx-build` program, called like -this:: +docs. A build is started with the :program:`sphinx-build` program: + +.. code-block:: shell $ sphinx-build -b html sourcedir builddir @@ -130,13 +131,15 @@ Sphinx will build HTML files. options that :program:`sphinx-build` supports. However, :program:`sphinx-quickstart` script creates a :file:`Makefile` and a -:file:`make.bat` which make life even easier for you: with them you only need -to run :: +:file:`make.bat` which make life even easier for you. These can be executed by +running :command:`make` with the name of the builder. For example. + +.. code-block:: shell $ make html -to build HTML docs in the build directory you chose. Execute ``make`` without -an argument to see which targets are available. +This will build HTML docs in the build directory you chose. Execute +:command:`make` without an argument to see which targets are available. .. admonition:: How do I generate PDF documents? @@ -145,6 +148,8 @@ an argument to see which targets are available. toolchain for you. +.. todo:: Move this whole section into a guide on rST or directives + Documenting objects ------------------- @@ -155,7 +160,9 @@ descriptions of these objects. The most prominent domain is the Python domain. For example, to document Python's built-in function ``enumerate()``, you would add this to one of your -source files:: +source files. + +.. code-block:: restructuredtext .. py:function:: enumerate(sequence[, start=0]) @@ -174,7 +181,9 @@ describe, the content is the documentation for it. Multiple signatures can be given, each in its own line. The Python domain also happens to be the default domain, so you don't need to -prefix the markup with the domain name:: +prefix the markup with the domain name. + +.. code-block:: restructuredtext .. function:: enumerate(sequence[, start=0]) @@ -182,50 +191,56 @@ prefix the markup with the domain name:: does the same job if you keep the default setting for the default domain. -There are several more directives for documenting other types of Python objects, -for example :rst:dir:`py:class` or :rst:dir:`py:method`. There is also a -cross-referencing :dfn:`role` for each of these object types. This markup will -create a link to the documentation of ``enumerate()``:: +There are several more directives for documenting other types of Python +objects, for example :rst:dir:`py:class` or :rst:dir:`py:method`. There is +also a cross-referencing :dfn:`role` for each of these object types. This +markup will create a link to the documentation of ``enumerate()``. + +:: The :py:func:`enumerate` function can be used for ... And here is the proof: A link to :func:`enumerate`. Again, the ``py:`` can be left out if the Python domain is the default one. It -doesn't matter which file contains the actual documentation for ``enumerate()``; -Sphinx will find it and create a link to it. +doesn't matter which file contains the actual documentation for +``enumerate()``; Sphinx will find it and create a link to it. Each domain will have special rules for how the signatures can look like, and make the formatted output look pretty, or add specific features like links to parameter types, e.g. in the C/C++ domains. -|more| See :ref:`domains` for all the available domains and their -directives/roles. +|more| See :doc:`/usage/restructuredtext/domains` for all the available domains +and their directives/roles. Basic configuration ------------------- -Earlier we mentioned that the :file:`conf.py` file controls how Sphinx processes -your documents. In that file, which is executed as a Python source file, you -assign configuration values. For advanced users: since it is executed by -Sphinx, you can do non-trivial tasks in it, like extending :data:`sys.path` or -importing a module to find out the version you are documenting. +Earlier we mentioned that the :file:`conf.py` file controls how Sphinx +processes your documents. In that file, which is executed as a Python source +file, you assign configuration values. For advanced users: since it is +executed by Sphinx, you can do non-trivial tasks in it, like extending +:data:`sys.path` or importing a module to find out the version you are +documenting. The config values that you probably want to change are already put into the :file:`conf.py` by :program:`sphinx-quickstart` and initially commented out -(with standard Python syntax: a ``#`` comments the rest of the line). To change -the default value, remove the hash sign and modify the value. To customize a -config value that is not automatically added by :program:`sphinx-quickstart`, -just add an additional assignment. +(with standard Python syntax: a ``#`` comments the rest of the line). To +change the default value, remove the hash sign and modify the value. To +customize a config value that is not automatically added by +:program:`sphinx-quickstart`, just add an additional assignment. -Keep in mind that the file uses Python syntax for strings, numbers, lists and so -on. The file is saved in UTF-8 by default, as indicated by the encoding +Keep in mind that the file uses Python syntax for strings, numbers, lists and +so on. The file is saved in UTF-8 by default, as indicated by the encoding declaration in the first line. If you use non-ASCII characters in any string value, you need to use Python Unicode strings (like ``project = u'Exposé'``). -|more| See :ref:`build-config` for documentation of all available config values. +|more| See :doc:`/usage/configuration` for documentation of all available +config values. + +.. todo:: Move this entire doc to a different section Autodoc ------- @@ -233,15 +248,15 @@ Autodoc When documenting Python code, it is common to put a lot of documentation in the source files, in documentation strings. Sphinx supports the inclusion of docstrings from your modules with an :dfn:`extension` (an extension is a Python -module that provides additional features for Sphinx projects) called "autodoc". +module that provides additional features for Sphinx projects) called *autodoc*. -In order to use autodoc, you need to activate it in :file:`conf.py` by putting -the string ``'sphinx.ext.autodoc'`` into the list assigned to the +In order to use *autodoc*, you need to activate it in :file:`conf.py` by +putting the string ``'sphinx.ext.autodoc'`` into the list assigned to the :confval:`extensions` config value. Then, you have a few additional directives at your disposal. -For example, to document the function ``io.open()``, reading its -signature and docstring from the source file, you'd write this:: +For example, to document the function ``io.open()``, reading its signature and +docstring from the source file, you'd write this:: .. autofunction:: io.open @@ -251,7 +266,7 @@ options for the auto directives, like :: .. automodule:: io :members: -autodoc needs to import your modules in order to extract the docstrings. +*autodoc* needs to import your modules in order to extract the docstrings. Therefore, you must add the appropriate path to :py:data:`sys.path` in your :file:`conf.py`. @@ -261,17 +276,20 @@ Therefore, you must add the appropriate path to :py:data:`sys.path` in your modules have side effects on import, these will be executed by ``autodoc`` when ``sphinx-build`` is run. - If you document scripts (as opposed to library modules), make sure their main - routine is protected by a ``if __name__ == '__main__'`` condition. + If you document scripts (as opposed to library modules), make sure their + main routine is protected by a ``if __name__ == '__main__'`` condition. |more| See :mod:`sphinx.ext.autodoc` for the complete description of the features of autodoc. + +.. todo:: Move this doc to another section + Intersphinx ----------- -Many Sphinx documents including the `Python documentation`_ are published on the -internet. When you want to make links to such documents from your +Many Sphinx documents including the `Python documentation`_ are published on +the internet. When you want to make links to such documents from your documentation, you can do it with :mod:`sphinx.ext.intersphinx`. .. _Python documentation: https://docs.python.org/3 @@ -290,7 +308,7 @@ cross-reference that has no matching target in the current documentation set, will be looked up in the documentation sets configured in :confval:`intersphinx_mapping` (this needs access to the URL in order to download the list of valid targets). Intersphinx also works for some other -:ref:`domains' <domains>` roles including ``:ref:``, however it doesn't work for +:term:`domain`\'s roles including ``:ref:``, however it doesn't work for ``:doc:`` as that is non-domain role. |more| See :mod:`sphinx.ext.intersphinx` for the complete description of the @@ -300,15 +318,15 @@ features of intersphinx. More topics to be covered ------------------------- -- :doc:`Other extensions <extensions>`: +- :doc:`Other extensions </extensions>`: - * :doc:`ext/math`, - * :doc:`ext/viewcode`, - * :doc:`ext/doctest`, + * :doc:`/ext/math`, + * :doc:`/ext/viewcode`, + * :doc:`/ext/doctest`, * ... - Static files -- :doc:`Selecting a theme <theming>` -- :doc:`setuptools` +- :doc:`Selecting a theme </theming>` +- :doc:`/setuptools` - :ref:`Templating <templating>` - Using extensions - :ref:`Writing extensions <dev-extensions>` @@ -320,6 +338,6 @@ More topics to be covered another directory, the :term:`configuration directory`. Refer to the :program:`sphinx-build man page <sphinx-build>` for more information. -.. |more| image:: more.png +.. |more| image:: /_static/more.png :align: middle :alt: more info diff --git a/doc/rest.rst b/doc/usage/restructuredtext/basics.rst index fbc3f2254..3e1253544 100644 --- a/doc/rest.rst +++ b/doc/usage/restructuredtext/basics.rst @@ -1,12 +1,14 @@ -.. highlightlang:: rest +.. highlight:: rst .. _rst-primer: +======================= reStructuredText Primer ======================= -This section is a brief introduction to reStructuredText (reST) concepts and -syntax, intended to provide authors with enough information to author documents +reStructuredText is the default plaintext markup language used by Sphinx. This +section is a brief introduction to reStructuredText (reST) concepts and syntax, +intended to provide authors with enough information to author documents productively. Since reST was designed to be a simple, unobtrusive markup language, this will not take too long. @@ -27,7 +29,7 @@ lines. As in Python, indentation is significant in reST, so all lines of the same paragraph must be left-aligned to the same level of indentation. -.. _inlinemarkup: +.. _rst-inline-markup: Inline markup ------------- @@ -50,30 +52,16 @@ Be aware of some restrictions of this markup: These restrictions may be lifted in future versions of the docutils. -reST also allows for custom "interpreted text roles", which signify that the -enclosed text should be interpreted in a specific way. Sphinx uses this to -provide semantic markup and cross-referencing of identifiers, as described in -the appropriate section. The general syntax is ``:rolename:`content```. - -Standard reST provides the following roles: - -* :durole:`emphasis` -- alternate spelling for ``*emphasis*`` -* :durole:`strong` -- alternate spelling for ``**strong**`` -* :durole:`literal` -- alternate spelling for ````literal```` -* :durole:`subscript` -- subscript text -* :durole:`superscript` -- superscript text -* :durole:`title-reference` -- for titles of books, periodicals, and other - materials - -See :ref:`inline-markup` for roles added by Sphinx. +It is also possible to replace or expand upon some of this inline markup with +roles. Refer to :ref:`rst-roles-alt` for more information. Lists and Quote-like blocks --------------------------- List markup (:duref:`ref <bullet-lists>`) is natural: just place an asterisk at -the start of a paragraph and indent properly. The same goes for numbered lists; -they can also be autonumbered using a ``#`` sign:: +the start of a paragraph and indent properly. The same goes for numbered +lists; they can also be autonumbered using a ``#`` sign:: * This is a bulleted list. * It has two items, the second @@ -85,7 +73,6 @@ they can also be autonumbered using a ``#`` sign:: #. This is a numbered list. #. It has two items too. - Nested lists are possible, but be aware that they must be separated from the parent list items by blank lines:: @@ -120,18 +107,22 @@ Line blocks (:duref:`ref <line-blocks>`) are a way of preserving line breaks:: There are also several more special blocks available: -* field lists (:duref:`ref <field-lists>`) +* field lists (:duref:`ref <field-lists>`, with caveats noted in + :ref:`rst-field-lists`) * option lists (:duref:`ref <option-lists>`) * quoted literal blocks (:duref:`ref <quoted-literal-blocks>`) * doctest blocks (:duref:`ref <doctest-blocks>`) -Source Code ------------ +.. _rst-literal-blocks: + +Literal blocks +-------------- Literal code blocks (:duref:`ref <literal-blocks>`) are introduced by ending a paragraph with the special marker ``::``. The literal block must be indented -(and, like all paragraphs, separated from the surrounding ones by blank lines):: +(and, like all paragraphs, separated from the surrounding ones by blank +lines):: This is a normal text paragraph. The next paragraph is a code sample:: @@ -144,8 +135,8 @@ paragraph with the special marker ``::``. The literal block must be indented The handling of the ``::`` marker is smart: -* If it occurs as a paragraph of its own, that paragraph is completely left - out of the document. +* If it occurs as a paragraph of its own, that paragraph is completely left out + of the document. * If it is preceded by whitespace, the marker is removed. * If it is preceded by non-whitespace, the marker is replaced by a single colon. @@ -153,15 +144,33 @@ The handling of the ``::`` marker is smart: That way, the second sentence in the above example's first paragraph would be rendered as "The next paragraph is a code sample:". +Code highlighting can be enabled for these literal blocks on a document-wide +basis using the :rst:dir:`highlight` directive and on a project-wide basis +using the :confval:`highlight_language` configuration option. The +:rst:dir:`code-block` directive can be used to set highlighting on a +block-by-block basis. These directives are discussed later. + + +.. _rst-doctest-blocks: + +Doctest blocks +-------------- + +Doctest blocks (:duref:`ref <doctest-blocks>`) are interactive Python sessions +cut-and-pasted into docstrings. They do not require the +:ref:`literal blocks <rst-literal-blocks>` syntax. The doctest block must end +with a blank line and should *not* end with with an unused prompt:: + + >>> 1 + 1 + 2 .. _rst-tables: Tables ------ -For *grid tables* (:duref:`ref -<grid-tables>`), you have to "paint" the cell grid yourself. They look like -this:: +For *grid tables* (:duref:`ref <grid-tables>`), you have to "paint" the cell +grid yourself. They look like this:: +------------------------+------------+----------+----------+ | Header row, column 1 | Header 2 | Header 3 | Header 4 | @@ -186,17 +195,18 @@ contain multiple lines. They look like this:: ===== ===== ======= Two more syntaxes are supported: *CSV tables* and *List tables*. They use an -*explicit markup block*, see `Directives`_ section. +*explicit markup block*. Refer to :ref:`table-directives` for more information. + Hyperlinks ---------- External links -^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~ -Use ```Link text <http://example.com/>`_`` for inline web links. If the link -text should be the web address, you don't need special markup at all, the parser -finds links and mail addresses in ordinary text. +Use ```Link text <https://domain.invalid/>`_`` for inline web links. If the +link text should be the web address, you don't need special markup at all, the +parser finds links and mail addresses in ordinary text. .. important:: There must be a space between the link text and the opening \< for the URL. @@ -205,11 +215,10 @@ You can also separate the link and the target definition (:duref:`ref This is a paragraph that contains `a link`_. - .. _a link: http://example.com/ - + .. _a link: https://domain.invalid/ Internal links -^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~ Internal linking is done via a special reST role provided by Sphinx, see the section on specific markup, :ref:`ref-role`. @@ -244,6 +253,59 @@ documentation), and use a deeper nesting level, but keep in mind that most target formats (HTML, LaTeX) have a limited supported nesting depth. +.. _rst-field-lists: + +Field Lists +----------- + +Field lists (:duref:`ref <field-lists>`) are sequences of fields marked up like +this:: + + :fieldname: Field content + +They are commonly used in Python documentation:: + + def my_function(my_arg, my_other_arg): + """A function just for me. + + :param my_arg: The first of my arguments. + :param my_other_arg: The second of my arguments. + + :returns: A message (just for me, of course). + """ + +Sphinx extends standard docutils behavior and intercepts field lists specified +at the beginning of documents. Refer to :doc:`field-lists` for more +information. + + +.. TODO This ref should be 'rst-roles', but that already exists. Rename the +.. other ones + +.. _rst-roles-alt: + +Roles +----- + +A role or "custom interpreted text role" (:duref:`ref <roles>`) is an inline +piece of explicit markup. It signifies that that the enclosed text should be +interpreted in a specific way. Sphinx uses this to provide semantic markup and +cross-referencing of identifiers, as described in the appropriate section. The +general syntax is ``:rolename:`content```. + +Docutils supports the following roles: + +* :durole:`emphasis` -- equivalent of ``*emphasis*`` +* :durole:`strong` -- equivalent of ``**strong**`` +* :durole:`literal` -- equivalent of ````literal```` +* :durole:`subscript` -- subscript text +* :durole:`superscript` -- superscript text +* :durole:`title-reference` -- for titles of books, periodicals, and other + materials + +Refer to :doc:`roles` for roles added by Sphinx. + + Explicit Markup --------------- @@ -253,19 +315,19 @@ specially-highlighted paragraphs, comments, and generic directives. An explicit markup block begins with a line starting with ``..`` followed by whitespace and is terminated by the next paragraph at the same level of -indentation. (There needs to be a blank line between explicit markup and normal -paragraphs. This may all sound a bit complicated, but it is intuitive enough -when you write it.) +indentation. (There needs to be a blank line between explicit markup and +normal paragraphs. This may all sound a bit complicated, but it is intuitive +enough when you write it.) -.. _directives: +.. _rst-directives: Directives ---------- A directive (:duref:`ref <directives>`) is a generic block of explicit markup. -Besides roles, it is one of the extension mechanisms of reST, and Sphinx makes -heavy use of it. +Along with roles, it is one of the extension mechanisms of reST, and Sphinx +makes heavy use of it. Docutils supports the following directives: @@ -303,9 +365,9 @@ Docutils supports the following directives: * Special directives: - :dudir:`raw <raw-data-pass-through>` (include raw target-format markup) - - :dudir:`include` (include reStructuredText from another file) - -- in Sphinx, when given an absolute include file path, this directive takes - it as relative to the source directory + - :dudir:`include` (include reStructuredText from another file) -- in Sphinx, + when given an absolute include file path, this directive takes it as + relative to the source directory - :dudir:`class` (assign a class attribute to the next element) [1]_ * HTML specifics: @@ -321,14 +383,16 @@ Docutils supports the following directives: Since these are only per-file, better use Sphinx's facilities for setting the :confval:`default_role`. -Do *not* use the directives :dudir:`sectnum`, :dudir:`header` and -:dudir:`footer`. +.. warning:: -Directives added by Sphinx are described in :ref:`sphinxmarkup`. + Do *not* use the directives :dudir:`sectnum`, :dudir:`header` and + :dudir:`footer`. -Basically, a directive consists of a name, arguments, options and content. (Keep -this terminology in mind, it is used in the next chapter describing custom -directives.) Looking at this example, :: +Directives added by Sphinx are described in :doc:`directives`. + +Basically, a directive consists of a name, arguments, options and content. +(Keep this terminology in mind, it is used in the next chapter describing +custom directives.) Looking at this example, :: .. function:: foo(x) foo(y, z) @@ -338,12 +402,12 @@ directives.) Looking at this example, :: ``function`` is the directive name. It is given two arguments here, the remainder of the first line and the second line, as well as one option -``module`` (as you can see, options are given in the lines immediately following -the arguments and indicated by the colons). Options must be indented to the -same level as the directive content. +``module`` (as you can see, options are given in the lines immediately +following the arguments and indicated by the colons). Options must be indented +to the same level as the directive content. -The directive content follows after a blank line and is indented relative to the -directive start. +The directive content follows after a blank line and is indented relative to +the directive start. Images @@ -356,8 +420,8 @@ reST supports an image directive (:dudir:`ref <image>`), used like so:: When used within Sphinx, the file name given (here ``gnu.png``) must either be relative to the source file, or absolute which means that they are relative to -the top source directory. For example, the file ``sketch/spam.rst`` could refer -to the image ``images/spam.png`` as ``../images/spam.png`` or +the top source directory. For example, the file ``sketch/spam.rst`` could +refer to the image ``images/spam.png`` as ``../images/spam.png`` or ``/images/spam.png``. Sphinx will automatically copy image files over to a subdirectory of the output @@ -365,21 +429,22 @@ directory on building (e.g. the ``_static`` directory for HTML output.) Interpretation of image size options (``width`` and ``height``) is as follows: if the size has no unit or the unit is pixels, the given size will only be -respected for output channels that support pixels. Other units (like ``pt`` -for points) will be used for HTML and LaTeX output (the latter replaces ``pt`` -by ``bp`` as this is the TeX unit such that ``72bp=1in``). +respected for output channels that support pixels. Other units (like ``pt`` for +points) will be used for HTML and LaTeX output (the latter replaces ``pt`` by +``bp`` as this is the TeX unit such that ``72bp=1in``). Sphinx extends the standard docutils behavior by allowing an asterisk for the extension:: .. image:: gnu.* -Sphinx then searches for all images matching the provided pattern and determines -their type. Each builder then chooses the best image out of these candidates. -For instance, if the file name ``gnu.*`` was given and two files :file:`gnu.pdf` -and :file:`gnu.png` existed in the source tree, the LaTeX builder would choose -the former, while the HTML builder would prefer the latter. -Supported image types and choosing priority are defined at :ref:`builders`. +Sphinx then searches for all images matching the provided pattern and +determines their type. Each builder then chooses the best image out of these +candidates. For instance, if the file name ``gnu.*`` was given and two files +:file:`gnu.pdf` and :file:`gnu.png` existed in the source tree, the LaTeX +builder would choose the former, while the HTML builder would prefer the +latter. Supported image types and choosing priority are defined at +:ref:`builders`. Note that image file names should not contain spaces. @@ -443,11 +508,14 @@ or this:: See the :duref:`reST reference for substitutions <substitution-definitions>` for details. +.. index:: ! pair: global; substitutions + If you want to use some substitutions for all documents, put them into -:confval:`rst_prolog` or put them into a separate file and include it into all -documents you want to use them in, using the :rst:dir:`include` directive. (Be -sure to give the include file a file name extension differing from that of other -source files, to avoid Sphinx finding it as a standalone document.) +:confval:`rst_prolog` or :confval:`rst_epilog` or put them into a separate file +and include it into all documents you want to use them in, using the +:rst:dir:`include` directive. (Be sure to give the include file a file name +extension differing from that of other source files, to avoid Sphinx finding it +as a standalone document.) Sphinx defines some default substitutions, see :ref:`default-substitutions`. @@ -486,8 +554,8 @@ There are some problems one commonly runs into while authoring reST documents: * **Separation of inline markup:** As said above, inline markup spans must be separated from the surrounding text by non-word characters, you have to use a - backslash-escaped space to get around that. See - :duref:`the reference <substitution-definitions>` for the details. + backslash-escaped space to get around that. See :duref:`the reference + <substitution-definitions>` for the details. * **No nested inline markup:** Something like ``*see :func:`foo`*`` is not possible. diff --git a/doc/usage/restructuredtext/directives.rst b/doc/usage/restructuredtext/directives.rst new file mode 100644 index 000000000..a99ccdbf5 --- /dev/null +++ b/doc/usage/restructuredtext/directives.rst @@ -0,0 +1,1074 @@ +.. highlight:: rst + +========== +Directives +========== + +:ref:`As previously discussed <rst-directives>`, a directive is a generic block +of explicit markup. While Docutils provides a number of directives, Sphinx +provides many more and uses directives as one of the primary extension +mechanisms. + +See :doc:`/usage/restructuredtext/domains` for roles added by domains. + +.. seealso:: + + Refer to the :ref:`reStructuredText Primer <rst-directives>` for an overview + of the directives provided by Docutils. + + +.. _toctree-directive: + +Table of contents +----------------- + +.. index:: pair: table of; contents + +Since reST does not have facilities to interconnect several documents, or split +documents into multiple output files, Sphinx uses a custom directive to add +relations between the single files the documentation is made of, as well as +tables of contents. The ``toctree`` directive is the central element. + +.. note:: + + Simple "inclusion" of one file in another can be done with the + :dudir:`include` directive. + +.. note:: + + For local tables of contents, use the standard reST :dudir:`contents + directive <table-of-contents>`. + +.. rst:directive:: toctree + + This directive inserts a "TOC tree" at the current location, using the + individual TOCs (including "sub-TOC trees") of the documents given in the + directive body. Relative document names (not beginning with a slash) are + relative to the document the directive occurs in, absolute names are relative + to the source directory. A numeric ``maxdepth`` option may be given to + indicate the depth of the tree; by default, all levels are included. [#]_ + + Consider this example (taken from the Python docs' library reference index):: + + .. toctree:: + :maxdepth: 2 + + intro + strings + datatypes + numeric + (many more documents listed here) + + This accomplishes two things: + + * Tables of contents from all those documents are inserted, with a maximum + depth of two, that means one nested heading. ``toctree`` directives in + those documents are also taken into account. + * Sphinx knows the relative order of the documents ``intro``, + ``strings`` and so forth, and it knows that they are children of the shown + document, the library index. From this information it generates "next + chapter", "previous chapter" and "parent chapter" links. + + **Entries** + + Document titles in the :rst:dir:`toctree` will be automatically read from the + title of the referenced document. If that isn't what you want, you can + specify an explicit title and target using a similar syntax to reST + hyperlinks (and Sphinx's :ref:`cross-referencing syntax <xref-syntax>`). This + looks like:: + + .. toctree:: + + intro + All about strings <strings> + datatypes + + The second line above will link to the ``strings`` document, but will use the + title "All about strings" instead of the title of the ``strings`` document. + + You can also add external links, by giving an HTTP URL instead of a document + name. + + **Section numbering** + + If you want to have section numbers even in HTML output, give the + **toplevel** toctree a ``numbered`` option. For example:: + + .. toctree:: + :numbered: + + foo + bar + + Numbering then starts at the heading of ``foo``. Sub-toctrees are + automatically numbered (don't give the ``numbered`` flag to those). + + Numbering up to a specific depth is also possible, by giving the depth as a + numeric argument to ``numbered``. + + **Additional options** + + You can use ``caption`` option to provide a toctree caption and you can use + ``name`` option to provide implicit target name that can be referenced by + using :rst:role:`ref`:: + + .. toctree:: + :caption: Table of Contents + :name: mastertoc + + foo + + If you want only the titles of documents in the tree to show up, not other + headings of the same level, you can use the ``titlesonly`` option:: + + .. toctree:: + :titlesonly: + + foo + bar + + You can use "globbing" in toctree directives, by giving the ``glob`` flag + option. All entries are then matched against the list of available + documents, and matches are inserted into the list alphabetically. Example:: + + .. toctree:: + :glob: + + intro* + recipe/* + * + + This includes first all documents whose names start with ``intro``, then all + documents in the ``recipe`` folder, then all remaining documents (except the + one containing the directive, of course.) [#]_ + + The special entry name ``self`` stands for the document containing the + toctree directive. This is useful if you want to generate a "sitemap" from + the toctree. + + You can use the ``reversed`` flag option to reverse the order of the entries + in the list. This can be useful when using the ``glob`` flag option to + reverse the ordering of the files. Example:: + + .. toctree:: + :glob: + :reversed: + + recipe/* + + You can also give a "hidden" option to the directive, like this:: + + .. toctree:: + :hidden: + + doc_1 + doc_2 + + This will still notify Sphinx of the document hierarchy, but not insert links + into the document at the location of the directive -- this makes sense if you + intend to insert these links yourself, in a different style, or in the HTML + sidebar. + + In cases where you want to have only one top-level toctree and hide all other + lower level toctrees you can add the "includehidden" option to the top-level + toctree entry:: + + .. toctree:: + :includehidden: + + doc_1 + doc_2 + + All other toctree entries can then be eliminated by the "hidden" option. + + In the end, all documents in the :term:`source directory` (or subdirectories) + must occur in some ``toctree`` directive; Sphinx will emit a warning if it + finds a file that is not included, because that means that this file will not + be reachable through standard navigation. + + Use :confval:`exclude_patterns` to explicitly exclude documents or + directories from building completely. Use :ref:`the "orphan" metadata + <metadata>` to let a document be built, but notify Sphinx that it is not + reachable via a toctree. + + The "master document" (selected by :confval:`master_doc`) is the "root" of + the TOC tree hierarchy. It can be used as the documentation's main page, or + as a "full table of contents" if you don't give a ``maxdepth`` option. + + .. versionchanged:: 0.3 + Added "globbing" option. + + .. versionchanged:: 0.6 + Added "numbered" and "hidden" options as well as external links and + support for "self" references. + + .. versionchanged:: 1.0 + Added "titlesonly" option. + + .. versionchanged:: 1.1 + Added numeric argument to "numbered". + + .. versionchanged:: 1.2 + Added "includehidden" option. + + .. versionchanged:: 1.3 + Added "caption" and "name" option. + +Special names +^^^^^^^^^^^^^ + +Sphinx reserves some document names for its own use; you should not try to +create documents with these names -- it will cause problems. + +The special document names (and pages generated for them) are: + +* ``genindex``, ``modindex``, ``search`` + + These are used for the general index, the Python module index, and the search + page, respectively. + + The general index is populated with entries from modules, all + index-generating :ref:`object descriptions <basic-domain-markup>`, and from + :rst:dir:`index` directives. + + The Python module index contains one entry per :rst:dir:`py:module` + directive. + + The search page contains a form that uses the generated JSON search index and + JavaScript to full-text search the generated documents for search words; it + should work on every major browser that supports modern JavaScript. + +* every name beginning with ``_`` + + Though only few such names are currently used by Sphinx, you should not + create documents or document-containing directories with such names. (Using + ``_`` as a prefix for a custom template directory is fine.) + +.. warning:: + + Be careful with unusual characters in filenames. Some formats may interpret + these characters in unexpected ways: + + * Do not use the colon ``:`` for HTML based formats. Links to other parts + may not work. + + * Do not use the plus ``+`` for the ePub format. Some resources may not be + found. + + +Paragraph-level markup +---------------------- + +.. index:: note, warning + pair: changes; in version + +These directives create short paragraphs and can be used inside information +units as well as normal text. + +.. rst:directive:: .. note:: + + An especially important bit of information about an API that a user should be + aware of when using whatever bit of API the note pertains to. The content of + the directive should be written in complete sentences and include all + appropriate punctuation. + + Example:: + + .. note:: + + This function is not suitable for sending spam e-mails. + +.. rst:directive:: .. warning:: + + An important bit of information about an API that a user should be very aware + of when using whatever bit of API the warning pertains to. The content of + the directive should be written in complete sentences and include all + appropriate punctuation. This differs from :rst:dir:`note` in that it is + recommended over :rst:dir:`note` for information regarding security. + +.. rst:directive:: .. versionadded:: version + + This directive documents the version of the project which added the described + feature to the library or C API. When this applies to an entire module, it + should be placed at the top of the module section before any prose. + + The first argument must be given and is the version in question; you can add + a second argument consisting of a *brief* explanation of the change. + + Example:: + + .. versionadded:: 2.5 + The *spam* parameter. + + Note that there must be no blank line between the directive head and the + explanation; this is to make these blocks visually continuous in the markup. + +.. rst:directive:: .. versionchanged:: version + + Similar to :rst:dir:`versionadded`, but describes when and what changed in + the named feature in some way (new parameters, changed side effects, etc.). + +.. rst:directive:: .. deprecated:: version + + Similar to :rst:dir:`versionchanged`, but describes when the feature was + deprecated. An explanation can also be given, for example to inform the + reader what should be used instead. Example:: + + .. deprecated:: 3.1 + Use :func:`spam` instead. + +.. rst:directive:: seealso + + Many sections include a list of references to module documentation or + external documents. These lists are created using the :rst:dir:`seealso` + directive. + + The :rst:dir:`seealso` directive is typically placed in a section just before + any subsections. For the HTML output, it is shown boxed off from the main + flow of the text. + + The content of the :rst:dir:`seealso` directive should be a reST definition + list. Example:: + + .. seealso:: + + Module :py:mod:`zipfile` + Documentation of the :py:mod:`zipfile` standard module. + + `GNU tar manual, Basic Tar Format <http://link>`_ + Documentation for tar archive files, including GNU tar extensions. + + There's also a "short form" allowed that looks like this:: + + .. seealso:: modules :py:mod:`zipfile`, :py:mod:`tarfile` + + .. versionadded:: 0.5 + The short form. + +.. rst:directive:: .. rubric:: title + + This directive creates a paragraph heading that is not used to create a + table of contents node. + + .. note:: + + If the *title* of the rubric is "Footnotes" (or the selected language's + equivalent), this rubric is ignored by the LaTeX writer, since it is + assumed to only contain footnote definitions and therefore would create an + empty heading. + +.. rst:directive:: centered + + This directive creates a centered boldfaced line of text. Use it as + follows:: + + .. centered:: LICENSE AGREEMENT + + .. deprecated:: 1.1 + This presentation-only directive is a legacy from older versions. Use a + :rst:dir:`rst-class` directive instead and add an appropriate style. + +.. rst:directive:: hlist + + This directive must contain a bullet list. It will transform it into a more + compact list by either distributing more than one item horizontally, or + reducing spacing between items, depending on the builder. + + For builders that support the horizontal distribution, there is a ``columns`` + option that specifies the number of columns; it defaults to 2. Example:: + + .. hlist:: + :columns: 3 + + * A list of + * short items + * that should be + * displayed + * horizontally + + .. versionadded:: 0.6 + + +.. _code-examples: + +Showing code examples +--------------------- + +.. index:: pair: code; examples + single: sourcecode + +There are multiple ways to show syntax-highlighted literal code blocks in +Sphinx: using :ref:`reST doctest blocks <rst-doctest-blocks>`; using :ref:`reST +literal blocks <rst-literal-blocks>`, optionally in combination with the +:rst:dir:`highlight` directive; using the :rst:dir:`code-block` directive; and +using the :rst:dir:`literalinclude` directive. Doctest blocks can only be used +to show interactive Python sessions, while the remaining three can be used for +other languages. Of these three, literal blocks are useful when an entire +document, or at least large sections of it, use code blocks with the same +syntax and which should be styled in the same manner. On the other hand, the +:rst:dir:`code-block` directive makes more sense when you want more fine-tuned +control over the styling of each block or when you have a document containing +code blocks using multiple varied syntaxes. Finally, the +:rst:dir:`literalinclude` directive is useful for including entire code files +in your documentation. + +In all cases, Syntax highlighting is provided by `Pygments +<http://pygments.org>`_. When using literal blocks, this is configured using +any :rst:dir:`highlight` directives in the source file. When a ``highlight`` +directive is encountered, it is used until the next ``highlight`` directive is +encountered. If there is no ``highlight`` directive in the file, the global +highlighting language is used. This defaults to ``python`` but can be +configured using the :confval:`highlight_language` config value. The following +values are supported: + +* ``none`` (no highlighting) +* ``default`` (similar to ``python3`` but with a fallback to ``none`` without + warning highlighting fails; the default when :confval:`highlight_language` + isn't set) +* ``guess`` (let Pygments guess the lexer based on contents, only works with + certain well-recognizable languages) +* ``python`` +* ``rest`` +* ``c`` +* ... and any other `lexer alias that Pygments supports`__ + +If highlighting with the selected language fails (i.e. Pygments emits an +"Error" token), the block is not highlighted in any way. + +.. important:: + + The list of lexer aliases supported is tied to the Pygment version. If you + want to ensure consistent highlighting, you should fix your version of + Pygments. + +__ http://pygments.org/docs/lexers/ + +.. rst:directive:: .. highlight:: language + + Example:: + + .. highlight:: c + + This language is used until the next ``highlight`` directive is encountered. + As discussed previously, *language* can be any lexer alias supported by + Pygments. + + **Additional options** + + Pygments can generate line numbers for code blocks. To enable this, use the + ``linenothreshold`` option. :: + + .. highlight:: python + :linenothreshold: 5 + + This will produce line numbers for all code blocks longer than five lines. + +.. rst:directive:: .. code-block:: language + + Example:: + + .. code-block:: ruby + + Some Ruby code. + + The directive's alias name :rst:dir:`sourcecode` works as well. As with + :rst:dir:`highlight`\ 's ``language`` option, ``language`` can be any lexer + alias supported by Pygments. + + **Additional options** + + Pygments can generate line numbers for code blocks. To enable this for, use + the ``linenos`` flag option. :: + + .. code-block:: ruby + :linenos: + + Some more Ruby code. + + The first line number can be selected with the ``lineno-start`` option. If + present, ``linenos`` flag is automatically activated:: + + .. code-block:: ruby + :lineno-start: 10 + + Some more Ruby code, with line numbering starting at 10. + + Additionally, an ``emphasize-lines`` option can be given to have Pygments + emphasize particular lines:: + + .. code-block:: python + :emphasize-lines: 3,5 + + def some_function(): + interesting = False + print 'This line is highlighted.' + print 'This one is not...' + print '...but this one is.' + + A ``caption`` option can be given to show that name before the code block. + A ``name`` option can be provided implicit target name that can be + referenced by using :rst:role:`ref`. For example:: + + .. code-block:: python + :caption: this.py + :name: this-py + + print 'Explicit is better than implicit.' + + A ``dedent`` option can be given to strip indentation characters from the + code block. For example:: + + .. code-block:: ruby + :dedent: 4 + + some ruby code + + .. versionchanged:: 1.1 + The ``emphasize-lines`` option has been added. + + .. versionchanged:: 1.3 + The ``lineno-start``, ``caption``, ``name`` and ``dedent`` options have + been added. + + .. versionchanged:: 1.6.6 + LaTeX supports the ``emphasize-lines`` option. + +.. rst:directive:: .. literalinclude:: filename + + Longer displays of verbatim text may be included by storing the example text + in an external file containing only plain text. The file may be included + using the ``literalinclude`` directive. [#]_ For example, to include the + Python source file :file:`example.py`, use:: + + .. literalinclude:: example.py + + The file name is usually relative to the current file's path. However, if + it is absolute (starting with ``/``), it is relative to the top source + directory. + + **Additional options** + + Like :rst:dir:`code-block`, the directive supports the ``linenos`` flag + option to switch on line numbers, the ``lineno-start`` option to select the + first line number, the ``emphasize-lines`` option to emphasize particular + lines, the ``name`` option to provide an implicit target name, the + ``dedent`` option to strip indentation characters for the code block, and a + ``language`` option to select a language different from the current file's + standard language. In addition, it supports the ``caption`` option; however, + this can be provided with no argument to use the filename as the caption. + Example with options:: + + .. literalinclude:: example.rb + :language: ruby + :emphasize-lines: 12,15-18 + :linenos: + + Tabs in the input are expanded if you give a ``tab-width`` option with the + desired tab width. + + Include files are assumed to be encoded in the :confval:`source_encoding`. + If the file has a different encoding, you can specify it with the + ``encoding`` option:: + + .. literalinclude:: example.py + :encoding: latin-1 + + The directive also supports including only parts of the file. If it is a + Python module, you can select a class, function or method to include using + the ``pyobject`` option:: + + .. literalinclude:: example.py + :pyobject: Timer.start + + This would only include the code lines belonging to the ``start()`` method + in the ``Timer`` class within the file. + + Alternately, you can specify exactly which lines to include by giving a + ``lines`` option:: + + .. literalinclude:: example.py + :lines: 1,3,5-10,20- + + This includes the lines 1, 3, 5 to 10 and lines 20 to the last line. + + Another way to control which part of the file is included is to use the + ``start-after`` and ``end-before`` options (or only one of them). If + ``start-after`` is given as a string option, only lines that follow the + first line containing that string are included. If ``end-before`` is given + as a string option, only lines that precede the first lines containing that + string are included. The ``start-at`` and ``end-at`` options behave in a + similar way, but the lines containing the matched string are included. + + With lines selected using ``start-after`` it is still possible to use + ``lines``, the first allowed line having by convention the line number + ``1``. + + When lines have been selected in any of the ways described above, the line + numbers in ``emphasize-lines`` refer to those selected lines, counted + consecutively starting at ``1``. + + When specifying particular parts of a file to display, it can be useful to + display the original line numbers. This can be done using the + ``lineno-match`` option, which is however allowed only when the selection + consists of contiguous lines. + + You can prepend and/or append a line to the included code, using the + ``prepend`` and ``append`` option, respectively. This is useful e.g. for + highlighting PHP code that doesn't include the ``<?php``/``?>`` markers. + + If you want to show the diff of the code, you can specify the old file by + giving a ``diff`` option:: + + .. literalinclude:: example.py + :diff: example.py.orig + + This shows the diff between ``example.py`` and ``example.py.orig`` with + unified diff format. + + .. versionchanged:: 0.4.3 + Added the ``encoding`` option. + + .. versionchanged:: 0.6 + Added the ``pyobject``, ``lines``, ``start-after`` and ``end-before`` + options, as well as support for absolute filenames. + + .. versionchanged:: 1.0 + Added the ``prepend``, ``append``, and ``tab-width`` options. + + .. versionchanged:: 1.3 + Added the ``diff``, ``lineno-match``, ``caption``, ``name``, and + ``dedent`` options. + + .. versionchanged:: 1.5 + Added the ``start-at``, and ``end-at`` options. + + .. versionchanged:: 1.6 + With both ``start-after`` and ``lines`` in use, the first line as per + ``start-after`` is considered to be with line number ``1`` for ``lines``. + +.. _glossary-directive: + +Glossary +-------- + +.. rst:directive:: .. glossary:: + + This directive must contain a reST definition-list-like markup with terms and + definitions. The definitions will then be referencable with the + :rst:role:`term` role. Example:: + + .. glossary:: + + environment + A structure where information about all documents under the root is + saved, and used for cross-referencing. The environment is pickled + after the parsing stage, so that successive runs only need to read + and parse new and changed documents. + + source directory + The directory which, including its subdirectories, contains all + source files for one Sphinx project. + + In contrast to regular definition lists, *multiple* terms per entry are + allowed, and inline markup is allowed in terms. You can link to all of the + terms. For example:: + + .. glossary:: + + term 1 + term 2 + Definition of both terms. + + (When the glossary is sorted, the first term determines the sort order.) + + If you want to specify "grouping key" for general index entries, you can put a "key" + as "term : key". For example:: + + .. glossary:: + + term 1 : A + term 2 : B + Definition of both terms. + + Note that "key" is used for grouping key as is. + The "key" isn't normalized; key "A" and "a" become different groups. + The whole characters in "key" is used instead of a first character; it is used for + "Combining Character Sequence" and "Surrogate Pairs" grouping key. + + In i18n situation, you can specify "localized term : key" even if original text only + have "term" part. In this case, translated "localized term" will be categorized in + "key" group. + + .. versionadded:: 0.6 + You can now give the glossary directive a ``:sorted:`` flag that will + automatically sort the entries alphabetically. + + .. versionchanged:: 1.1 + Now supports multiple terms and inline markup in terms. + + .. versionchanged:: 1.4 + Index key for glossary term should be considered *experimental*. + + +Meta-information markup +----------------------- + +.. rst:directive:: .. sectionauthor:: name <email> + + Identifies the author of the current section. The argument should include + the author's name such that it can be used for presentation and email + address. The domain name portion of the address should be lower case. + Example:: + + .. sectionauthor:: Guido van Rossum <guido@python.org> + + By default, this markup isn't reflected in the output in any way (it helps + keep track of contributions), but you can set the configuration value + :confval:`show_authors` to ``True`` to make them produce a paragraph in the + output. + + +.. rst:directive:: .. codeauthor:: name <email> + + The :rst:dir:`codeauthor` directive, which can appear multiple times, names + the authors of the described code, just like :rst:dir:`sectionauthor` names + the author(s) of a piece of documentation. It too only produces output if + the :confval:`show_authors` configuration value is ``True``. + + +Index-generating markup +----------------------- + +Sphinx automatically creates index entries from all object descriptions (like +functions, classes or attributes) like discussed in +:doc:`/usage/restructuredtext/domains`. + +However, there is also explicit markup available, to make the index more +comprehensive and enable index entries in documents where information is not +mainly contained in information units, such as the language reference. + +.. rst:directive:: .. index:: <entries> + + This directive contains one or more index entries. Each entry consists of a + type and a value, separated by a colon. + + For example:: + + .. index:: + single: execution; context + module: __main__ + module: sys + triple: module; search; path + + The execution context + --------------------- + + ... + + This directive contains five entries, which will be converted to entries in + the generated index which link to the exact location of the index statement + (or, in case of offline media, the corresponding page number). + + Since index directives generate cross-reference targets at their location in + the source, it makes sense to put them *before* the thing they refer to -- + e.g. a heading, as in the example above. + + The possible entry types are: + + single + Creates a single index entry. Can be made a subentry by separating the + subentry text with a semicolon (this notation is also used below to + describe what entries are created). + pair + ``pair: loop; statement`` is a shortcut that creates two index entries, + namely ``loop; statement`` and ``statement; loop``. + triple + Likewise, ``triple: module; search; path`` is a shortcut that creates + three index entries, which are ``module; search path``, ``search; path, + module`` and ``path; module search``. + see + ``see: entry; other`` creates an index entry that refers from ``entry`` to + ``other``. + seealso + Like ``see``, but inserts "see also" instead of "see". + module, keyword, operator, object, exception, statement, builtin + These all create two index entries. For example, ``module: hashlib`` + creates the entries ``module; hashlib`` and ``hashlib; module``. (These + are Python-specific and therefore deprecated.) + + You can mark up "main" index entries by prefixing them with an exclamation + mark. The references to "main" entries are emphasized in the generated + index. For example, if two pages contain :: + + .. index:: Python + + and one page contains :: + + .. index:: ! Python + + then the backlink to the latter page is emphasized among the three backlinks. + + For index directives containing only "single" entries, there is a shorthand + notation:: + + .. index:: BNF, grammar, syntax, notation + + This creates four index entries. + + .. versionchanged:: 1.1 + Added ``see`` and ``seealso`` types, as well as marking main entries. + +.. rst:role:: index + + While the :rst:dir:`index` directive is a block-level markup and links to the + beginning of the next paragraph, there is also a corresponding role that sets + the link target directly where it is used. + + The content of the role can be a simple phrase, which is then kept in the + text and used as an index entry. It can also be a combination of text and + index entry, styled like with explicit targets of cross-references. In that + case, the "target" part can be a full entry as described for the directive + above. For example:: + + This is a normal reST :index:`paragraph` that contains several + :index:`index entries <pair: index; entry>`. + + .. versionadded:: 1.1 + + +.. _tags: + +Including content based on tags +------------------------------- + +.. rst:directive:: .. only:: <expression> + + Include the content of the directive only if the *expression* is true. The + expression should consist of tags, like this:: + + .. only:: html and draft + + Undefined tags are false, defined tags (via the ``-t`` command-line option or + within :file:`conf.py`, see :ref:`here <conf-tags>`) are true. Boolean + expressions, also using parentheses (like ``html and (latex or draft)``) are + supported. + + The *format* and the *name* of the current builder (``html``, ``latex`` or + ``text``) are always set as a tag [#]_. To make the distinction between + format and name explicit, they are also added with the prefix ``format_`` and + ``builder_``, e.g. the epub builder defines the tags ``html``, ``epub``, + ``format_html`` and ``builder_epub``. + + These standard tags are set *after* the configuration file is read, so they + are not available there. + + All tags must follow the standard Python identifier syntax as set out in + the `Identifiers and keywords + <https://docs.python.org/2/reference/lexical_analysis.html#identifiers>`_ + documentation. That is, a tag expression may only consist of tags that + conform to the syntax of Python variables. In ASCII, this consists of the + uppercase and lowercase letters ``A`` through ``Z``, the underscore ``_`` + and, except for the first character, the digits ``0`` through ``9``. + + .. versionadded:: 0.6 + .. versionchanged:: 1.2 + Added the name of the builder and the prefixes. + + .. warning:: + + This directive is designed to control only content of document. It could + not control sections, labels and so on. + +.. _table-directives: + +Tables +------ + +Use :ref:`reStructuredText tables <rst-tables>`, i.e. either + +- grid table syntax (:duref:`ref <grid-tables>`), +- simple table syntax (:duref:`ref <simple-tables>`), +- :dudir:`csv-table` syntax, +- or :dudir:`list-table` syntax. + +The :dudir:`table` directive serves as optional wrapper of the *grid* and +*simple* syntaxes. + +They work fine in HTML output, however there are some gotchas when using tables +in LaTeX: the column width is hard to determine correctly automatically. For +this reason, the following directive exists: + +.. rst:directive:: .. tabularcolumns:: column spec + + This directive gives a "column spec" for the next table occurring in the + source file. The spec is the second argument to the LaTeX ``tabulary`` + package's environment (which Sphinx uses to translate tables). It can have + values like :: + + |l|l|l| + + which means three left-adjusted, nonbreaking columns. For columns with + longer text that should automatically be broken, use either the standard + ``p{width}`` construct, or tabulary's automatic specifiers: + + +-----+------------------------------------------+ + |``L``| flush left column with automatic width | + +-----+------------------------------------------+ + |``R``| flush right column with automatic width | + +-----+------------------------------------------+ + |``C``| centered column with automatic width | + +-----+------------------------------------------+ + |``J``| justified column with automatic width | + +-----+------------------------------------------+ + + The automatic widths of the ``LRCJ`` columns are attributed by ``tabulary`` + in proportion to the observed shares in a first pass where the table cells + are rendered at their natural "horizontal" widths. + + By default, Sphinx uses a table layout with ``J`` for every column. + + .. versionadded:: 0.3 + + .. versionchanged:: 1.6 + Merged cells may now contain multiple paragraphs and are much better + handled, thanks to custom Sphinx LaTeX macros. This novel situation + motivated the switch to ``J`` specifier and not ``L`` by default. + + .. hint:: + + Sphinx actually uses ``T`` specifier having done ``\newcolumntype{T}{J}``. + To revert to previous default, insert ``\newcolumntype{T}{L}`` in the + LaTeX preamble (see :confval:`latex_elements`). + + A frequent issue with tabulary is that columns with little contents are + "squeezed". The minimal column width is a tabulary parameter called + ``\tymin``. You may set it globally in the LaTeX preamble via + ``\setlength{\tymin}{40pt}`` for example. + + Else, use the :rst:dir:`tabularcolumns` directive with an explicit + ``p{40pt}`` (for example) for that column. You may use also ``l`` + specifier but this makes the task of setting column widths more difficult + if some merged cell intersects that column. + + .. warning:: + + Tables with more than 30 rows are rendered using ``longtable``, not + ``tabulary``, in order to allow pagebreaks. The ``L``, ``R``, ... specifiers + do not work for these tables. + + Tables that contain list-like elements such as object descriptions, + blockquotes or any kind of lists cannot be set out of the box with + ``tabulary``. They are therefore set with the standard LaTeX ``tabular`` (or + ``longtable``) environment if you don't give a ``tabularcolumns`` directive. + If you do, the table will be set with ``tabulary`` but you must use the + ``p{width}`` construct (or Sphinx's ``\X`` and ``\Y`` specifiers described + below) for the columns containing these elements. + + Literal blocks do not work with ``tabulary`` at all, so tables containing + a literal block are always set with ``tabular``. The verbatim environment + used for literal blocks only works in ``p{width}`` (and ``\X`` or ``\Y``) + columns, hence Sphinx generates such column specs for tables containing + literal blocks. + + Since Sphinx 1.5, the ``\X{a}{b}`` specifier is used (there *is* a backslash + in the specifier letter). It is like ``p{width}`` with the width set to a + fraction ``a/b`` of the current line width. You can use it in the + :rst:dir:`tabularcolumns` (it is not a problem if some LaTeX macro is also + called ``\X``.) + + It is *not* needed for ``b`` to be the total number of columns, nor for the + sum of the fractions of the ``\X`` specifiers to add up to one. For example + ``|\X{2}{5}|\X{1}{5}|\X{1}{5}|`` is legitimate and the table will occupy + 80% of the line width, the first of its three columns having the same width + as the sum of the next two. + + This is used by the ``:widths:`` option of the :dudir:`table` directive. + + Since Sphinx 1.6, there is also the ``\Y{f}`` specifier which admits a + decimal argument, such has ``\Y{0.15}``: this would have the same effect as + ``\X{3}{20}``. + + .. versionchanged:: 1.6 + + Merged cells from complex grid tables (either multi-row, multi-column, or + both) now allow blockquotes, lists, literal blocks, ... as do regular cells. + + Sphinx's merged cells interact well with ``p{width}``, ``\X{a}{b}``, ``Y{f}`` + and tabulary's columns. + + .. note:: + + :rst:dir:`tabularcolumns` conflicts with ``:widths:`` option of table + directives. If both are specified, ``:widths:`` option will be ignored. + + +Math +---- + +.. todo:: Move this in here. + +See :ref:`math-support`. + + +Grammar production displays +--------------------------- + +Special markup is available for displaying the productions of a formal grammar. +The markup is simple and does not attempt to model all aspects of BNF (or any +derived forms), but provides enough to allow context-free grammars to be +displayed in a way that causes uses of a symbol to be rendered as hyperlinks to +the definition of the symbol. There is this directive: + +.. rst:directive:: .. productionlist:: [name] + + This directive is used to enclose a group of productions. Each production + is given on a single line and consists of a name, separated by a colon from + the following definition. If the definition spans multiple lines, each + continuation line must begin with a colon placed at the same column as in + the first line. + + The argument to :rst:dir:`productionlist` serves to distinguish different + sets of production lists that belong to different grammars. + + Blank lines are not allowed within ``productionlist`` directive arguments. + + The definition can contain token names which are marked as interpreted text + (e.g. ``sum ::= `integer` "+" `integer```) -- this generates + cross-references to the productions of these tokens. Outside of the + production list, you can reference to token productions using + :rst:role:`token`. + + Note that no further reST parsing is done in the production, so that you + don't have to escape ``*`` or ``|`` characters. + +The following is an example taken from the Python Reference Manual:: + + .. productionlist:: + try_stmt: try1_stmt | try2_stmt + try1_stmt: "try" ":" `suite` + : ("except" [`expression` ["," `target`]] ":" `suite`)+ + : ["else" ":" `suite`] + : ["finally" ":" `suite`] + try2_stmt: "try" ":" `suite` + : "finally" ":" `suite` + + +.. rubric:: Footnotes + +.. [#] The LaTeX writer only refers the ``maxdepth`` option of first toctree + directive in the document. + +.. [#] A note on available globbing syntax: you can use the standard shell + constructs ``*``, ``?``, ``[...]`` and ``[!...]`` with the feature that + these all don't match slashes. A double star ``**`` can be used to + match any sequence of characters *including* slashes. + +.. [#] There is a standard ``.. include`` directive, but it raises errors if the + file is not found. This one only emits a warning. + +.. [#] For most builders name and format are the same. At the moment only + builders derived from the html builder distinguish between the builder + format and the builder name. + + Note that the current builder tag is not available in ``conf.py``, it is + only available after the builder is initialized. diff --git a/doc/domains.rst b/doc/usage/restructuredtext/domains.rst index 5bed02cf4..452cf63ae 100644 --- a/doc/domains.rst +++ b/doc/usage/restructuredtext/domains.rst @@ -1,15 +1,11 @@ .. highlight:: rst -.. _domains: - -Sphinx Domains -============== +======= +Domains +======= .. versionadded:: 1.0 -What is a Domain? ------------------ - Originally, Sphinx was conceived for a single project, the documentation of the Python language. Shortly afterwards, it was made available for everyone as a documentation tool, but the documentation of Python modules remained deeply @@ -19,20 +15,20 @@ developed in using it for many different purposes: C/C++ projects, JavaScript, or even reStructuredText markup (like in this documentation). While this was always possible, it is now much easier to easily support -documentation of projects using different programming languages or even ones not -supported by the main Sphinx distribution, by providing a **domain** for every -such purpose. +documentation of projects using different programming languages or even ones +not supported by the main Sphinx distribution, by providing a **domain** for +every such purpose. A domain is a collection of markup (reStructuredText :term:`directive`\ s and :term:`role`\ s) to describe and link to :term:`object`\ s belonging together, e.g. elements of a programming language. Directive and role names in a domain -have names like ``domain:name``, e.g. ``py:function``. Domains can also provide -custom indices (like the Python Module Index). +have names like ``domain:name``, e.g. ``py:function``. Domains can also +provide custom indices (like the Python Module Index). Having domains means that there are no naming problems when one set of -documentation wants to refer to e.g. C++ and Python classes. It also means that -extensions that support the documentation of whole new languages are much easier -to write. +documentation wants to refer to e.g. C++ and Python classes. It also means +that extensions that support the documentation of whole new languages are much +easier to write. This section describes what the domains that are included with Sphinx provide. The domain API is documented as well, in the section :ref:`domain-api`. @@ -65,9 +61,9 @@ that are continued in the next line. Example:: (This example also shows how to use the ``:noindex:`` flag.) -The domains also provide roles that link back to these object descriptions. For -example, to link to one of the functions described in the example above, you -could say :: +The domains also provide roles that link back to these object descriptions. +For example, to link to one of the functions described in the example above, +you could say :: The function :py:func:`spam` does a similar thing. @@ -86,9 +82,9 @@ value :confval:`primary_domain` or via this directive: Select a new default domain. While the :confval:`primary_domain` selects a global default, this only has an effect within the same file. -If no other default is selected, the Python domain (named ``py``) is the default -one, mostly for compatibility with documentation written for older versions of -Sphinx. +If no other default is selected, the Python domain (named ``py``) is the +default one, mostly for compatibility with documentation written for older +versions of Sphinx. Directives and roles that belong to the default domain can be mentioned without giving the domain name, i.e. :: @@ -99,7 +95,6 @@ giving the domain name, i.e. :: Reference to :func:`pyfunc`. - Cross-referencing syntax ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -145,7 +140,6 @@ declarations: The ``deprecated`` option can be given (with no value) to mark a module as deprecated; it will be designated as such in various locations then. - .. rst:directive:: .. py:currentmodule:: name This directive tells Sphinx that the classes, functions etc. documented from @@ -156,7 +150,6 @@ declarations: location has the :rst:dir:`py:module` directive, the others only :rst:dir:`py:currentmodule`. - The following directives are provided for module and class contents: .. rst:directive:: .. py:function:: name(parameters) @@ -275,7 +268,6 @@ The following directives are provided for module and class contents: Refer to a decorator method using the :rst:role:`py:meth` role. - .. _signatures: Python Signatures @@ -290,15 +282,14 @@ can also be given as well as return type annotations:: .. py:function:: compile(source : string, filename, symbol='file') -> ast object -For functions with optional parameters that don't have default values (typically -functions implemented in C extension modules without keyword argument support), -you can use brackets to specify the optional parts: +For functions with optional parameters that don't have default values +(typically functions implemented in C extension modules without keyword +argument support), you can use brackets to specify the optional parts: .. py:function:: compile(source[, filename[, symbol]]) It is customary to put the opening bracket before the comma. - .. _info-field-lists: Info field lists @@ -321,8 +312,8 @@ are recognized and formatted nicely: .. note:: - In current release, all ``var``, ``ivar`` and ``cvar`` are represented as "Variable". - There is no difference at all. + In current release, all ``var``, ``ivar`` and ``cvar`` are represented as + "Variable". There is no difference at all. The field names must consist of one of these keywords and an argument (except for ``returns`` and ``rtype``, which do not need an argument). This is best @@ -364,7 +355,6 @@ single word, like this:: :param int priority: The priority of the message, can be a number 1-5 - .. versionadded:: 1.5 Container types such as lists and dictionaries can be linked automatically @@ -377,8 +367,8 @@ using the following syntax:: :type point: tuple(float, float) :type point: tuple[float, float] -Multiple types in a type field will be linked automatically if separated by -the word "or":: +Multiple types in a type field will be linked automatically if separated by the +word "or":: :type an_arg: int or None :vartype a_var: str or int @@ -419,9 +409,9 @@ a matching identifier is found: .. rst:role:: py:meth - Reference a method of an object. The role text can include the type name and - the method name; if it occurs within the description of a type, the type name - can be omitted. A dotted name may be used. + Reference a method of an object. The role text can include the type name + and the method name; if it occurs within the description of a type, the type + name can be omitted. A dotted name may be used. .. rst:role:: py:attr @@ -439,24 +429,24 @@ a matching identifier is found: .. versionadded:: 0.4 The name enclosed in this markup can include a module name and/or a class name. -For example, ``:py:func:`filter``` could refer to a function named ``filter`` in -the current module, or the built-in function of that name. In contrast, +For example, ``:py:func:`filter``` could refer to a function named ``filter`` +in the current module, or the built-in function of that name. In contrast, ``:py:func:`foo.filter``` clearly refers to the ``filter`` function in the ``foo`` module. Normally, names in these roles are searched first without any further qualification, then with the current module name prepended, then with the -current module and class name (if any) prepended. If you prefix the name with a -dot, this order is reversed. For example, in the documentation of Python's +current module and class name (if any) prepended. If you prefix the name with +a dot, this order is reversed. For example, in the documentation of Python's :mod:`codecs` module, ``:py:func:`open``` always refers to the built-in function, while ``:py:func:`.open``` refers to :func:`codecs.open`. -A similar heuristic is used to determine whether the name is an attribute of the -currently documented class. +A similar heuristic is used to determine whether the name is an attribute of +the currently documented class. Also, if the name is prefixed with a dot, and no exact match is found, the -target is taken as a suffix and all object names with that suffix are -searched. For example, ``:py:meth:`.TarFile.close``` references the +target is taken as a suffix and all object names with that suffix are searched. +For example, ``:py:meth:`.TarFile.close``` references the ``tarfile.TarFile.close()`` function, even if the current module is not ``tarfile``. Since this can get ambiguous, if there is more than one possible match, you will get a warning from Sphinx. @@ -515,14 +505,13 @@ The C domain (name **c**) is suited for documentation of C API. .. c:var:: PyObject* PyClass_Type - .. _c-roles: Cross-referencing C constructs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The following roles create cross-references to C-language constructs if they are -defined in the documentation: +The following roles create cross-references to C-language constructs if they +are defined in the documentation: .. rst:role:: c:func @@ -544,6 +533,7 @@ defined in the documentation: Reference a C-language variable. + .. _cpp-domain: The C++ Domain @@ -551,12 +541,11 @@ The C++ Domain The C++ domain (name **cpp**) supports documenting C++ projects. - Directives ~~~~~~~~~~ -The following directives are available. All declarations can start with -a visibility statement (``public``, ``private`` or ``protected``). +The following directives are available. All declarations can start with a +visibility statement (``public``, ``private`` or ``protected``). .. rst:directive:: .. cpp:class:: class specifier @@ -585,7 +574,6 @@ a visibility statement (``public``, ``private`` or ``protected``). .. cpp:class:: template<typename T> \ std::array<T, 42> - .. rst:directive:: .. cpp:function:: (member) function prototype Describe a function or member function, e.g.,:: @@ -624,7 +612,6 @@ a visibility statement (``public``, ``private`` or ``protected``). .. cpp:function:: template<> \ void print(int i) - .. rst:directive:: .. cpp:member:: (member) variable declaration .. cpp:var:: (member) variable declaration @@ -641,13 +628,12 @@ a visibility statement (``public``, ``private`` or ``protected``). .. cpp:member:: template<class T> \ constexpr T pi = T(3.1415926535897932385) - .. rst:directive:: .. cpp:type:: typedef declaration .. cpp:type:: name .. cpp:type:: type alias declaration - Describe a type as in a typedef declaration, a type alias declaration, - or simply the name of a type with unspecified type, e.g.,:: + Describe a type as in a typedef declaration, a type alias declaration, or + simply the name of a type with unspecified type, e.g.,:: .. cpp:type:: std::vector<int> MyList @@ -683,15 +669,13 @@ a visibility statement (``public``, ``private`` or ``protected``). .. cpp:type:: template<typename T> \ MyContainer = std::vector<T> - .. rst:directive:: .. cpp:enum:: unscoped enum declaration .. cpp:enum-struct:: scoped enum declaration .. cpp:enum-class:: scoped enum declaration - Describe a (scoped) enum, possibly with the underlying type specified. - Any enumerators declared inside an unscoped enum will be declared both in the enum scope - and in the parent scope. - Examples:: + Describe a (scoped) enum, possibly with the underlying type specified. Any + enumerators declared inside an unscoped enum will be declared both in the + enum scope and in the parent scope. Examples:: .. cpp:enum:: MyEnum @@ -718,15 +702,14 @@ a visibility statement (``public``, ``private`` or ``protected``). .. cpp:enumerator:: MyEnum::myOtherEnumerator = 42 - .. rst:directive:: .. cpp:concept:: template-parameter-list name .. warning:: The support for concepts is experimental. It is based on the current draft standard and the Concepts Technical Specification. The features may change as they evolve. - Describe a concept. It must have exactly 1 template parameter list. The name may be a - nested name. Example:: + Describe a concept. It must have exactly 1 template parameter list. The name + may be a nested name. Example:: .. cpp:concept:: template<typename It> std::Iterator @@ -760,10 +743,11 @@ a visibility statement (``public``, ``private`` or ``protected``). **Valid Expressions** - :cpp:expr:`*r`, when :cpp:expr:`r` is dereferenceable. - - :cpp:expr:`++r`, with return type :cpp:expr:`It&`, when :cpp:expr:`r` is incrementable. + - :cpp:expr:`++r`, with return type :cpp:expr:`It&`, when :cpp:expr:`r` + is incrementable. Options -....... +^^^^^^^ Some directives support options: @@ -771,7 +755,6 @@ Some directives support options: - ``:tparam-line-spec:``, for templated declarations. If specified, each template parameter will be rendered on a separate line. - Constrained Templates ~~~~~~~~~~~~~~~~~~~~~ @@ -782,10 +765,11 @@ Constrained Templates .. note:: Sphinx does not currently support ``requires`` clauses. Placeholders -............ +^^^^^^^^^^^^ Declarations may use the name of a concept to introduce constrained template -parameters, or the keyword ``auto`` to introduce unconstrained template parameters:: +parameters, or the keyword ``auto`` to introduce unconstrained template +parameters:: .. cpp:function:: void f(auto &&arg) @@ -797,18 +781,20 @@ parameters, or the keyword ``auto`` to introduce unconstrained template paramete Iterator concept. Template Introductions -...................... +^^^^^^^^^^^^^^^^^^^^^^ -Simple constrained function or class templates can be declared with a -`template introduction` instead of a template parameter list:: +Simple constrained function or class templates can be declared with a `template +introduction` instead of a template parameter list:: .. cpp:function:: std::Iterator{It} void advance(It &it) - A function template with a template parameter constrained to be an Iterator. + A function template with a template parameter constrained to be an + Iterator. .. cpp:class:: std::LessThanComparable{T} MySortedContainer - A class template with a template parameter constrained to be LessThanComparable. + A class template with a template parameter constrained to be + LessThanComparable. They are rendered as follows. @@ -818,20 +804,19 @@ They are rendered as follows. .. cpp:class:: std::LessThanComparable{T} MySortedContainer - A class template with a template parameter constrained to be LessThanComparable. + A class template with a template parameter constrained to be + LessThanComparable. Note however that no checking is performed with respect to parameter compatibility. E.g., ``Iterator{A, B, C}`` will be accepted as an introduction even though it would not be valid C++. - Inline Expressions and Tpes ~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. rst:role:: cpp:expr - A role for inserting a C++ expression or type as inline text. - For example:: + A role for inserting a C++ expression or type as inline text. For example:: .. cpp:var:: int a = 42 @@ -846,34 +831,37 @@ Inline Expressions and Tpes .. cpp:function:: int f(int i) - An expression: :cpp:expr:`a * f(a)`. - A type: :cpp:expr:`const MySortedContainer<int>&`. + An expression: :cpp:expr:`a * f(a)`. A type: :cpp:expr:`const + MySortedContainer<int>&`. Namespacing -~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~ + +Declarations in the C++ domain are as default placed in global scope. The +current scope can be changed using three namespace directives. They manage a +stack declarations where ``cpp:namespace`` resets the stack and changes a given +scope. -Declarations in the C++ domain are as default placed in global scope. -The current scope can be changed using three namespace directives. -They manage a stack declarations where ``cpp:namespace`` resets the stack and -changes a given scope. The ``cpp:namespace-push`` directive changes the scope to a given inner scope of the current one. -The ``cpp:namespace-pop`` directive undos the most recent ``cpp:namespace-push`` -directive. + +The ``cpp:namespace-pop`` directive undoes the most recent +``cpp:namespace-push`` directive. .. rst:directive:: .. cpp:namespace:: scope specification - Changes the current scope for the subsequent objects to the given scope, - and resets the namespace directive stack. - Note that the namespace does not need to correspond to C++ namespaces, - but can end in names of classes, e.g.,:: + Changes the current scope for the subsequent objects to the given scope, and + resets the namespace directive stack. Note that the namespace does not need + to correspond to C++ namespaces, but can end in names of classes, e.g.,:: .. cpp:namespace:: Namespace1::Namespace2::SomeClass::AnInnerClass - All subsequent objects will be defined as if their name were declared with the scope - prepended. The subsequent cross-references will be searched for starting in the current scope. + All subsequent objects will be defined as if their name were declared with + the scope prepended. The subsequent cross-references will be searched for + starting in the current scope. - Using ``NULL``, ``0``, or ``nullptr`` as the scope will change to global scope. + Using ``NULL``, ``0``, or ``nullptr`` as the scope will change to global + scope. A namespace declaration can also be templated, e.g.,:: @@ -884,8 +872,8 @@ directive. .. cpp:function:: std::size_t size() const - declares ``size`` as a member function of the class template ``std::vector``. - Equivalently this could have been declared using:: + declares ``size`` as a member function of the class template + ``std::vector``. Equivalently this could have been declared using:: .. cpp:class:: template<typename T> \ std::vector @@ -897,7 +885,6 @@ directive. .. cpp:class:: template<typename T> \ std::vector - .. rst:directive:: .. cpp:namespace-push:: scope specification Change the scope relatively to the current scope. For example, after:: @@ -921,26 +908,25 @@ directive. the current scope will be ``A::B`` (*not* ``A::B::C``). - If no previous ``cpp:namespace-push`` directive has been used, but only a ``cpp:namespace`` - directive, then the current scope will be reset to global scope. - That is, ``.. cpp:namespace:: A::B`` is equivalent to:: + If no previous ``cpp:namespace-push`` directive has been used, but only a + ``cpp:namespace`` directive, then the current scope will be reset to global + scope. That is, ``.. cpp:namespace:: A::B`` is equivalent to:: .. cpp:namespace:: nullptr .. cpp:namespace-push:: A::B - Info field lists ~~~~~~~~~~~~~~~~~ -The C++ directives support the following info fields (see also :ref:`info-field-lists`): +The C++ directives support the following info fields (see also +:ref:`info-field-lists`): * `param`, `parameter`, `arg`, `argument`: Description of a parameter. * `tparam`: Description of a template parameter. * `returns`, `return`: Description of a return value. * `throws`, `throw`, `exception`: Description of a possibly thrown exception. - .. _cpp-roles: Cross-referencing @@ -958,35 +944,34 @@ These roles link to the given declaration types: cpp:enum cpp:enumerator - Reference a C++ declaration by name (see below for details). - The name must be properly qualified relative to the position of the link. + Reference a C++ declaration by name (see below for details). The name must + be properly qualified relative to the position of the link. .. admonition:: Note on References with Templates Parameters/Arguments - Sphinx's syntax to give references a custom title can interfere with - linking to class templates, if nothing follows the closing angle - bracket, i.e. if the link looks like this: ``:cpp:class:`MyClass<int>```. - This is interpreted as a link to ``int`` with a title of ``MyClass``. - In this case, please escape the opening angle bracket with a backslash, - like this: ``:cpp:class:`MyClass\<int>```. + Sphinx's syntax to give references a custom title can interfere with linking + to class templates, if nothing follows the closing angle bracket, i.e. if + the link looks like this: ``:cpp:class:`MyClass<int>```. This is + interpreted as a link to ``int`` with a title of ``MyClass``. In this case, + please escape the opening angle bracket with a backslash, like this: + ``:cpp:class:`MyClass\<int>```. .. admonition:: Note on References to Overloaded Functions - It is currently impossible to link to a specific version of an - overloaded method. Currently the C++ domain is the first domain - that has basic support for overloaded methods and until there is more - data for comparison we don't want to select a bad syntax to reference a - specific overload. Currently Sphinx will link to the first overloaded - version of the method / function. + It is currently impossible to link to a specific version of an overloaded + method. Currently the C++ domain is the first domain that has basic support + for overloaded methods and until there is more data for comparison we don't + want to select a bad syntax to reference a specific overload. Currently + Sphinx will link to the first overloaded version of the method / function. Declarations without template parameters and template arguments -................................................................. +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -For linking to non-templated declarations the name must be a nested name, -e.g., ``f`` or ``MyClass::f``. +For linking to non-templated declarations the name must be a nested name, e.g., +``f`` or ``MyClass::f``. Templated declarations -...................... +^^^^^^^^^^^^^^^^^^^^^^ Assume the following declarations. @@ -998,23 +983,25 @@ Assume the following declarations. .. cpp:class:: template<typename TInner> \ Inner -In general the reference must include the template paraemter declarations, e.g., -``template\<typename TOuter> Wrapper::Outer`` (:cpp:class:`template\<typename TOuter> Wrapper::Outer`). -Currently the lookup only succeed if the template parameter identifiers are equal strings. That is, +In general the reference must include the template paraemter declarations, +e.g., ``template\<typename TOuter> Wrapper::Outer`` +(:cpp:class:`template\<typename TOuter> Wrapper::Outer`). Currently the lookup +only succeed if the template parameter identifiers are equal strings. That is, ``template\<typename UOuter> Wrapper::Outer`` will not work. -The inner class template can not be directly referenced, unless the current namespace -is changed or the following shorthand is used. -If a template parameter list is omitted, then the lookup will assume either a template or a non-template, -but not a partial template specialisation. -This means the following references work. +The inner class template can not be directly referenced, unless the current +namespace is changed or the following shorthand is used. If a template +parameter list is omitted, then the lookup will assume either a template or a +non-template, but not a partial template specialisation. This means the +following references work. - ``Wrapper::Outer`` (:cpp:class:`Wrapper::Outer`) - ``Wrapper::Outer::Inner`` (:cpp:class:`Wrapper::Outer::Inner`) -- ``template\<typename TInner> Wrapper::Outer::Inner`` (:cpp:class:`template\<typename TInner> Wrapper::Outer::Inner`) +- ``template\<typename TInner> Wrapper::Outer::Inner`` + (:cpp:class:`template\<typename TInner> Wrapper::Outer::Inner`) (Full) Template Specialisations -................................ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Assume the following declarations. @@ -1033,25 +1020,27 @@ Assume the following declarations. .. cpp:class:: template<> \ Inner<bool> -In general the reference must include a template parameter list for each template argument list. -The full specialisation above can therefore be referenced with ``template\<> Outer\<int>`` (:cpp:class:`template\<> Outer\<int>`) -and ``template\<> template\<> Outer\<int>::Inner\<bool>`` (:cpp:class:`template\<> template\<> Outer\<int>::Inner\<bool>`). -As a shorthand the empty template parameter list can be omitted, e.g., ``Outer\<int>`` (:cpp:class:`Outer\<int>`) -and ``Outer\<int>::Inner\<bool>`` (:cpp:class:`Outer\<int>::Inner\<bool>`). - +In general the reference must include a template parameter list for each +template argument list. The full specialisation above can therefore be +referenced with ``template\<> Outer\<int>`` (:cpp:class:`template\<> +Outer\<int>`) and ``template\<> template\<> Outer\<int>::Inner\<bool>`` +(:cpp:class:`template\<> template\<> Outer\<int>::Inner\<bool>`). As a +shorthand the empty template parameter list can be omitted, e.g., +``Outer\<int>`` (:cpp:class:`Outer\<int>`) and ``Outer\<int>::Inner\<bool>`` +(:cpp:class:`Outer\<int>::Inner\<bool>`). Partial Template Specialisations -................................. +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Assume the following declaration. .. cpp:class:: template<typename T> \ Outer<T*> -References to partial specialisations must always include the template parameter lists, e.g., -``template\<typename T> Outer\<T*>`` (:cpp:class:`template\<typename T> Outer\<T*>`). -Currently the lookup only succeed if the template parameter identifiers are equal strings. - +References to partial specialisations must always include the template +parameter lists, e.g., ``template\<typename T> Outer\<T*>`` +(:cpp:class:`template\<typename T> Outer\<T*>`). Currently the lookup only +succeed if the template parameter identifiers are equal strings. Configuration Variables ~~~~~~~~~~~~~~~~~~~~~~~ @@ -1073,8 +1062,8 @@ There is a set of directives allowing documenting command-line programs: .. rst:directive:: .. option:: name args, name args, ... - Describes a command line argument or switch. Option argument names should be - enclosed in angle brackets. Examples:: + Describes a command line argument or switch. Option argument names should + be enclosed in angle brackets. Examples:: .. option:: dest_dir @@ -1092,8 +1081,8 @@ There is a set of directives allowing documenting command-line programs: .. rst:directive:: .. envvar:: name - Describes an environment variable that the documented code or program uses or - defines. Referencable by :rst:role:`envvar`. + Describes an environment variable that the documented code or program uses + or defines. Referencable by :rst:role:`envvar`. .. rst:directive:: .. program:: name @@ -1120,12 +1109,11 @@ There is a set of directives allowing documenting command-line programs: then ``:option:`rm -r``` would refer to the first option, while ``:option:`svn -r``` would refer to the second one. - The program name may contain spaces (in case you want to document subcommands - like ``svn add`` and ``svn commit`` separately). + The program name may contain spaces (in case you want to document + subcommands like ``svn add`` and ``svn commit`` separately). .. versionadded:: 0.5 - There is also a very generic object description directive, which is not tied to any domain: @@ -1154,9 +1142,9 @@ The JavaScript domain (name **js**) provides the following directives: :rst:dir:`py:class` would, for example. By default, this directive will create a linkable entity and will cause an - entry in the global module index, unless the ``noindex`` option is specified. - If this option is specified, the directive will only update the current - module name. + entry in the global module index, unless the ``noindex`` option is + specified. If this option is specified, the directive will only update the + current module name. To clear the current module, set the module name to ``null`` or ``None`` @@ -1165,8 +1153,8 @@ The JavaScript domain (name **js**) provides the following directives: .. rst:directive:: .. js:function:: name(signature) Describes a JavaScript function or method. If you want to describe - arguments as optional use square brackets as :ref:`documented - <signatures>` for Python signatures. + arguments as optional use square brackets as :ref:`documented <signatures>` + for Python signatures. You can use fields to give more details about arguments and their expected types, errors which may be thrown by the function, and the value being @@ -1196,15 +1184,15 @@ The JavaScript domain (name **js**) provides the following directives: .. rst:directive:: .. js:method:: name(signature) - This directive is an alias for :rst:dir:`js:function`, however it describes a - function that is implemented as a method on a class object. + This directive is an alias for :rst:dir:`js:function`, however it describes + a function that is implemented as a method on a class object. .. versionadded:: 1.6 .. rst:directive:: .. js:class:: name - Describes a constructor that creates an object. This is basically like - a function but will show up with a `class` prefix:: + Describes a constructor that creates an object. This is basically like a + function but will show up with a `class` prefix:: .. js:class:: MyAnimal(name[, age]) @@ -1297,21 +1285,20 @@ currently Ada_, CoffeeScript_, Erlang_, HTTP_, Lasso_, MATLAB_, PHP_, and Ruby_ domains. Also available are domains for `Chapel`_, `Common Lisp`_, dqn_, Go_, Jinja_, Operation_, and Scala_. - .. _sphinx-contrib: https://bitbucket.org/birkenfeld/sphinx-contrib/ -.. _Ada: https://pypi.python.org/pypi/sphinxcontrib-adadomain -.. _Chapel: https://pypi.python.org/pypi/sphinxcontrib-chapeldomain -.. _CoffeeScript: https://pypi.python.org/pypi/sphinxcontrib-coffee -.. _Common Lisp: https://pypi.python.org/pypi/sphinxcontrib-cldomain -.. _dqn: https://pypi.python.org/pypi/sphinxcontrib-dqndomain -.. _Erlang: https://pypi.python.org/pypi/sphinxcontrib-erlangdomain -.. _Go: https://pypi.python.org/pypi/sphinxcontrib-golangdomain -.. _HTTP: https://pypi.python.org/pypi/sphinxcontrib-httpdomain -.. _Jinja: https://pypi.python.org/pypi/sphinxcontrib-jinjadomain -.. _Lasso: https://pypi.python.org/pypi/sphinxcontrib-lassodomain -.. _MATLAB: https://pypi.python.org/pypi/sphinxcontrib-matlabdomain -.. _Operation: https://pypi.python.org/pypi/sphinxcontrib-operationdomain -.. _PHP: https://pypi.python.org/pypi/sphinxcontrib-phpdomain +.. _Ada: https://pypi.org/project/sphinxcontrib-adadomain/ +.. _Chapel: https://pypi.org/project/sphinxcontrib-chapeldomain/ +.. _CoffeeScript: https://pypi.org/project/sphinxcontrib-coffee/ +.. _Common Lisp: https://pypi.org/project/sphinxcontrib-cldomain/ +.. _dqn: https://pypi.org/project/sphinxcontrib-dqndomain/ +.. _Erlang: https://pypi.org/project/sphinxcontrib-erlangdomain/ +.. _Go: https://pypi.org/project/sphinxcontrib-golangdomain/ +.. _HTTP: https://pypi.org/project/sphinxcontrib-httpdomain/ +.. _Jinja: https://pypi.org/project/sphinxcontrib-jinjadomain/ +.. _Lasso: https://pypi.org/project/sphinxcontrib-lassodomain/ +.. _MATLAB: https://pypi.org/project/sphinxcontrib-matlabdomain/ +.. _Operation: https://pypi.org/project/sphinxcontrib-operationdomain/ +.. _PHP: https://pypi.org/project/sphinxcontrib-phpdomain/ .. _Ruby: https://bitbucket.org/birkenfeld/sphinx-contrib/src/default/rubydomain -.. _Scala: https://pypi.python.org/pypi/sphinxcontrib-scaladomain +.. _Scala: https://pypi.org/project/sphinxcontrib-scaladomain/ diff --git a/doc/usage/restructuredtext/field-lists.rst b/doc/usage/restructuredtext/field-lists.rst new file mode 100644 index 000000000..fcecfe708 --- /dev/null +++ b/doc/usage/restructuredtext/field-lists.rst @@ -0,0 +1,47 @@ +.. highlight:: rst + +=========== +Field Lists +=========== + +:ref:`As previously discussed <rst-field-lists>`, field lists are sequences of +fields marked up like this:: + + :fieldname: Field content + +Sphinx provides custom behavior for bibliographic fields compared to docutils. + +.. _metadata: + +File-wide metadata +------------------ + +A field list near the top of a file is normally parsed by docutils as the +*docinfo* which is generally used to record the author, date of publication and +other metadata. However, in Sphinx, a field list preceding any other markup is +moved from the *docinfo* to the Sphinx environment as document metadata and is +not displayed in the output; a field list appearing after the document title +will be part of the *docinfo* as normal and will be displayed in the output. + +At the moment, these metadata fields are recognized: + +``tocdepth`` + The maximum depth for a table of contents of this file. :: + + :tocdepth: 2 + + .. versionadded:: 0.4 + +``nocomments`` + If set, the web application won't display a comment form for a page + generated from this source file. :: + + :nocomments: + +``orphan`` + If set, warnings about this file not being included in any toctree will be + suppressed. :: + + :orphan: + + .. versionadded:: 1.0 diff --git a/doc/usage/restructuredtext/index.rst b/doc/usage/restructuredtext/index.rst new file mode 100644 index 000000000..ea346b3aa --- /dev/null +++ b/doc/usage/restructuredtext/index.rst @@ -0,0 +1,24 @@ +.. _rst-index: + +================ +reStructuredText +================ + +reStructuredText (reST) is the default plaintext markup language used by both +Docutils and Sphinx. Docutils provides the basic reStructuredText syntax, while +Sphinx extends this to support additional functionality. + +The below guides go through the most important aspects of reST. For the +authoritative reStructuredText reference, refer to the `docutils +documentation`__. + +__ http://docutils.sourceforge.net/rst.html + +.. toctree:: + :maxdepth: 2 + + basics + roles + directives + field-lists + domains diff --git a/doc/markup/inline.rst b/doc/usage/restructuredtext/roles.rst index c8dfb6ff7..04991c3a9 100644 --- a/doc/markup/inline.rst +++ b/doc/usage/restructuredtext/roles.rst @@ -1,9 +1,8 @@ -.. highlight:: rest +.. highlight:: rst -.. _inline-markup: - -Inline markup -============= +===== +Roles +===== Sphinx uses interpreted text roles to insert semantic markup into documents. They are written as ``:rolename:`content```. @@ -16,21 +15,21 @@ They are written as ``:rolename:`content```. :rst:role:`any` role to find anything or the :rst:role:`py:obj` role to find Python objects are very useful for this. -See :ref:`domains` for roles added by domains. +See :doc:`/usage/restructuredtext/domains` for roles added by domains. .. _xref-syntax: Cross-referencing syntax -~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------ Cross-references are generated by many semantic interpreted text roles. -Basically, you only need to write ``:role:`target```, and a link will be created -to the item named *target* of the type indicated by *role*. The link's text -will be the same as *target*. +Basically, you only need to write ``:role:`target```, and a link will be +created to the item named *target* of the type indicated by *role*. The link's +text will be the same as *target*. -There are some additional facilities, however, that make cross-referencing roles -more versatile: +There are some additional facilities, however, that make cross-referencing +roles more versatile: * You may supply an explicit title and reference target, like in reST direct hyperlinks: ``:role:`title <target>``` will refer to *target*, but the link @@ -50,7 +49,7 @@ more versatile: .. _any-role: Cross-referencing anything --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^ .. rst:role:: any @@ -90,9 +89,8 @@ Cross-referencing anything :mod:`~sphinx.ext.intersphinx` extension: when no local cross-reference is found, all object types of intersphinx inventories are also searched. - Cross-referencing objects -------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^ These roles are described with their respective domains: @@ -106,7 +104,7 @@ These roles are described with their respective domains: .. _ref-role: Cross-referencing arbitrary locations -------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. rst:role:: ref @@ -116,7 +114,7 @@ Cross-referencing arbitrary locations refer to labels: * If you place a label directly before a section title, you can reference to - it with ``:ref:`label-name```. Example:: + it with ``:ref:`label-name```. For example:: .. _my-reference-label: @@ -127,11 +125,11 @@ Cross-referencing arbitrary locations It refers to the section itself, see :ref:`my-reference-label`. - The ``:ref:`` role would then generate a link to the section, with the link - title being "Section to cross-reference". This works just as well when - section and reference are in different source files. + The ``:ref:`` role would then generate a link to the section, with the + link title being "Section to cross-reference". This works just as well + when section and reference are in different source files. - Automatic labels also work with figures: given :: + Automatic labels also work with figures. For example:: .. _my-figure: @@ -139,8 +137,8 @@ Cross-referencing arbitrary locations Figure caption - a reference ``:ref:`my-figure``` would insert a reference to the figure - with link text "Figure caption". + In this case, a reference ``:ref:`my-figure``` would insert a reference + to the figure with link text "Figure caption". The same works for tables that are given an explicit caption using the :dudir:`table` directive. @@ -151,17 +149,17 @@ Cross-referencing arbitrary locations .. note:: - Reference labels must start with an underscore. When referencing a - label, the underscore must be omitted (see examples above). + Reference labels must start with an underscore. When referencing a label, + the underscore must be omitted (see examples above). Using :rst:role:`ref` is advised over standard reStructuredText links to sections (like ```Section title`_``) because it works across files, when - section headings are changed, and for all builders that support - cross-references. + section headings are changed, will raise warnings if incorrect, and works + for all builders that support cross-references. Cross-referencing documents ---------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. versionadded:: 0.6 @@ -180,7 +178,7 @@ There is also a way to directly link to documents: Referencing downloadable files ------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. versionadded:: 0.6 @@ -213,7 +211,7 @@ Referencing downloadable files See :download:`this example script <../example.py>`. Cross-referencing figures by figure number ------------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. versionadded:: 1.3 @@ -238,7 +236,7 @@ Cross-referencing figures by figure number so this role inserts not a reference but the label or the link text. Cross-referencing other items of interest ------------------------------------------ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The following roles do possibly create a cross-reference, but do not refer to objects: @@ -280,10 +278,10 @@ The following role creates a cross-reference to a term in a Other semantic markup -~~~~~~~~~~~~~~~~~~~~~ +--------------------- -The following roles don't do anything special except formatting the text -in a different style: +The following roles don't do anything special except formatting the text in a +different style: .. rst:role:: abbr @@ -331,22 +329,22 @@ in a different style: .. rst:role:: kbd Mark a sequence of keystrokes. What form the key sequence takes may depend - on platform- or application-specific conventions. When there are no relevant - conventions, the names of modifier keys should be spelled out, to improve - accessibility for new users and non-native speakers. For example, an - *xemacs* key sequence may be marked like ``:kbd:`C-x C-f```, but without + on platform- or application-specific conventions. When there are no + relevant conventions, the names of modifier keys should be spelled out, to + improve accessibility for new users and non-native speakers. For example, + an *xemacs* key sequence may be marked like ``:kbd:`C-x C-f```, but without reference to a specific application or platform, the same sequence should be marked as ``:kbd:`Control-x Control-f```. .. rst:role:: mailheader The name of an RFC 822-style mail header. This markup does not imply that - the header is being used in an email message, but can be used to refer to any - header of the same "style." This is also used for headers defined by the - various MIME specifications. The header name should be entered in the same - way it would normally be found in practice, with the camel-casing conventions - being preferred where there is more than one common usage. For example: - ``:mailheader:`Content-Type```. + the header is being used in an email message, but can be used to refer to + any header of the same "style." This is also used for headers defined by + the various MIME specifications. The header name should be entered in the + same way it would normally be found in practice, with the camel-casing + conventions being preferred where there is more than one common usage. For + example: ``:mailheader:`Content-Type```. .. rst:role:: makevar @@ -354,9 +352,9 @@ in a different style: .. rst:role:: manpage - A reference to a Unix manual page including the section, - e.g. ``:manpage:`ls(1)```. Creates a hyperlink to an external site - rendering the manpage if :confval:`manpages_url` is defined. + A reference to a Unix manual page including the section, e.g. + ``:manpage:`ls(1)```. Creates a hyperlink to an external site rendering the + manpage if :confval:`manpages_url` is defined. .. rst:role:: menuselection @@ -370,8 +368,8 @@ in a different style: :menuselection:`Start --> Programs` - When including a selection that includes some trailing indicator, such as the - ellipsis some operating systems use to indicate that the command opens a + When including a selection that includes some trailing indicator, such as + the ellipsis some operating systems use to indicate that the command opens a dialog, the indicator should be omitted from the selection name. ``menuselection`` also supports ampersand accelerators just like @@ -386,6 +384,8 @@ in a different style: The name of a Usenet newsgroup. +.. todo:: Is this not part of the standard domain? + .. rst:role:: program The name of an executable program. This may differ from the file name for @@ -406,6 +406,9 @@ in a different style: If you don't need the "variable part" indication, use the standard ````code```` instead. + .. versionchanged:: 1.8 + Allowed to escape curly braces with backslash + There is also an :rst:role:`index` role to generate index entries. The following roles generate external links: @@ -432,7 +435,7 @@ the standard reST markup for that purpose. .. _default-substitutions: Substitutions -~~~~~~~~~~~~~ +------------- The documentation system provides three substitutions that are defined by default. They are set in the build configuration file. diff --git a/doc/web/api.rst b/doc/web/api.rst index 05ea68e71..81d25b79f 100644 --- a/doc/web/api.rst +++ b/doc/web/api.rst @@ -40,9 +40,17 @@ The WebSupport Class comment that was added. staticdir - If static files are served from a location besides ``'/static'``, this - should be a string with the name of that location - (e.g. ``'/static_files'``). + If the static files should be created in a different location + **and not in** ``'/static'``, this should be a string with the name of + that location (e.g. ``builddir + '/static_files'``). + + .. note:: + If you specify ``staticdir``, you will typically want to adjust + ``staticroot`` accordingly. + + staticroot + If the static files are not served from ``'/static'``, this should be a + string with the name of that location (e.g. ``'/static_files'``). docroot If the documentation is not served from the base path of a URL, this @@ -2,6 +2,7 @@ import os import sys from distutils import log +from io import StringIO from setuptools import find_packages, setup @@ -63,6 +64,20 @@ extras_require = { cmdclass = {} + +class Tee(object): + def __init__(self, stream): + self.stream = stream + self.buffer = StringIO() + + def write(self, s): + self.stream.write(s) + self.buffer.write(s) + + def flush(self): + self.stream.flush() + + try: from babel.messages.pofile import read_po from babel.messages.frontend import compile_catalog @@ -80,7 +95,13 @@ else: """ def run(self): - compile_catalog.run(self) + try: + sys.stderr = Tee(sys.stderr) + compile_catalog.run(self) + finally: + if sys.stderr.buffer.getvalue(): + print("Compiling failed.") + sys.exit(1) if isinstance(self.domain, list): for domain in self.domain: @@ -155,7 +176,7 @@ setup( name='Sphinx', version=sphinx.__version__, url='http://sphinx-doc.org/', - download_url='https://pypi.python.org/pypi/Sphinx', + download_url='https://pypi.org/project/Sphinx/', license='BSD', author='Georg Brandl', author_email='georg@python.org', diff --git a/sphinx/__init__.py b/sphinx/__init__.py index 7b3ddfea8..8ffeb4d93 100644 --- a/sphinx/__init__.py +++ b/sphinx/__init__.py @@ -22,6 +22,12 @@ from os import path from .deprecation import RemovedInNextVersionWarning from .deprecation import RemovedInSphinx20Warning +if False: + # For type annotation + # note: Don't use typing.TYPE_CHECK here (for py27 and py34). + from typing import Any # NOQA + + # by default, all DeprecationWarning under sphinx package will be emit. # Users can avoid this by using environment variable: PYTHONWARNINGS= if 'PYTHONWARNINGS' not in os.environ: @@ -31,13 +37,18 @@ if 'PYTHONWARNINGS' not in os.environ: warnings.filterwarnings('ignore', "'U' mode is deprecated", DeprecationWarning, module='docutils.io') -__version__ = '1.7.5+' -__released__ = '1.7.5' # used when Sphinx builds its own docs +__version__ = '1.8.0+' +__released__ = '1.8.0' # used when Sphinx builds its own docs -# version info for better programmatic use -# possible values for 3rd element: 'alpha', 'beta', 'rc', 'final' -# 'final' has 0 as the last element -version_info = (1, 7, 5, 'beta', 0) +#: Version info for better programmatic use. +#: +#: A tuple of five elements; for Sphinx version 1.2.1 beta 3 this would be +#: ``(1, 2, 1, 'beta', 3)``. The fourth element can be one of: ``alpha``, +#: ``beta``, ``rc``, ``final``. ``final`` always has 0 as the last element. +#: +#: .. versionadded:: 1.2 +#: Before version 1.2, check the string ``sphinx.__version__``. +version_info = (1, 8, 0, 'beta', 0) package_dir = path.abspath(path.dirname(__file__)) @@ -61,6 +72,7 @@ if __version__.endswith('+'): def main(*args, **kwargs): + # type: (Any, Any) -> int from .cmd import build warnings.warn( '`sphinx.main()` has moved to `sphinx.cmd.build.main()`.', diff --git a/sphinx/__main__.py b/sphinx/__main__.py index 47a183d08..02bc806e7 100644 --- a/sphinx/__main__.py +++ b/sphinx/__main__.py @@ -13,4 +13,4 @@ import sys from sphinx.cmd.build import main -sys.exit(main(sys.argv[1:])) +sys.exit(main(sys.argv[1:])) # type: ignore diff --git a/sphinx/apidoc.py b/sphinx/apidoc.py index be5e5c5ab..dffa4a9ef 100644 --- a/sphinx/apidoc.py +++ b/sphinx/apidoc.py @@ -14,8 +14,14 @@ import warnings from sphinx.deprecation import RemovedInSphinx20Warning from sphinx.ext.apidoc import main as _main +if False: + # For type annotation + from typing import Any # NOQA + from sphinx.application import Sphinx # NOQA + def main(*args, **kwargs): + # type: (Any, Any) -> None warnings.warn( '`sphinx.apidoc.main()` has moved to `sphinx.ext.apidoc.main()`.', RemovedInSphinx20Warning, diff --git a/sphinx/application.py b/sphinx/application.py index 7af1c8d39..9d3d5de9f 100644 --- a/sphinx/application.py +++ b/sphinx/application.py @@ -3,7 +3,7 @@ sphinx.application ~~~~~~~~~~~~~~~~~~ - Sphinx application object. + Sphinx application class and extensibility interface. Gracefully adapted from the TextPress system by Armin. @@ -17,35 +17,41 @@ import posixpath import sys import warnings from collections import deque +from inspect import isclass from os import path -from docutils import nodes -from docutils.parsers.rst import directives, roles -from six import iteritems, itervalues +from docutils.parsers.rst import Directive, directives, roles +from six import itervalues +from six.moves import cPickle as pickle from six.moves import cStringIO import sphinx from sphinx import package_dir, locale -from sphinx.config import Config -from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx.config import Config, check_unicode +from sphinx.config import CONFIG_FILENAME # NOQA # for compatibility (RemovedInSphinx30) +from sphinx.deprecation import ( + RemovedInSphinx20Warning, RemovedInSphinx30Warning, RemovedInSphinx40Warning +) from sphinx.environment import BuildEnvironment -from sphinx.errors import ConfigError, ExtensionError, VersionRequirementError +from sphinx.errors import ApplicationError, ConfigError, VersionRequirementError from sphinx.events import EventManager -from sphinx.extension import verify_required_extensions from sphinx.locale import __ from sphinx.registry import SphinxComponentRegistry +from sphinx.util import docutils from sphinx.util import import_object from sphinx.util import logging from sphinx.util import pycompat # noqa: F401 +from sphinx.util.build_phase import BuildPhase from sphinx.util.console import bold # type: ignore -from sphinx.util.docutils import is_html5_writer_available, directive_helper +from sphinx.util.docutils import directive_helper from sphinx.util.i18n import find_catalog_source_files -from sphinx.util.osutil import ENOENT, ensuredir, relpath +from sphinx.util.osutil import abspath, ensuredir, relpath from sphinx.util.tags import Tags if False: # For type annotation from typing import Any, Callable, Dict, IO, Iterable, Iterator, List, Tuple, Type, Union # NOQA + from docutils import nodes # NOQA from docutils.parsers import Parser # NOQA from docutils.transforms import Transform # NOQA from sphinx.builders import Builder # NOQA @@ -54,7 +60,7 @@ if False: from sphinx.extension import Extension # NOQA from sphinx.roles import XRefRole # NOQA from sphinx.theming import Theme # NOQA - from sphinx.util.typing import RoleFunction # NOQA + from sphinx.util.typing import RoleFunction, TitleGetter # NOQA builtin_extensions = ( 'sphinx.builders.applehelp', @@ -73,6 +79,7 @@ builtin_extensions = ( 'sphinx.builders.text', 'sphinx.builders.websupport', 'sphinx.builders.xml', + 'sphinx.config', 'sphinx.domains.c', 'sphinx.domains.cpp', 'sphinx.domains.javascript', @@ -83,8 +90,10 @@ builtin_extensions = ( 'sphinx.directives.code', 'sphinx.directives.other', 'sphinx.directives.patches', + 'sphinx.extension', 'sphinx.io', 'sphinx.parsers', + 'sphinx.registry', 'sphinx.roles', 'sphinx.transforms.post_transforms', 'sphinx.transforms.post_transforms.images', @@ -101,32 +110,52 @@ builtin_extensions = ( 'alabaster', ) # type: Tuple[unicode, ...] -CONFIG_FILENAME = 'conf.py' ENV_PICKLE_FILENAME = 'environment.pickle' logger = logging.getLogger(__name__) class Sphinx(object): + """The main application class and extensibility interface. + + :ivar srcdir: Directory containing source. + :ivar confdir: Directory containing ``conf.py``. + :ivar doctreedir: Directory for storing pickled doctrees. + :ivar outdir: Directory for storing build documents. + """ def __init__(self, srcdir, confdir, outdir, doctreedir, buildername, confoverrides=None, status=sys.stdout, warning=sys.stderr, freshenv=False, warningiserror=False, tags=None, verbosity=0, parallel=0): # type: (unicode, unicode, unicode, unicode, unicode, Dict, IO, IO, bool, bool, List[unicode], int, int) -> None # NOQA + self.phase = BuildPhase.INITIALIZATION self.verbosity = verbosity self.extensions = {} # type: Dict[unicode, Extension] self._setting_up_extension = ['?'] # type: List[unicode] self.builder = None # type: Builder self.env = None # type: BuildEnvironment self.registry = SphinxComponentRegistry() - self.enumerable_nodes = {} # type: Dict[nodes.Node, Tuple[unicode, Callable]] # NOQA self.html_themes = {} # type: Dict[unicode, unicode] - self.srcdir = srcdir # type: unicode + # validate provided directories + self.srcdir = abspath(srcdir) # type: unicode + self.outdir = abspath(outdir) # type: unicode + self.doctreedir = abspath(doctreedir) # type: unicode self.confdir = confdir - self.outdir = outdir - self.doctreedir = doctreedir + if self.confdir: # confdir is optional + self.confdir = abspath(self.confdir) + if not path.isfile(path.join(self.confdir, 'conf.py')): + raise ApplicationError(__("config directory doesn't contain a " + "conf.py file (%s)") % confdir) + + if not path.isdir(self.srcdir): + raise ApplicationError(__('Cannot find source directory (%s)') % + self.srcdir) + + if self.srcdir == self.outdir: + raise ApplicationError(__('Source directory and destination ' + 'directory cannot be identical')) self.parallel = parallel @@ -152,17 +181,18 @@ class Sphinx(object): self.messagelog = deque(maxlen=10) # type: deque # say hello to the world - logger.info(bold('Running Sphinx v%s' % sphinx.__display_version__)) + logger.info(bold(__('Running Sphinx v%s') % sphinx.__display_version__)) # status code for command-line application self.statuscode = 0 # read config self.tags = Tags(tags) - self.config = Config(confdir, CONFIG_FILENAME, - confoverrides or {}, self.tags) - self.config.check_unicode() - # defer checking types until i18n has been initialized + if self.confdir is None: + self.config = Config({}, confoverrides or {}) + else: + self.config = Config.read(self.confdir, confoverrides or {}, self.tags) + check_unicode(self.config) # initialize some limited config variables before initialize i18n and loading # extensions @@ -194,14 +224,13 @@ class Sphinx(object): self.preload_builder(buildername) if not path.isdir(outdir): - logger.info('making output directory...') + logger.info(__('making output directory...')) ensuredir(outdir) # the config file itself can be an extension if self.config.setup: self._setting_up_extension = ['conf.py'] - # py31 doesn't have 'callable' function for below check - if hasattr(self.config.setup, '__call__'): + if callable(self.config.setup): self.config.setup(self) else: raise ConfigError( @@ -212,9 +241,7 @@ class Sphinx(object): # now that we know all config values, collect them from conf.py self.config.init_values() - - # check extension versions if requested - verify_required_extensions(self, self.config.needs_extensions) + self.emit('config-inited', self.config) # check primary_domain if requested primary_domain = self.config.primary_domain @@ -223,16 +250,10 @@ class Sphinx(object): # create the builder self.builder = self.create_builder(buildername) - # check all configuration values for permissible types - self.config.check_types() - # set up source_parsers - self._init_source_parsers() # set up the build environment self._init_env(freshenv) # set up the builder self._init_builder() - # set up the enumerable nodes - self._init_enumerable_nodes() def _init_i18n(self): # type: () -> None @@ -240,7 +261,7 @@ class Sphinx(object): the configuration. """ if self.config.language is not None: - logger.info(bold('loading translations [%s]... ' % self.config.language), + logger.info(bold(__('loading translations [%s]... ') % self.config.language), nonl=True) user_locale_dirs = [ path.join(self.srcdir, x) for x in self.config.locale_dirs] @@ -252,25 +273,18 @@ class Sphinx(object): locale_dirs = [None, path.join(package_dir, 'locale')] + user_locale_dirs # type: ignore # NOQA else: locale_dirs = [] - self.translator, has_translation = locale.init(locale_dirs, self.config.language) + self.translator, has_translation = locale.init(locale_dirs, self.config.language) # type: ignore # NOQA if self.config.language is not None: if has_translation or self.config.language == 'en': # "en" never needs to be translated logger.info(__('done')) else: - logger.info('not available for built-in messages') - - def _init_source_parsers(self): - # type: () -> None - for suffix, parser in iteritems(self.config.source_parsers): - self.add_source_parser(suffix, parser) - for suffix, parser in iteritems(self.registry.get_source_parsers()): - if suffix not in self.config.source_suffix and suffix != '*': - self.config.source_suffix.append(suffix) + logger.info(__('not available for built-in messages')) def _init_env(self, freshenv): # type: (bool) -> None - if freshenv: + filename = path.join(self.doctreedir, ENV_PICKLE_FILENAME) + if freshenv or not os.path.exists(filename): self.env = BuildEnvironment(self) self.env.find_files(self.config, self.builder) for domain in self.registry.create_domains(self.env): @@ -278,18 +292,20 @@ class Sphinx(object): else: try: logger.info(bold(__('loading pickled environment... ')), nonl=True) - filename = path.join(self.doctreedir, ENV_PICKLE_FILENAME) - self.env = BuildEnvironment.frompickle(filename, self) + with open(filename, 'rb') as f: + self.env = pickle.load(f) + self.env.app = self + self.env.config.values = self.config.values + needed, reason = self.env.need_refresh(self) + if needed: + raise IOError(reason) self.env.domains = {} for domain in self.registry.create_domains(self.env): # this can raise if the data version doesn't fit self.env.domains[domain.name] = domain logger.info(__('done')) except Exception as err: - if isinstance(err, IOError) and err.errno == ENOENT: - logger.info(__('not yet created')) - else: - logger.info(__('failed: %s'), err) + logger.info(__('failed: %s'), err) self._init_env(freshenv=True) def preload_builder(self, name): @@ -310,15 +326,11 @@ class Sphinx(object): self.builder.init() self.emit('builder-inited') - def _init_enumerable_nodes(self): - # type: () -> None - for node, settings in iteritems(self.enumerable_nodes): - self.env.get_domain('std').enumerable_nodes[node] = settings # type: ignore - # ---- main "build" method ------------------------------------------------- def build(self, force_all=False, filenames=None): # type: (bool, List[unicode]) -> None + self.phase = BuildPhase.READING try: if force_all: self.builder.compile_all_catalogs() @@ -334,7 +346,7 @@ class Sphinx(object): __('succeeded') or __('finished with problems')) if self._warncount: logger.info(bold(__('build %s, %s warning.', - 'build %s, %s warnings.', self._warncount) % + 'build %s, %s warnings.', self._warncount) % (status, self._warncount))) else: logger.info(bold(__('build %s.') % status)) @@ -361,10 +373,15 @@ class Sphinx(object): # type: (unicode, unicode, unicode, unicode) -> None """Emit a warning. - If *location* is given, it should either be a tuple of (docname, lineno) - or a string describing the location of the warning as well as possible. + If *location* is given, it should either be a tuple of (*docname*, + *lineno*) or a string describing the location of the warning as well as + possible. - *type* and *subtype* are used to suppress warnings with :confval:`suppress_warnings`. + *type* and *subtype* are used to suppress warnings with + :confval:`suppress_warnings`. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. """ warnings.warn('app.warning() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) @@ -376,6 +393,9 @@ class Sphinx(object): If *nonl* is true, don't emit a newline at the end (which implies that more info output will follow soon.) + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. """ warnings.warn('app.info() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) @@ -383,21 +403,33 @@ class Sphinx(object): def verbose(self, message, *args, **kwargs): # type: (unicode, Any, Any) -> None - """Emit a verbose informational message.""" + """Emit a verbose informational message. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. + """ warnings.warn('app.verbose() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) logger.verbose(message, *args, **kwargs) def debug(self, message, *args, **kwargs): # type: (unicode, Any, Any) -> None - """Emit a debug-level informational message.""" + """Emit a debug-level informational message. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. + """ warnings.warn('app.debug() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) logger.debug(message, *args, **kwargs) def debug2(self, message, *args, **kwargs): # type: (unicode, Any, Any) -> None - """Emit a lowlevel debug-level informational message.""" + """Emit a lowlevel debug-level informational message. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. + """ warnings.warn('app.debug2() is now deprecated. Use debug() instead.', RemovedInSphinx20Warning) logger.debug(message, *args, **kwargs) @@ -406,35 +438,68 @@ class Sphinx(object): def setup_extension(self, extname): # type: (unicode) -> None - """Import and setup a Sphinx extension module. No-op if called twice.""" + """Import and setup a Sphinx extension module. + + Load the extension given by the module *name*. Use this if your + extension needs the features provided by another extension. No-op if + called twice. + """ logger.debug('[app] setting up extension: %r', extname) self.registry.load_extension(self, extname) def require_sphinx(self, version): # type: (unicode) -> None - # check the Sphinx version if requested + """Check the Sphinx version if requested. + + Compare *version* (which must be a ``major.minor`` version string, e.g. + ``'1.1'``) with the version of the running Sphinx, and abort the build + when it is too old. + + .. versionadded:: 1.0 + """ if version > sphinx.__display_version__[:3]: raise VersionRequirementError(version) def import_object(self, objname, source=None): # type: (str, unicode) -> Any - """Import an object from a 'module.name' string.""" + """Import an object from a ``module.name`` string. + + .. deprecated:: 1.8 + Use ``sphinx.util.import_object()`` instead. + """ + warnings.warn('app.import_object() is deprecated. ' + 'Use sphinx.util.add_object_type() instead.', + RemovedInSphinx30Warning) return import_object(objname, source=None) # event interface def connect(self, event, callback): # type: (unicode, Callable) -> int + """Register *callback* to be called when *event* is emitted. + + For details on available core events and the arguments of callback + functions, please see :ref:`events`. + + The method returns a "listener ID" that can be used as an argument to + :meth:`disconnect`. + """ listener_id = self.events.connect(event, callback) logger.debug('[app] connecting event %r: %r [id=%s]', event, callback, listener_id) return listener_id def disconnect(self, listener_id): # type: (int) -> None + """Unregister callback by *listener_id*.""" logger.debug('[app] disconnecting event: [id=%s]', listener_id) self.events.disconnect(listener_id) def emit(self, event, *args): # type: (unicode, Any) -> List + """Emit *event* and pass *arguments* to the callback functions. + + Return the return values of all callbacks as a list. Do not emit core + Sphinx events in extensions! + """ try: logger.debug('[app] emitting event: %r%s', event, repr(args)[:100]) except Exception: @@ -445,110 +510,262 @@ class Sphinx(object): def emit_firstresult(self, event, *args): # type: (unicode, Any) -> Any + """Emit *event* and pass *arguments* to the callback functions. + + Return the result of the first callback that doesn't return ``None``. + + .. versionadded:: 0.5 + """ return self.events.emit_firstresult(event, self, *args) # registering addon parts - def add_builder(self, builder): - # type: (Type[Builder]) -> None - self.registry.add_builder(builder) + def add_builder(self, builder, override=False): + # type: (Type[Builder], bool) -> None + """Register a new builder. + + *builder* must be a class that inherits from + :class:`~sphinx.builders.Builder`. + + .. versionchanged:: 1.8 + Add *override* keyword. + """ + self.registry.add_builder(builder, override=override) + # TODO(stephenfin): Describe 'types' parameter def add_config_value(self, name, default, rebuild, types=()): # type: (unicode, Any, Union[bool, unicode], Any) -> None + """Register a configuration value. + + This is necessary for Sphinx to recognize new values and set default + values accordingly. The *name* should be prefixed with the extension + name, to avoid clashes. The *default* value can be any Python object. + The string value *rebuild* must be one of those values: + + * ``'env'`` if a change in the setting only takes effect when a + document is parsed -- this means that the whole environment must be + rebuilt. + * ``'html'`` if a change in the setting needs a full rebuild of HTML + documents. + * ``''`` if a change in the setting will not need any special rebuild. + + .. versionchanged:: 0.6 + Changed *rebuild* from a simple boolean (equivalent to ``''`` or + ``'env'``) to a string. However, booleans are still accepted and + converted internally. + + .. versionchanged:: 0.4 + If the *default* value is a callable, it will be called with the + config object as its argument in order to get the default value. + This can be used to implement config values whose default depends on + other values. + """ logger.debug('[app] adding config value: %r', (name, default, rebuild) + ((types,) if types else ())) # type: ignore - if name in self.config: - raise ExtensionError(__('Config value %r already present') % name) if rebuild in (False, True): rebuild = rebuild and 'env' or '' self.config.add(name, default, rebuild, types) def add_event(self, name): # type: (unicode) -> None + """Register an event called *name*. + + This is needed to be able to emit it. + """ logger.debug('[app] adding event: %r', name) self.events.add(name) - def set_translator(self, name, translator_class): - # type: (unicode, Type[nodes.NodeVisitor]) -> None - self.registry.add_translator(name, translator_class) + def set_translator(self, name, translator_class, override=False): + # type: (unicode, Type[nodes.NodeVisitor], bool) -> None + """Register or override a Docutils translator class. + + This is used to register a custom output translator or to replace a + builtin translator. This allows extensions to use custom translator + and define custom nodes for the translator (see :meth:`add_node`). - def add_node(self, node, **kwds): - # type: (nodes.Node, Any) -> None + .. versionadded:: 1.3 + .. versionchanged:: 1.8 + Add *override* keyword. + """ + self.registry.add_translator(name, translator_class, override=override) + + def add_node(self, node, override=False, **kwds): + # type: (nodes.Node, bool, Any) -> None + """Register a Docutils node class. + + This is necessary for Docutils internals. It may also be used in the + future to validate nodes in the parsed documents. + + Node visitor functions for the Sphinx HTML, LaTeX, text and manpage + writers can be given as keyword arguments: the keyword should be one or + more of ``'html'``, ``'latex'``, ``'text'``, ``'man'``, ``'texinfo'`` + or any other supported translators, the value a 2-tuple of ``(visit, + depart)`` methods. ``depart`` can be ``None`` if the ``visit`` + function raises :exc:`docutils.nodes.SkipNode`. Example: + + .. code-block:: python + + class math(docutils.nodes.Element): pass + + def visit_math_html(self, node): + self.body.append(self.starttag(node, 'math')) + def depart_math_html(self, node): + self.body.append('</math>') + + app.add_node(math, html=(visit_math_html, depart_math_html)) + + Obviously, translators for which you don't specify visitor methods will + choke on the node when encountered in a document to translate. + + .. versionchanged:: 0.5 + Added the support for keyword arguments giving visit functions. + """ logger.debug('[app] adding node: %r', (node, kwds)) - if not kwds.pop('override', False) and \ - hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__): + if not override and docutils.is_node_registered(node): logger.warning(__('while setting up extension %s: node class %r is ' 'already registered, its visitors will be overridden'), self._setting_up_extension, node.__name__, type='app', subtype='add_node') - nodes._add_node_class_names([node.__name__]) - for key, val in iteritems(kwds): - try: - visit, depart = val - except ValueError: - raise ExtensionError(__('Value for key %r must be a ' - '(visit, depart) function tuple') % key) - translator = self.registry.translators.get(key) - translators = [] - if translator is not None: - translators.append(translator) - elif key == 'html': - from sphinx.writers.html import HTMLTranslator - translators.append(HTMLTranslator) - if is_html5_writer_available(): - from sphinx.writers.html5 import HTML5Translator - translators.append(HTML5Translator) - elif key == 'latex': - from sphinx.writers.latex import LaTeXTranslator - translators.append(LaTeXTranslator) - elif key == 'text': - from sphinx.writers.text import TextTranslator - translators.append(TextTranslator) - elif key == 'man': - from sphinx.writers.manpage import ManualPageTranslator - translators.append(ManualPageTranslator) - elif key == 'texinfo': - from sphinx.writers.texinfo import TexinfoTranslator - translators.append(TexinfoTranslator) - - for translator in translators: - setattr(translator, 'visit_' + node.__name__, visit) - if depart: - setattr(translator, 'depart_' + node.__name__, depart) - - def add_enumerable_node(self, node, figtype, title_getter=None, **kwds): - # type: (nodes.Node, unicode, Callable, Any) -> None - self.enumerable_nodes[node] = (figtype, title_getter) - self.add_node(node, **kwds) - - def add_directive(self, name, obj, content=None, arguments=None, **options): - # type: (unicode, Any, bool, Tuple[int, int, bool], Any) -> None + docutils.register_node(node) + self.registry.add_translation_handlers(node, **kwds) + + def add_enumerable_node(self, node, figtype, title_getter=None, override=False, **kwds): + # type: (nodes.Node, unicode, TitleGetter, bool, Any) -> None + """Register a Docutils node class as a numfig target. + + Sphinx numbers the node automatically. And then the users can refer it + using :rst:role:`numref`. + + *figtype* is a type of enumerable nodes. Each figtypes have individual + numbering sequences. As a system figtypes, ``figure``, ``table`` and + ``code-block`` are defined. It is able to add custom nodes to these + default figtypes. It is also able to define new custom figtype if new + figtype is given. + + *title_getter* is a getter function to obtain the title of node. It + takes an instance of the enumerable node, and it must return its title + as string. The title is used to the default title of references for + :rst:role:`ref`. By default, Sphinx searches + ``docutils.nodes.caption`` or ``docutils.nodes.title`` from the node as + a title. + + Other keyword arguments are used for node visitor functions. See the + :meth:`Sphinx.add_node` for details. + + .. versionadded:: 1.4 + """ + self.registry.add_enumerable_node(node, figtype, title_getter, override=override) + self.add_node(node, override=override, **kwds) + + @property + def enumerable_nodes(self): + # type: () -> Dict[nodes.Node, Tuple[unicode, TitleGetter]] + warnings.warn('app.enumerable_nodes() is deprecated. ' + 'Use app.get_domain("std").enumerable_nodes instead.', + RemovedInSphinx30Warning) + return self.registry.enumerable_nodes + + def add_directive(self, name, obj, content=None, arguments=None, override=False, **options): # NOQA + # type: (unicode, Any, bool, Tuple[int, int, bool], bool, Any) -> None + """Register a Docutils directive. + + *name* must be the prospective directive name. There are two possible + ways to write a directive: + + - In the docutils 0.4 style, *obj* is the directive function. + *content*, *arguments* and *options* are set as attributes on the + function and determine whether the directive has content, arguments + and options, respectively. **This style is deprecated.** + + - In the docutils 0.5 style, *obj* is the directive class. + It must already have attributes named *has_content*, + *required_arguments*, *optional_arguments*, + *final_argument_whitespace* and *option_spec* that correspond to the + options for the function way. See `the Docutils docs + <http://docutils.sourceforge.net/docs/howto/rst-directives.html>`_ + for details. + + The directive class must inherit from the class + ``docutils.parsers.rst.Directive``. + + For example, the (already existing) :rst:dir:`literalinclude` directive + would be added like this: + + .. code-block:: python + + from docutils.parsers.rst import Directive, directives + + class LiteralIncludeDirective(Directive): + has_content = True + required_arguments = 1 + optional_arguments = 0 + final_argument_whitespace = True + option_spec = { + 'class': directives.class_option, + 'name': directives.unchanged, + } + + def run(self): + ... + + add_directive('literalinclude', LiteralIncludeDirective) + + .. versionchanged:: 0.6 + Docutils 0.5-style directive classes are now supported. + .. deprecated:: 1.8 + Docutils 0.4-style (function based) directives support is deprecated. + .. versionchanged:: 1.8 + Add *override* keyword. + """ logger.debug('[app] adding directive: %r', (name, obj, content, arguments, options)) - if name in directives._directives: + if name in directives._directives and not override: logger.warning(__('while setting up extension %s: directive %r is ' 'already registered, it will be overridden'), self._setting_up_extension[-1], name, type='app', subtype='add_directive') - directive = directive_helper(obj, content, arguments, **options) - directives.register_directive(name, directive) - def add_role(self, name, role): - # type: (unicode, Any) -> None + if not isclass(obj) or not issubclass(obj, Directive): + directive = directive_helper(obj, content, arguments, **options) + directives.register_directive(name, directive) + else: + directives.register_directive(name, obj) + + def add_role(self, name, role, override=False): + # type: (unicode, Any, bool) -> None + """Register a Docutils role. + + *name* must be the role name that occurs in the source, *role* the role + function. Refer to the `Docutils documentation + <http://docutils.sourceforge.net/docs/howto/rst-roles.html>`_ for + more information. + + .. versionchanged:: 1.8 + Add *override* keyword. + """ logger.debug('[app] adding role: %r', (name, role)) - if name in roles._roles: + if name in roles._roles and not override: logger.warning(__('while setting up extension %s: role %r is ' 'already registered, it will be overridden'), self._setting_up_extension[-1], name, type='app', subtype='add_role') roles.register_local_role(name, role) - def add_generic_role(self, name, nodeclass): - # type: (unicode, Any) -> None - # don't use roles.register_generic_role because it uses - # register_canonical_role + def add_generic_role(self, name, nodeclass, override=False): + # type: (unicode, Any, bool) -> None + """Register a generic Docutils role. + + Register a Docutils role that does nothing but wrap its contents in the + node given by *nodeclass*. + + .. versionadded:: 0.6 + .. versionchanged:: 1.8 + Add *override* keyword. + """ + # Don't use ``roles.register_generic_role`` because it uses + # ``register_canonical_role``. logger.debug('[app] adding generic role: %r', (name, nodeclass)) - if name in roles._roles: + if name in roles._roles and not override: logger.warning(__('while setting up extension %s: role %r is ' 'already registered, it will be overridden'), self._setting_up_extension[-1], name, @@ -556,39 +773,153 @@ class Sphinx(object): role = roles.GenericRole(name, nodeclass) roles.register_local_role(name, role) - def add_domain(self, domain): - # type: (Type[Domain]) -> None - self.registry.add_domain(domain) + def add_domain(self, domain, override=False): + # type: (Type[Domain], bool) -> None + """Register a domain. + + Make the given *domain* (which must be a class; more precisely, a + subclass of :class:`~sphinx.domains.Domain`) known to Sphinx. + + .. versionadded:: 1.0 + .. versionchanged:: 1.8 + Add *override* keyword. + """ + self.registry.add_domain(domain, override=override) def override_domain(self, domain): # type: (Type[Domain]) -> None - self.registry.override_domain(domain) + """Override a registered domain. + + Make the given *domain* class known to Sphinx, assuming that there is + already a domain with its ``.name``. The new domain must be a subclass + of the existing one. - def add_directive_to_domain(self, domain, name, obj, - has_content=None, argument_spec=None, **option_spec): - # type: (unicode, unicode, Any, bool, Any, Any) -> None + .. versionadded:: 1.0 + .. deprecated:: 1.8 + Integrated to :meth:`add_domain`. + """ + warnings.warn('app.override_domain() is deprecated. ' + 'Use app.add_domain() with override option instead.', + RemovedInSphinx30Warning) + self.registry.add_domain(domain, override=True) + + def add_directive_to_domain(self, domain, name, obj, has_content=None, argument_spec=None, + override=False, **option_spec): + # type: (unicode, unicode, Any, bool, Any, bool, Any) -> None + """Register a Docutils directive in a domain. + + Like :meth:`add_directive`, but the directive is added to the domain + named *domain*. + + .. versionadded:: 1.0 + .. versionchanged:: 1.8 + Add *override* keyword. + """ self.registry.add_directive_to_domain(domain, name, obj, - has_content, argument_spec, **option_spec) + has_content, argument_spec, override=override, + **option_spec) + + def add_role_to_domain(self, domain, name, role, override=False): + # type: (unicode, unicode, Union[RoleFunction, XRefRole], bool) -> None + """Register a Docutils role in a domain. + + Like :meth:`add_role`, but the role is added to the domain named + *domain*. + + .. versionadded:: 1.0 + .. versionchanged:: 1.8 + Add *override* keyword. + """ + self.registry.add_role_to_domain(domain, name, role, override=override) + + def add_index_to_domain(self, domain, index, override=False): + # type: (unicode, Type[Index], bool) -> None + """Register a custom index for a domain. - def add_role_to_domain(self, domain, name, role): - # type: (unicode, unicode, Union[RoleFunction, XRefRole]) -> None - self.registry.add_role_to_domain(domain, name, role) + Add a custom *index* class to the domain named *domain*. *index* must + be a subclass of :class:`~sphinx.domains.Index`. - def add_index_to_domain(self, domain, index): - # type: (unicode, Type[Index]) -> None + .. versionadded:: 1.0 + .. versionchanged:: 1.8 + Add *override* keyword. + """ self.registry.add_index_to_domain(domain, index) def add_object_type(self, directivename, rolename, indextemplate='', parse_node=None, ref_nodeclass=None, objname='', - doc_field_types=[]): - # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List) -> None + doc_field_types=[], override=False): + # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List, bool) -> None + """Register a new object type. + + This method is a very convenient way to add a new :term:`object` type + that can be cross-referenced. It will do this: + + - Create a new directive (called *directivename*) for documenting an + object. It will automatically add index entries if *indextemplate* + is nonempty; if given, it must contain exactly one instance of + ``%s``. See the example below for how the template will be + interpreted. * Create a new role (called *rolename*) to + cross-reference to these object descriptions. + - If you provide *parse_node*, it must be a function that takes a + string and a docutils node, and it must populate the node with + children parsed from the string. It must then return the name of the + item to be used in cross-referencing and index entries. See the + :file:`conf.py` file in the source for this documentation for an + example. + - The *objname* (if not given, will default to *directivename*) names + the type of object. It is used when listing objects, e.g. in search + results. + + For example, if you have this call in a custom Sphinx extension:: + + app.add_object_type('directive', 'dir', 'pair: %s; directive') + + you can use this markup in your documents:: + + .. rst:directive:: function + + Document a function. + + <...> + + See also the :rst:dir:`function` directive. + + For the directive, an index entry will be generated as if you had prepended :: + + .. index:: pair: function; directive + + The reference node will be of class ``literal`` (so it will be rendered + in a proportional font, as appropriate for code) unless you give the + *ref_nodeclass* argument, which must be a docutils node class. Most + useful are ``docutils.nodes.emphasis`` or ``docutils.nodes.strong`` -- + you can also use ``docutils.nodes.generated`` if you want no further + text decoration. If the text should be treated as literal (e.g. no + smart quote replacement), but not have typewriter styling, use + ``sphinx.addnodes.literal_emphasis`` or + ``sphinx.addnodes.literal_strong``. + + For the role content, you have the same syntactical possibilities as + for standard Sphinx roles (see :ref:`xref-syntax`). + + This method is also available under the deprecated alias + :meth:`add_description_unit`. + + .. versionchanged:: 1.8 + Add *override* keyword. + """ self.registry.add_object_type(directivename, rolename, indextemplate, parse_node, - ref_nodeclass, objname, doc_field_types) + ref_nodeclass, objname, doc_field_types, + override=override) def add_description_unit(self, directivename, rolename, indextemplate='', parse_node=None, ref_nodeclass=None, objname='', doc_field_types=[]): # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List) -> None + """Deprecated alias for :meth:`add_object_type`. + + .. deprecated:: 1.6 + Use :meth:`add_object_type` instead. + """ warnings.warn('app.add_description_unit() is now deprecated. ' 'Use app.add_object_type() instead.', RemovedInSphinx20Warning) @@ -596,21 +927,96 @@ class Sphinx(object): ref_nodeclass, objname, doc_field_types) def add_crossref_type(self, directivename, rolename, indextemplate='', - ref_nodeclass=None, objname=''): - # type: (unicode, unicode, unicode, nodes.Node, unicode) -> None + ref_nodeclass=None, objname='', override=False): + # type: (unicode, unicode, unicode, nodes.Node, unicode, bool) -> None + """Register a new crossref object type. + + This method is very similar to :meth:`add_object_type` except that the + directive it generates must be empty, and will produce no output. + + That means that you can add semantic targets to your sources, and refer + to them using custom roles instead of generic ones (like + :rst:role:`ref`). Example call:: + + app.add_crossref_type('topic', 'topic', 'single: %s', + docutils.nodes.emphasis) + + Example usage:: + + .. topic:: application API + + The application API + ------------------- + + Some random text here. + + See also :topic:`this section <application API>`. + + (Of course, the element following the ``topic`` directive needn't be a + section.) + + .. versionchanged:: 1.8 + Add *override* keyword. + """ self.registry.add_crossref_type(directivename, rolename, - indextemplate, ref_nodeclass, objname) + indextemplate, ref_nodeclass, objname, + override=override) def add_transform(self, transform): # type: (Type[Transform]) -> None + """Register a Docutils transform to be applied after parsing. + + Add the standard docutils :class:`Transform` subclass *transform* to + the list of transforms that are applied after Sphinx parses a reST + document. + + .. list-table:: priority range categories for Sphinx transforms + + * - Priority + - Main purpose in Sphinx + * - 0-99 + - Fix invalid nodes by docutils. Translate a doctree. + * - 100-299 + - Preparation + * - 300-399 + - early + * - 400-699 + - main + * - 700-799 + - Post processing. Deadline to modify text and referencing. + * - 800-899 + - Collect referencing and referenced nodes. Domain processing. + * - 900-999 + - Finalize and clean up. + + refs: `Transform Priority Range Categories`__ + + __ http://docutils.sourceforge.net/docs/ref/transforms.html#transform-priority-range-categories + """ # NOQA self.registry.add_transform(transform) def add_post_transform(self, transform): # type: (Type[Transform]) -> None + """Register a Docutils transform to be applied before writing. + + Add the standard docutils :class:`Transform` subclass *transform* to + the list of transforms that are applied before Sphinx writes a + document. + """ self.registry.add_post_transform(transform) def add_javascript(self, filename): # type: (unicode) -> None + """Register a JavaScript file to include in the HTML output. + + Add *filename* to the list of JavaScript files that the default HTML + template will include. The filename must be relative to the HTML + static path, see :confval:`the docs for the config value + <html_static_path>`. A full URI with scheme, like + ``http://example.org/foo.js``, is also supported. + + .. versionadded:: 0.5 + """ logger.debug('[app] adding javascript: %r', filename) from sphinx.builders.html import StandaloneHTMLBuilder if '://' in filename: @@ -619,27 +1025,90 @@ class Sphinx(object): StandaloneHTMLBuilder.script_files.append( posixpath.join('_static', filename)) + def add_css_file(self, filename, **kwargs): + # type: (unicode, **unicode) -> None + """Register a stylesheet to include in the HTML output. + + Add *filename* to the list of CSS files that the default HTML template + will include. The filename must be relative to the HTML static path, + or a full URI with scheme. The keyword arguments are also accepted for + attributes of ``<link>`` tag. + + Example:: + + app.add_css_file('custom.css') + # => <link rel="stylesheet" href="_static/custom.css" type="text/css" /> + + app.add_css_file('print.css', media='print') + # => <link rel="stylesheet" href="_static/print.css" + # type="text/css" media="print" /> + + app.add_css_file('fancy.css', rel='alternate stylesheet', title='fancy') + # => <link rel="alternate stylesheet" href="_static/fancy.css" + # type="text/css" title="fancy" /> + + .. versionadded:: 1.0 + + .. versionchanged:: 1.6 + Optional ``alternate`` and/or ``title`` attributes can be supplied + with the *alternate* (of boolean type) and *title* (a string) + arguments. The default is no title and *alternate* = ``False``. For + more information, refer to the `documentation + <https://mdn.io/Web/CSS/Alternative_style_sheets>`__. + + .. versionchanged:: 1.8 + Renamed from ``app.add_stylesheet()``. + And it allows keyword arguments as attributes of link tag. + """ + logger.debug('[app] adding stylesheet: %r', filename) + self.registry.add_css_files(filename, **kwargs) + def add_stylesheet(self, filename, alternate=False, title=None): # type: (unicode, bool, unicode) -> None - logger.debug('[app] adding stylesheet: %r', filename) - from sphinx.builders.html import StandaloneHTMLBuilder, Stylesheet - if '://' not in filename: - filename = posixpath.join('_static', filename) + """An alias of :meth:`add_css_file`.""" + warnings.warn('The app.add_stylesheet() is deprecated. ' + 'Please use app.add_css_file() instead.', + RemovedInSphinx40Warning) + + attributes = {} # type: Dict[unicode, unicode] if alternate: - rel = u'alternate stylesheet' + attributes['rel'] = 'alternate stylesheet' else: - rel = u'stylesheet' - css = Stylesheet(filename, title, rel) # type: ignore - StandaloneHTMLBuilder.css_files.append(css) + attributes['rel'] = 'stylesheet' + + if title: + attributes['title'] = title + + self.add_css_file(filename, **attributes) def add_latex_package(self, packagename, options=None): # type: (unicode, unicode) -> None - logger.debug('[app] adding latex package: %r', packagename) - if hasattr(self.builder, 'usepackages'): # only for LaTeX builder - self.builder.usepackages.append((packagename, options)) # type: ignore + r"""Register a package to include in the LaTeX source code. + + Add *packagename* to the list of packages that LaTeX source code will + include. If you provide *options*, it will be taken to `\usepackage` + declaration. + + .. code-block:: python + + app.add_latex_package('mypackage') + # => \usepackage{mypackage} + app.add_latex_package('mypackage', 'foo,bar') + # => \usepackage[foo,bar]{mypackage} + + .. versionadded:: 1.3 + """ + self.registry.add_latex_package(packagename, options) def add_lexer(self, alias, lexer): # type: (unicode, Any) -> None + """Register a new lexer for source code. + + Use *lexer*, which must be an instance of a Pygments lexer class, to + highlight code blocks with the given language *alias*. + + .. versionadded:: 0.6 + """ logger.debug('[app] adding lexer: %r', (alias, lexer)) from sphinx.highlighting import lexers if lexers is None: @@ -648,6 +1117,18 @@ class Sphinx(object): def add_autodocumenter(self, cls): # type: (Any) -> None + """Register a new documenter class for the autodoc extension. + + Add *cls* as a new documenter class for the :mod:`sphinx.ext.autodoc` + extension. It must be a subclass of + :class:`sphinx.ext.autodoc.Documenter`. This allows to auto-document + new types of objects. See the source of the autodoc module for + examples on how to subclass :class:`Documenter`. + + .. todo:: Add real docs for Documenter and subclassing + + .. versionadded:: 0.6 + """ logger.debug('[app] adding autodocumenter: %r', cls) from sphinx.ext.autodoc.directive import AutodocDirective self.registry.add_documenter(cls.objtype, cls) @@ -655,30 +1136,96 @@ class Sphinx(object): def add_autodoc_attrgetter(self, typ, getter): # type: (Type, Callable[[Any, unicode, Any], Any]) -> None + """Register a new ``getattr``-like function for the autodoc extension. + + Add *getter*, which must be a function with an interface compatible to + the :func:`getattr` builtin, as the autodoc attribute getter for + objects that are instances of *typ*. All cases where autodoc needs to + get an attribute of a type are then handled by this function instead of + :func:`getattr`. + + .. versionadded:: 0.6 + """ logger.debug('[app] adding autodoc attrgetter: %r', (typ, getter)) self.registry.add_autodoc_attrgetter(typ, getter) def add_search_language(self, cls): # type: (Any) -> None + """Register a new language for the HTML search index. + + Add *cls*, which must be a subclass of + :class:`sphinx.search.SearchLanguage`, as a support language for + building the HTML full-text search index. The class must have a *lang* + attribute that indicates the language it should be used for. See + :confval:`html_search_language`. + + .. versionadded:: 1.1 + """ logger.debug('[app] adding search language: %r', cls) from sphinx.search import languages, SearchLanguage assert issubclass(cls, SearchLanguage) languages[cls.lang] = cls - def add_source_parser(self, suffix, parser): - # type: (unicode, Parser) -> None - self.registry.add_source_parser(suffix, parser) + def add_source_suffix(self, suffix, filetype, override=False): + # type: (unicode, unicode, bool) -> None + """Register a suffix of source files. + + Same as :confval:`source_suffix`. The users can override this + using the setting. + + .. versionadded:: 1.8 + """ + self.registry.add_source_suffix(suffix, filetype, override=override) + + def add_source_parser(self, *args, **kwargs): + # type: (Any, Any) -> None + """Register a parser class. + + .. versionadded:: 1.4 + .. versionchanged:: 1.8 + *suffix* argument is deprecated. It only accepts *parser* argument. + Use :meth:`add_source_suffix` API to register suffix instead. + .. versionchanged:: 1.8 + Add *override* keyword. + """ + self.registry.add_source_parser(*args, **kwargs) def add_env_collector(self, collector): # type: (Type[EnvironmentCollector]) -> None + """Register an environment collector class. + + Refer to :ref:`collector-api`. + + .. versionadded:: 1.6 + """ logger.debug('[app] adding environment collector: %r', collector) collector().enable(self) def add_html_theme(self, name, theme_path): # type: (unicode, unicode) -> None + """Register a HTML Theme. + + The *name* is a name of theme, and *path* is a full path to the theme + (refs: :ref:`distribute-your-theme`). + + .. versionadded:: 1.6 + """ logger.debug('[app] adding HTML theme: %r, %r', name, theme_path) self.html_themes[name] = theme_path + def add_message_catalog(self, catalog, locale_dir): + # type: (unicode, unicode) -> None + """Register a message catalog. + + The *catalog* is a name of catalog, and *locale_dir* is a base path + of message catalog. For more details, see + :func:`sphinx.locale.get_translation()`. + + .. versionadded:: 1.8 + """ + locale.init([locale_dir], self.config.language, catalog) + locale.init_console(locale_dir, catalog) + # ---- other methods ------------------------------------------------- def is_parallel_allowed(self, typ): # type: (unicode) -> bool @@ -705,7 +1252,7 @@ class Sphinx(object): allowed = getattr(ext, attrname, None) if allowed is None: logger.warning(message, ext.name) - logger.warning('doing serial %s', typ) + logger.warning(__('doing serial %s'), typ) return False elif not allowed: return False diff --git a/sphinx/builders/__init__.py b/sphinx/builders/__init__.py index a587f11e0..b44139a55 100644 --- a/sphinx/builders/__init__.py +++ b/sphinx/builders/__init__.py @@ -9,15 +9,22 @@ :license: BSD, see LICENSE for details. """ +import time import warnings from os import path from docutils import nodes +from six.moves import cPickle as pickle from sphinx.deprecation import RemovedInSphinx20Warning from sphinx.environment.adapters.asset import ImageAdapter -from sphinx.util import i18n, logging, status_iterator +from sphinx.errors import SphinxError +from sphinx.io import read_doc +from sphinx.locale import __ +from sphinx.util import i18n, import_object, logging, rst, status_iterator +from sphinx.util.build_phase import BuildPhase from sphinx.util.console import bold # type: ignore +from sphinx.util.docutils import sphinx_domains from sphinx.util.i18n import find_catalog from sphinx.util.osutil import SEP, ensuredir, relative_uri, relpath from sphinx.util.parallel import ParallelTasks, SerialTasks, make_chunks, \ @@ -59,8 +66,8 @@ class Builder(object): #: ``project`` epilog = '' # type: unicode - # default translator class for the builder. This will be overrided by - # ``app.set_translator()``. + #: default translator class for the builder. This can be overrided by + #: :py:meth:`app.set_translator()`. default_translator_class = None # type: nodes.NodeVisitor # doctree versioning method versioning_method = 'none' # type: unicode @@ -73,7 +80,9 @@ class Builder(object): #: The list of MIME types of image formats supported by the builder. #: Image files are searched in the order in which they appear here. supported_image_types = [] # type: List[unicode] + #: The builder supports remote images or not. supported_remote_images = False + #: The builder supports data URIs or not. supported_data_uri_images = False def __init__(self, app): @@ -125,9 +134,7 @@ class Builder(object): This method returns an instance of ``default_translator_class`` by default. Users can replace the translator class with ``app.set_translator()`` API. """ - translator_class = self.app.registry.get_translator_class(self) - assert translator_class, "translator not found for %s" % self.__class__.__name__ - return translator_class(*args) + return self.app.registry.create_translator(self, *args) @property def translator_class(self): @@ -157,8 +164,8 @@ class Builder(object): # type: () -> None """Return the template bridge configured.""" if self.config.template_bridge: - self.templates = self.app.import_object( - self.config.template_bridge, 'template_bridge setting')() + self.templates = import_object(self.config.template_bridge, + 'template_bridge setting')() else: from sphinx.jinja2glue import BuiltinTemplateLoader self.templates = BuiltinTemplateLoader() @@ -214,10 +221,11 @@ class Builder(object): mimetypes = sorted(node['candidates']) image_uri = images.get_original_image_uri(node['uri']) if mimetypes: - logger.warning('a suitable image for %s builder not found: %s (%s)', + logger.warning(__('a suitable image for %s builder not found: ' + '%s (%s)'), self.name, mimetypes, image_uri, location=node) else: - logger.warning('a suitable image for %s builder not found: %s', + logger.warning(__('a suitable image for %s builder not found: %s'), self.name, image_uri, location=node) continue node['uri'] = candidate @@ -239,8 +247,8 @@ class Builder(object): # type: (CatalogInfo) -> unicode return relpath(cat.mo_path, self.env.srcdir).replace(path.sep, SEP) - logger.info(bold('building [mo]: ') + message) - for catalog in status_iterator(catalogs, 'writing output... ', "darkgreen", + logger.info(bold(__('building [mo]: ')) + message) + for catalog in status_iterator(catalogs, __('writing output... '), "darkgreen", len(catalogs), self.app.verbosity, stringify_func=cat2relpath): catalog.write_mo(self.config.language) @@ -253,7 +261,7 @@ class Builder(object): charset=self.config.source_encoding, gettext_compact=self.config.gettext_compact, force_all=True) - message = 'all of %d po files' % len(catalogs) + message = __('all of %d po files') % len(catalogs) self.compile_catalogs(catalogs, message) def compile_specific_catalogs(self, specified_files): @@ -274,7 +282,7 @@ class Builder(object): domains=list(specified_domains), charset=self.config.source_encoding, gettext_compact=self.config.gettext_compact) - message = 'targets for %d po files that are specified' % len(catalogs) + message = __('targets for %d po files that are specified') % len(catalogs) self.compile_catalogs(catalogs, message) def compile_update_catalogs(self): @@ -284,7 +292,7 @@ class Builder(object): self.config.language, charset=self.config.source_encoding, gettext_compact=self.config.gettext_compact) - message = 'targets for %d po files that are out of date' % len(catalogs) + message = __('targets for %d po files that are out of date') % len(catalogs) self.compile_catalogs(catalogs, message) # build methods @@ -292,7 +300,7 @@ class Builder(object): def build_all(self): # type: () -> None """Build all source files.""" - self.build(None, summary='all source files', method='all') + self.build(None, summary=__('all source files'), method='all') def build_specific(self, filenames): # type: (List[unicode]) -> None @@ -306,13 +314,13 @@ class Builder(object): for filename in filenames: filename = path.normpath(path.abspath(filename)) if not filename.startswith(self.srcdir): - logger.warning('file %r given on command line is not under the ' - 'source directory, ignoring', filename) + logger.warning(__('file %r given on command line is not under the ' + 'source directory, ignoring'), filename) continue if not (path.isfile(filename) or any(path.isfile(filename + suffix) for suffix in suffixes)): - logger.warning('file %r given on command line does not exist, ' - 'ignoring', filename) + logger.warning(__('file %r given on command line does not exist, ' + 'ignoring'), filename) continue filename = filename[dirlen:] for suffix in suffixes: @@ -322,8 +330,7 @@ class Builder(object): filename = filename.replace(path.sep, SEP) to_write.append(filename) self.build(to_write, method='specific', - summary='%d source files given on command ' - 'line' % len(to_write)) + summary=__('%d source files given on command line') % len(to_write)) def build_update(self): # type: () -> None @@ -334,8 +341,8 @@ class Builder(object): else: to_build = list(to_build) self.build(to_build, - summary='targets for %d source files that are ' - 'out of date' % len(to_build)) + summary=__('targets for %d source files that are out of date') % + len(to_build)) def build(self, docnames, summary=None, method='update'): # type: (Iterable[unicode], unicode, unicode) -> None @@ -344,38 +351,42 @@ class Builder(object): First updates the environment, and then calls :meth:`write`. """ if summary: - logger.info(bold('building [%s]' % self.name) + ': ' + summary) + logger.info(bold(__('building [%s]') % self.name) + ': ' + summary) # while reading, collect all warnings from docutils with logging.pending_warnings(): - updated_docnames = set(self.env.update(self.config, self.srcdir, self.doctreedir)) + updated_docnames = set(self.read()) doccount = len(updated_docnames) - logger.info(bold('looking for now-outdated files... '), nonl=1) + logger.info(bold(__('looking for now-outdated files... ')), nonl=1) for docname in self.env.check_dependents(self.app, updated_docnames): updated_docnames.add(docname) outdated = len(updated_docnames) - doccount if outdated: - logger.info('%d found', outdated) + logger.info(__('%d found'), outdated) else: - logger.info('none found') + logger.info(__('none found')) if updated_docnames: # save the environment from sphinx.application import ENV_PICKLE_FILENAME - logger.info(bold('pickling environment... '), nonl=True) - self.env.topickle(path.join(self.doctreedir, ENV_PICKLE_FILENAME)) - logger.info('done') + logger.info(bold(__('pickling environment... ')), nonl=True) + with open(path.join(self.doctreedir, ENV_PICKLE_FILENAME), 'wb') as f: + pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL) + logger.info(__('done')) # global actions - logger.info(bold('checking consistency... '), nonl=True) + self.app.phase = BuildPhase.CONSISTENCY_CHECK + logger.info(bold(__('checking consistency... ')), nonl=True) self.env.check_consistency() - logger.info('done') + logger.info(__('done')) else: if method == 'update' and not docnames: - logger.info(bold('no targets are out of date.')) + logger.info(bold(__('no targets are out of date.'))) return + self.app.phase = BuildPhase.RESOLVING + # filter "docnames" (list of outdated files) by the updated # found_docs of the environment; this will remove docs that # have since been removed @@ -404,6 +415,147 @@ class Builder(object): # wait for all tasks self.finish_tasks.join() + def read(self): + # type: () -> List[unicode] + """(Re-)read all files new or changed since last update. + + Store all environment docnames in the canonical format (ie using SEP as + a separator in place of os.path.sep). + """ + updated, reason = self.env.update_config(self.config, self.srcdir, self.doctreedir) + + logger.info(bold('updating environment: '), nonl=True) + + self.env.find_files(self.config, self) + added, changed, removed = self.env.get_outdated_files(updated) + + # allow user intervention as well + for docs in self.app.emit('env-get-outdated', self, added, changed, removed): + changed.update(set(docs) & self.env.found_docs) + + # if files were added or removed, all documents with globbed toctrees + # must be reread + if added or removed: + # ... but not those that already were removed + changed.update(self.env.glob_toctrees & self.env.found_docs) + + if changed: + logger.info('[%s] ', reason, nonl=True) + logger.info('%s added, %s changed, %s removed', + len(added), len(changed), len(removed)) + + # clear all files no longer present + for docname in removed: + self.app.emit('env-purge-doc', self.env, docname) + self.env.clear_doc(docname) + + # read all new and changed files + docnames = sorted(added | changed) + # allow changing and reordering the list of docs to read + self.app.emit('env-before-read-docs', self.env, docnames) + + # check if we should do parallel or serial read + if parallel_available and len(docnames) > 5 and self.app.parallel > 1: + par_ok = self.app.is_parallel_allowed('read') + else: + par_ok = False + + if par_ok: + self._read_parallel(docnames, nproc=self.app.parallel) + else: + self._read_serial(docnames) + + if self.config.master_doc not in self.env.all_docs: + raise SphinxError('master file %s not found' % + self.env.doc2path(self.config.master_doc)) + + for retval in self.app.emit('env-updated', self.env): + if retval is not None: + docnames.extend(retval) + + return sorted(docnames) + + def _read_serial(self, docnames): + # type: (List[unicode]) -> None + for docname in status_iterator(docnames, 'reading sources... ', "purple", + len(docnames), self.app.verbosity): + # remove all inventory entries for that file + self.app.emit('env-purge-doc', self.env, docname) + self.env.clear_doc(docname) + self.read_doc(docname) + + def _read_parallel(self, docnames, nproc): + # type: (List[unicode], int) -> None + # clear all outdated docs at once + for docname in docnames: + self.app.emit('env-purge-doc', self.env, docname) + self.env.clear_doc(docname) + + def read_process(docs): + # type: (List[unicode]) -> bytes + self.env.app = self.app + for docname in docs: + self.read_doc(docname) + # allow pickling self to send it back + return pickle.dumps(self.env, pickle.HIGHEST_PROTOCOL) + + def merge(docs, otherenv): + # type: (List[unicode], bytes) -> None + env = pickle.loads(otherenv) + self.env.merge_info_from(docs, env, self.app) + + tasks = ParallelTasks(nproc) + chunks = make_chunks(docnames, nproc) + + for chunk in status_iterator(chunks, 'reading sources... ', "purple", + len(chunks), self.app.verbosity): + tasks.add_task(read_process, chunk, merge) + + # make sure all threads have finished + logger.info(bold('waiting for workers...')) + tasks.join() + + def read_doc(self, docname): + # type: (unicode) -> None + """Parse a file and add/update inventory entries for the doctree.""" + self.env.prepare_settings(docname) + + # Add confdir/docutils.conf to dependencies list if exists + docutilsconf = path.join(self.confdir, 'docutils.conf') + if path.isfile(docutilsconf): + self.env.note_dependency(docutilsconf) + + with sphinx_domains(self.env), rst.default_role(docname, self.config.default_role): + doctree = read_doc(self.app, self.env, self.env.doc2path(docname)) + + # store time of reading, for outdated files detection + # (Some filesystems have coarse timestamp resolution; + # therefore time.time() can be older than filesystem's timestamp. + # For example, FAT32 has 2sec timestamp resolution.) + self.env.all_docs[docname] = max(time.time(), + path.getmtime(self.env.doc2path(docname))) + + # cleanup + self.env.temp_data.clear() + self.env.ref_context.clear() + + self.write_doctree(docname, doctree) + + def write_doctree(self, docname, doctree): + # type: (unicode, nodes.Node) -> None + """Write the doctree to a file.""" + # make it picklable + doctree.reporter = None + doctree.transformer = None + doctree.settings.warning_stream = None + doctree.settings.env = None + doctree.settings.record_dependencies = None + + doctree_filename = self.env.doc2path(docname, self.env.doctreedir, '.doctree') + ensuredir(path.dirname(doctree_filename)) + with open(doctree_filename, 'wb') as f: + pickle.dump(doctree, f, pickle.HIGHEST_PROTOCOL) + def write(self, build_docnames, updated_docnames, method='update'): # type: (Iterable[unicode], Sequence[unicode], unicode) -> None if build_docnames is None or build_docnames == ['__all__']: @@ -414,7 +566,7 @@ class Builder(object): docnames = set(build_docnames) | set(updated_docnames) else: docnames = set(build_docnames) - logger.debug('docnames to write: %s', ', '.join(sorted(docnames))) + logger.debug(__('docnames to write: %s'), ', '.join(sorted(docnames))) # add all toctree-containing files that may have changed for docname in list(docnames): @@ -423,9 +575,9 @@ class Builder(object): docnames.add(tocdocname) docnames.add(self.config.master_doc) - logger.info(bold('preparing documents... '), nonl=True) + logger.info(bold(__('preparing documents... ')), nonl=True) self.prepare_writing(docnames) - logger.info('done') + logger.info(__('done')) if self.parallel_ok: # number of subprocesses is parallel-1 because the main process @@ -438,9 +590,11 @@ class Builder(object): def _write_serial(self, docnames): # type: (Sequence[unicode]) -> None with logging.pending_warnings(): - for docname in status_iterator(docnames, 'writing output... ', "darkgreen", + for docname in status_iterator(docnames, __('writing output... '), "darkgreen", len(docnames), self.app.verbosity): + self.app.phase = BuildPhase.RESOLVING doctree = self.env.get_and_resolve_doctree(docname, self) + self.app.phase = BuildPhase.WRITING self.write_doc_serialized(docname, doctree) self.write_doc(docname, doctree) @@ -448,19 +602,23 @@ class Builder(object): # type: (Sequence[unicode], int) -> None def write_process(docs): # type: (List[Tuple[unicode, nodes.Node]]) -> None + self.app.phase = BuildPhase.WRITING for docname, doctree in docs: self.write_doc(docname, doctree) # warm up caches/compile templates using the first document firstname, docnames = docnames[0], docnames[1:] + self.app.phase = BuildPhase.RESOLVING doctree = self.env.get_and_resolve_doctree(firstname, self) + self.app.phase = BuildPhase.WRITING self.write_doc_serialized(firstname, doctree) self.write_doc(firstname, doctree) tasks = ParallelTasks(nproc) chunks = make_chunks(docnames, nproc) - for chunk in status_iterator(chunks, 'writing output... ', "darkgreen", + self.app.phase = BuildPhase.RESOLVING + for chunk in status_iterator(chunks, __('writing output... '), "darkgreen", len(chunks), self.app.verbosity): arg = [] for i, docname in enumerate(chunk): @@ -470,7 +628,7 @@ class Builder(object): tasks.add_task(write_process, arg) # make sure all threads have finished - logger.info(bold('waiting for workers...')) + logger.info(bold(__('waiting for workers...'))) tasks.join() def prepare_writing(self, docnames): diff --git a/sphinx/builders/_epub_base.py b/sphinx/builders/_epub_base.py index 04812c56f..fe285403b 100644 --- a/sphinx/builders/_epub_base.py +++ b/sphinx/builders/_epub_base.py @@ -20,6 +20,7 @@ from docutils.utils import smartquotes from sphinx import addnodes from sphinx.builders.html import BuildInfo, StandaloneHTMLBuilder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util import status_iterator from sphinx.util.fileutil import copy_asset_file @@ -401,13 +402,13 @@ class EpubBuilder(StandaloneHTMLBuilder): img = Image.open(path.join(self.srcdir, src)) except IOError: if not self.is_vector_graphics(src): - logger.warning('cannot read image file %r: copying it instead', + logger.warning(__('cannot read image file %r: copying it instead'), path.join(self.srcdir, src)) try: copyfile(path.join(self.srcdir, src), path.join(self.outdir, self.imagedir, dest)) except (IOError, OSError) as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) continue if self.config.epub_fix_images: @@ -423,7 +424,7 @@ class EpubBuilder(StandaloneHTMLBuilder): try: img.save(path.join(self.outdir, self.imagedir, dest)) except (IOError, OSError) as err: - logger.warning('cannot write image file %r: %s', + logger.warning(__('cannot write image file %r: %s'), path.join(self.srcdir, src), err) def copy_image_files(self): @@ -434,7 +435,7 @@ class EpubBuilder(StandaloneHTMLBuilder): if self.images: if self.config.epub_fix_images or self.config.epub_max_image_width: if not Image: - logger.warning('PIL not found - copying image files') + logger.warning(__('PIL not found - copying image files')) super(EpubBuilder, self).copy_image_files() else: self.copy_image_files_pil() @@ -464,14 +465,14 @@ class EpubBuilder(StandaloneHTMLBuilder): def build_mimetype(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file mimetype.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) copy_asset_file(path.join(self.template_dir, 'mimetype'), path.join(outdir, outname)) def build_container(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file META-INF/container.xml.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) filename = path.join(outdir, outname) ensuredir(path.dirname(filename)) copy_asset_file(path.join(self.template_dir, 'container.xml'), filename) @@ -501,7 +502,7 @@ class EpubBuilder(StandaloneHTMLBuilder): """Write the metainfo file content.opf It contains bibliographic data, a file list and the spine (the reading order). """ - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) metadata = self.content_metadata() # files @@ -527,7 +528,7 @@ class EpubBuilder(StandaloneHTMLBuilder): # we always have JS and potentially OpenSearch files, don't # always warn about them if ext not in ('.js', '.xml'): - logger.warning('unknown mimetype for %s, ignoring', filename, + logger.warning(__('unknown mimetype for %s, ignoring'), filename, type='epub', subtype='unknown_project_files') continue filename = filename.replace(os.sep, '/') @@ -680,7 +681,7 @@ class EpubBuilder(StandaloneHTMLBuilder): def build_toc(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file toc.ncx.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) if self.config.epub_tocscope == 'default': doctree = self.env.get_and_resolve_doctree(self.config.master_doc, @@ -705,7 +706,7 @@ class EpubBuilder(StandaloneHTMLBuilder): It is a zip file with the mimetype file stored uncompressed as the first entry. """ - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) epub_filename = path.join(outdir, outname) with ZipFile(epub_filename, 'w', ZIP_DEFLATED) as epub: epub.write(path.join(outdir, 'mimetype'), 'mimetype', ZIP_STORED) diff --git a/sphinx/builders/applehelp.py b/sphinx/builders/applehelp.py index d7974696c..79d57210c 100644 --- a/sphinx/builders/applehelp.py +++ b/sphinx/builders/applehelp.py @@ -20,6 +20,7 @@ from os import path, environ from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.config import string_classes from sphinx.errors import SphinxError +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.console import bold # type: ignore from sphinx.util.fileutil import copy_asset @@ -60,11 +61,11 @@ access_page_template = '''\ class AppleHelpIndexerFailed(SphinxError): - category = 'Help indexer failed' + category = __('Help indexer failed') class AppleHelpCodeSigningFailed(SphinxError): - category = 'Code signing failed' + category = __('Code signing failed') class AppleHelpBuilder(StandaloneHTMLBuilder): @@ -73,10 +74,10 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): on the ``hiutil`` command line tool. """ name = 'applehelp' - epilog = ('The help book is in %(outdir)s.\n' - 'Note that won\'t be able to view it unless you put it in ' - '~/Library/Documentation/Help or install it in your application ' - 'bundle.') + epilog = __('The help book is in %(outdir)s.\n' + 'Note that won\'t be able to view it unless you put it in ' + '~/Library/Documentation/Help or install it in your application ' + 'bundle.') # don't copy the reST source copysource = False @@ -100,8 +101,8 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): self.link_suffix = '.html' if self.config.applehelp_bundle_id is None: - raise SphinxError('You must set applehelp_bundle_id before ' - 'building Apple Help output') + raise SphinxError(__('You must set applehelp_bundle_id before ' + 'building Apple Help output')) self.bundle_path = path.join(self.outdir, self.config.applehelp_bundle_name + @@ -124,13 +125,13 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): target_dir = self.outdir if path.isdir(source_dir): - logger.info(bold('copying localized files... '), nonl=True) + logger.info(bold(__('copying localized files... ')), nonl=True) excluded = Matcher(self.config.exclude_patterns + ['**/.*']) copy_asset(source_dir, target_dir, excluded, context=self.globalcontext, renderer=self.templates) - logger.info('done') + logger.info(__('done')) def build_helpbook(self): # type: () -> None @@ -171,36 +172,36 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): if self.config.applehelp_remote_url is not None: info_plist['HPDBookRemoteURL'] = self.config.applehelp_remote_url - logger.info(bold('writing Info.plist... '), nonl=True) + logger.info(bold(__('writing Info.plist... ')), nonl=True) with open(path.join(contents_dir, 'Info.plist'), 'wb') as f: write_plist(info_plist, f) - logger.info('done') + logger.info(__('done')) # Copy the icon, if one is supplied if self.config.applehelp_icon: - logger.info(bold('copying icon... '), nonl=True) + logger.info(bold(__('copying icon... ')), nonl=True) try: copyfile(path.join(self.srcdir, self.config.applehelp_icon), path.join(resources_dir, info_plist['HPDBookIconPath'])) - logger.info('done') + logger.info(__('done')) except Exception as err: - logger.warning('cannot copy icon file %r: %s', + logger.warning(__('cannot copy icon file %r: %s'), path.join(self.srcdir, self.config.applehelp_icon), err) del info_plist['HPDBookIconPath'] # Build the access page - logger.info(bold('building access page...'), nonl=True) + logger.info(bold(__('building access page...')), nonl=True) with codecs.open(path.join(language_dir, '_access.html'), 'w') as f: # type: ignore f.write(access_page_template % { 'toc': htmlescape(toc, quote=True), 'title': htmlescape(self.config.applehelp_title) }) - logger.info('done') + logger.info(__('done')) # Generate the help index - logger.info(bold('generating help index... '), nonl=True) + logger.info(bold(__('generating help index... ')), nonl=True) args = [ self.config.applehelp_indexer_path, @@ -222,9 +223,9 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): args += ['-l', self.config.applehelp_locale] if self.config.applehelp_disable_external_tools: - logger.info('skipping') + logger.info(__('skipping')) - logger.warning('you will need to index this help book with:\n %s', + logger.warning(__('you will need to index this help book with:\n %s'), ' '.join([pipes.quote(arg) for arg in args])) else: try: @@ -237,13 +238,13 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): if p.returncode != 0: raise AppleHelpIndexerFailed(output) else: - logger.info('done') + logger.info(__('done')) except OSError: - raise AppleHelpIndexerFailed('Command not found: %s' % args[0]) + raise AppleHelpIndexerFailed(__('Command not found: %s') % args[0]) # If we've been asked to, sign the bundle if self.config.applehelp_codesign_identity: - logger.info(bold('signing help book... '), nonl=True) + logger.info(bold(__('signing help book... ')), nonl=True) args = [ self.config.applehelp_codesign_path, @@ -256,8 +257,8 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): args.append(self.bundle_path) if self.config.applehelp_disable_external_tools: - logger.info('skipping') - logger.warning('you will need to sign this help book with:\n %s', + logger.info(__('skipping')) + logger.warning(__('you will need to sign this help book with:\n %s'), ' '.join([pipes.quote(arg) for arg in args])) else: try: @@ -270,9 +271,9 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): if p.returncode != 0: raise AppleHelpCodeSigningFailed(output) else: - logger.info('done') + logger.info(__('done')) except OSError: - raise AppleHelpCodeSigningFailed('Command not found: %s' % args[0]) + raise AppleHelpCodeSigningFailed(__('Command not found: %s') % args[0]) def setup(app): diff --git a/sphinx/builders/changes.py b/sphinx/builders/changes.py index e8b6650cb..011aa6ebe 100644 --- a/sphinx/builders/changes.py +++ b/sphinx/builders/changes.py @@ -16,7 +16,7 @@ from six import iteritems from sphinx import package_dir from sphinx.builders import Builder -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.theming import HTMLThemeFactory from sphinx.util import logging from sphinx.util.console import bold # type: ignore @@ -38,7 +38,7 @@ class ChangesBuilder(Builder): Write a summary with all versionadded/changed directives. """ name = 'changes' - epilog = 'The overview file is in %(outdir)s.' + epilog = __('The overview file is in %(outdir)s.') def init(self): # type: () -> None @@ -64,7 +64,7 @@ class ChangesBuilder(Builder): apichanges = [] # type: List[Tuple[unicode, unicode, int]] otherchanges = {} # type: Dict[Tuple[unicode, unicode], List[Tuple[unicode, unicode, int]]] # NOQA if version not in self.env.versionchanges: - logger.info(bold('no changes in version %s.' % version)) + logger.info(bold(__('no changes in version %s.') % version)) return logger.info(bold('writing summary file...')) for type, docname, lineno, module, descname, content in \ @@ -129,14 +129,14 @@ class ChangesBuilder(Builder): break return line - logger.info(bold('copying source files...')) + logger.info(bold(__('copying source files...'))) for docname in self.env.all_docs: with codecs.open(self.env.doc2path(docname), 'r', # type: ignore self.env.config.source_encoding) as f: try: lines = f.readlines() except UnicodeDecodeError: - logger.warning('could not read %r for changelog creation', docname) + logger.warning(__('could not read %r for changelog creation'), docname) continue targetfn = path.join(self.outdir, 'rst', os_path(docname)) + '.html' ensuredir(path.dirname(targetfn)) diff --git a/sphinx/builders/devhelp.py b/sphinx/builders/devhelp.py index 96e06afdc..fc2c0b1c9 100644 --- a/sphinx/builders/devhelp.py +++ b/sphinx/builders/devhelp.py @@ -5,7 +5,7 @@ Build HTML documentation and Devhelp_ support files. - .. _Devhelp: http://live.gnome.org/devhelp + .. _Devhelp: https://wiki.gnome.org/Apps/Devhelp :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. @@ -21,6 +21,7 @@ from docutils import nodes from sphinx import addnodes from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.environment.adapters.indexentries import IndexEntries +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import make_filename @@ -43,10 +44,10 @@ class DevhelpBuilder(StandaloneHTMLBuilder): Builder that also outputs GNOME Devhelp file. """ name = 'devhelp' - epilog = ('To view the help file:\n' - '$ mkdir -p $HOME/.local/share/devhelp/%(project)s\n' - '$ ln -s %(outdir)s $HOME/.local/share/devhelp/%(project)s\n' - '$ devhelp') + epilog = __('To view the help file:\n' + '$ mkdir -p $HOME/.local/share/devhelp/books\n' + '$ ln -s $PWD/%(outdir)s $HOME/.local/share/devhelp/books/%(project)s\n' + '$ devhelp') # don't copy the reST source copysource = False @@ -69,7 +70,7 @@ class DevhelpBuilder(StandaloneHTMLBuilder): def build_devhelp(self, outdir, outname): # type: (unicode, unicode) -> None - logger.info('dumping devhelp index...') + logger.info(__('dumping devhelp index...')) # Basic info root = etree.Element('book', diff --git a/sphinx/builders/dummy.py b/sphinx/builders/dummy.py index 08d99a584..805924290 100644 --- a/sphinx/builders/dummy.py +++ b/sphinx/builders/dummy.py @@ -11,6 +11,7 @@ from sphinx.builders import Builder +from sphinx.locale import __ if False: # For type annotation @@ -21,7 +22,7 @@ if False: class DummyBuilder(Builder): name = 'dummy' - epilog = 'The dummy builder generates no files.' + epilog = __('The dummy builder generates no files.') allow_parallel = True diff --git a/sphinx/builders/epub3.py b/sphinx/builders/epub3.py index 008942a6a..5d3384b46 100644 --- a/sphinx/builders/epub3.py +++ b/sphinx/builders/epub3.py @@ -13,9 +13,12 @@ from collections import namedtuple from os import path +from six import string_types + from sphinx import package_dir from sphinx.builders import _epub_base from sphinx.config import string_classes, ENUM +from sphinx.locale import __ from sphinx.util import logging, xmlname_checker from sphinx.util.fileutil import copy_asset_file from sphinx.util.i18n import format_date @@ -23,9 +26,10 @@ from sphinx.util.osutil import make_filename if False: # For type annotation - from typing import Any, Dict, Iterable, List # NOQA + from typing import Any, Dict, Iterable, List, Tuple # NOQA from docutils import nodes # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA logger = logging.getLogger(__name__) @@ -63,7 +67,7 @@ class Epub3Builder(_epub_base.EpubBuilder): an epub file. """ name = 'epub' - epilog = 'The ePub file is in %(outdir)s.' + epilog = __('The ePub file is in %(outdir)s.') supported_remote_images = False template_dir = path.join(package_dir, 'templates', 'epub3') @@ -85,39 +89,40 @@ class Epub3Builder(_epub_base.EpubBuilder): self.build_epub(self.outdir, self.config.epub_basename + '.epub') def validate_config_value(self): + # type: () -> None # <package> lang attribute, dc:language if not self.app.config.epub_language: - logger.warning('conf value "epub_language" (or "language") ' - 'should not be empty for EPUB3') + logger.warning(__('conf value "epub_language" (or "language") ' + 'should not be empty for EPUB3')) # <package> unique-identifier attribute if not xmlname_checker().match(self.app.config.epub_uid): - logger.warning('conf value "epub_uid" should be XML NAME for EPUB3') + logger.warning(__('conf value "epub_uid" should be XML NAME for EPUB3')) # dc:title if not self.app.config.epub_title: - logger.warning('conf value "epub_title" (or "html_title") ' - 'should not be empty for EPUB3') + logger.warning(__('conf value "epub_title" (or "html_title") ' + 'should not be empty for EPUB3')) # dc:creator if not self.app.config.epub_author: - logger.warning('conf value "epub_author" should not be empty for EPUB3') + logger.warning(__('conf value "epub_author" should not be empty for EPUB3')) # dc:contributor if not self.app.config.epub_contributor: - logger.warning('conf value "epub_contributor" should not be empty for EPUB3') + logger.warning(__('conf value "epub_contributor" should not be empty for EPUB3')) # dc:description if not self.app.config.epub_description: - logger.warning('conf value "epub_description" should not be empty for EPUB3') + logger.warning(__('conf value "epub_description" should not be empty for EPUB3')) # dc:publisher if not self.app.config.epub_publisher: - logger.warning('conf value "epub_publisher" should not be empty for EPUB3') + logger.warning(__('conf value "epub_publisher" should not be empty for EPUB3')) # dc:rights if not self.app.config.epub_copyright: - logger.warning('conf value "epub_copyright" (or "copyright")' - 'should not be empty for EPUB3') + logger.warning(__('conf value "epub_copyright" (or "copyright")' + 'should not be empty for EPUB3')) # dc:identifier if not self.app.config.epub_identifier: - logger.warning('conf value "epub_identifier" should not be empty for EPUB3') + logger.warning(__('conf value "epub_identifier" should not be empty for EPUB3')) # meta ibooks:version if not self.app.config.version: - logger.warning('conf value "version" should not be empty for EPUB3') + logger.warning(__('conf value "version" should not be empty for EPUB3')) def content_metadata(self): # type: () -> Dict @@ -203,7 +208,7 @@ class Epub3Builder(_epub_base.EpubBuilder): def build_navigation_doc(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file nav.xhtml.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) if self.config.epub_tocscope == 'default': doctree = self.env.get_and_resolve_doctree( @@ -224,6 +229,24 @@ class Epub3Builder(_epub_base.EpubBuilder): self.files.append(outname) +def convert_epub_css_files(app, config): + # type: (Sphinx, Config) -> None + """This converts string styled epub_css_files to tuple styled one.""" + epub_css_files = [] # type: List[Tuple[unicode, Dict]] + for entry in config.epub_css_files: + if isinstance(entry, string_types): + epub_css_files.append((entry, {})) + else: + try: + filename, attrs = entry + epub_css_files.append((filename, attrs)) + except Exception: + logger.warning(__('invalid css_file: %r, ignored'), entry) + continue + + config.epub_css_files = epub_css_files # type: ignore + + def setup(app): # type: (Sphinx) -> Dict[unicode, Any] app.add_builder(Epub3Builder) @@ -234,9 +257,9 @@ def setup(app): app.add_config_value('epub_theme', 'epub', 'epub') app.add_config_value('epub_theme_options', {}, 'epub') app.add_config_value('epub_title', lambda self: self.html_title, 'epub') - app.add_config_value('epub_author', 'unknown', 'epub') + app.add_config_value('epub_author', lambda self: self.author, 'epub') app.add_config_value('epub_language', lambda self: self.language or 'en', 'epub') - app.add_config_value('epub_publisher', 'unknown', 'epub') + app.add_config_value('epub_publisher', lambda self: self.author, 'epub') app.add_config_value('epub_copyright', lambda self: self.copyright, 'epub') app.add_config_value('epub_identifier', 'unknown', 'epub') app.add_config_value('epub_scheme', 'unknown', 'epub') @@ -245,6 +268,7 @@ def setup(app): app.add_config_value('epub_guide', (), 'env') app.add_config_value('epub_pre_files', [], 'env') app.add_config_value('epub_post_files', [], 'env') + app.add_config_value('epub_css_files', lambda config: config.html_css_files, 'epub') app.add_config_value('epub_exclude_files', [], 'env') app.add_config_value('epub_tocdepth', 3, 'env') app.add_config_value('epub_tocdup', True, 'env') @@ -258,6 +282,9 @@ def setup(app): app.add_config_value('epub_writing_mode', 'horizontal', 'epub', ENUM('horizontal', 'vertical')) + # event handlers + app.connect('config-inited', convert_epub_css_files) + return { 'version': 'builtin', 'parallel_read_safe': True, diff --git a/sphinx/builders/gettext.py b/sphinx/builders/gettext.py index 05f7d3a81..b43dcfb3b 100644 --- a/sphinx/builders/gettext.py +++ b/sphinx/builders/gettext.py @@ -21,7 +21,8 @@ from uuid import uuid4 from six import iteritems, StringIO from sphinx.builders import Builder -from sphinx.locale import pairindextypes +from sphinx.domains.python import pairindextypes +from sphinx.locale import __ from sphinx.util import split_index_msg, logging, status_iterator from sphinx.util.console import bold # type: ignore from sphinx.util.i18n import find_catalog @@ -192,6 +193,7 @@ ltz = LocalTimeZone() def should_write(filepath, new_content): + # type: (unicode, unicode) -> bool if not path.exists(filepath): return True try: @@ -214,7 +216,7 @@ class MessageCatalogBuilder(I18nBuilder): Builds gettext-style message catalogs (.pot files). """ name = 'gettext' - epilog = 'The message catalogs are in %(outdir)s.' + epilog = __('The message catalogs are in %(outdir)s.') def init(self): # type: () -> None @@ -238,12 +240,12 @@ class MessageCatalogBuilder(I18nBuilder): # type: () -> None files = list(self._collect_templates()) files.sort() - logger.info(bold('building [%s]: ' % self.name), nonl=1) - logger.info('targets for %d template files', len(files)) + logger.info(bold(__('building [%s]: ') % self.name), nonl=1) + logger.info(__('targets for %d template files'), len(files)) extract_translations = self.templates.environment.extract_translations - for template in status_iterator(files, 'reading templates... ', "purple", # type: ignore # NOQA + for template in status_iterator(files, __('reading templates... '), "purple", # type: ignore # NOQA len(files), self.app.verbosity): with open(template, 'r', encoding='utf-8') as f: # type: ignore context = f.read() @@ -267,7 +269,7 @@ class MessageCatalogBuilder(I18nBuilder): timestamp, ltz).strftime('%Y-%m-%d %H:%M%z'), ) for textdomain, catalog in status_iterator(iteritems(self.catalogs), # type: ignore - "writing message catalogs... ", + __("writing message catalogs... "), "darkgreen", len(self.catalogs), self.app.verbosity, lambda textdomain__: textdomain__[0]): diff --git a/sphinx/builders/html.py b/sphinx/builders/html.py index fad49486b..90558c0e4 100644 --- a/sphinx/builders/html.py +++ b/sphinx/builders/html.py @@ -31,13 +31,13 @@ from sphinx import package_dir, __display_version__ from sphinx.application import ENV_PICKLE_FILENAME from sphinx.builders import Builder from sphinx.config import string_classes -from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx.deprecation import RemovedInSphinx20Warning, RemovedInSphinx30Warning from sphinx.environment.adapters.asset import ImageAdapter from sphinx.environment.adapters.indexentries import IndexEntries from sphinx.environment.adapters.toctree import TocTree from sphinx.errors import ThemeError from sphinx.highlighting import PygmentsBridge -from sphinx.locale import _, __, l_ +from sphinx.locale import _, __ from sphinx.search import js_index from sphinx.theming import HTMLThemeFactory from sphinx.util import jsonimpl, logging, status_iterator @@ -50,6 +50,7 @@ from sphinx.util.matching import patmatch, Matcher, DOTFILES from sphinx.util.nodes import inline_all_toctrees from sphinx.util.osutil import SEP, os_path, relative_uri, ensuredir, \ movefile, copyfile +from sphinx.util.pycompat import htmlescape from sphinx.writers.html import HTMLWriter, HTMLTranslator if False: @@ -97,28 +98,32 @@ class CSSContainer(list): the entry with Stylesheet class. """ def append(self, obj): + # type: (Union[unicode, Stylesheet]) -> None if isinstance(obj, Stylesheet): super(CSSContainer, self).append(obj) else: - super(CSSContainer, self).append(Stylesheet(obj, None, 'stylesheet')) + super(CSSContainer, self).append(Stylesheet(obj)) def insert(self, index, obj): + # type: (int, Union[unicode, Stylesheet]) -> None warnings.warn('builder.css_files is deprecated. ' 'Please use app.add_stylesheet() instead.', RemovedInSphinx20Warning) if isinstance(obj, Stylesheet): super(CSSContainer, self).insert(index, obj) else: - super(CSSContainer, self).insert(index, Stylesheet(obj, None, 'stylesheet')) + super(CSSContainer, self).insert(index, Stylesheet(obj)) - def extend(self, other): + def extend(self, other): # type: ignore + # type: (List[Union[unicode, Stylesheet]]) -> None warnings.warn('builder.css_files is deprecated. ' 'Please use app.add_stylesheet() instead.', RemovedInSphinx20Warning) for item in other: self.append(item) - def __iadd__(self, other): + def __iadd__(self, other): # type: ignore + # type: (List[Union[unicode, Stylesheet]]) -> CSSContainer warnings.warn('builder.css_files is deprecated. ' 'Please use app.add_stylesheet() instead.', RemovedInSphinx20Warning) @@ -127,6 +132,7 @@ class CSSContainer(list): return self def __add__(self, other): + # type: (List[Union[unicode, Stylesheet]]) -> CSSContainer ret = CSSContainer(self) ret += other return ret @@ -139,12 +145,19 @@ class Stylesheet(text_type): its filename (str). """ - def __new__(cls, filename, title, rel): + attributes = None # type: Dict[unicode, unicode] + filename = None # type: unicode + + def __new__(cls, filename, *args, **attributes): # type: (unicode, unicode, unicode) -> None self = text_type.__new__(cls, filename) # type: ignore self.filename = filename - self.title = title - self.rel = rel + self.attributes = attributes + self.attributes.setdefault('rel', 'stylesheet') + self.attributes.setdefault('type', 'text/css') + if args: # old style arguments (rel, title) + self.attributes['rel'] = args[0] + self.attributes['title'] = args[1] return self @@ -170,7 +183,7 @@ class BuildInfo(object): build_info.tags_hash = lines[3].split()[1].strip() return build_info except Exception as exc: - raise ValueError('build info file is broken: %r' % exc) + raise ValueError(__('build info file is broken: %r') % exc) def __init__(self, config=None, tags=None, config_categories=[]): # type: (Config, Tags, List[unicode]) -> None @@ -209,7 +222,7 @@ class StandaloneHTMLBuilder(Builder): """ name = 'html' format = 'html' - epilog = 'The HTML pages are in %(outdir)s.' + epilog = __('The HTML pages are in %(outdir)s.') copysource = True allow_parallel = True @@ -236,8 +249,6 @@ class StandaloneHTMLBuilder(Builder): # This is a class attribute because it is mutated by Sphinx.add_javascript. script_files = ['_static/jquery.js', '_static/underscore.js', '_static/doctools.js'] # type: List[unicode] - # Ditto for this one (Sphinx.add_stylesheet). - css_files = CSSContainer() # type: List[Dict[unicode, unicode]] imgpath = None # type: unicode domain_indices = [] # type: List[Tuple[unicode, Type[Index], List[Tuple[unicode, List[List[Union[unicode, int]]]]], bool]] # NOQA @@ -245,6 +256,13 @@ class StandaloneHTMLBuilder(Builder): # cached publisher object for snippets _publisher = None + def __init__(self, app): + # type: (Sphinx) -> None + super(StandaloneHTMLBuilder, self).__init__(app) + + # CSS files + self.css_files = CSSContainer() # type: List[Dict[unicode, unicode]] + def init(self): # type: () -> None self.build_info = self.create_build_info() @@ -257,6 +275,7 @@ class StandaloneHTMLBuilder(Builder): self.init_templates() self.init_highlighter() + self.init_css_files() if self.config.html_file_suffix is not None: self.out_suffix = self.config.html_file_suffix @@ -319,8 +338,24 @@ class StandaloneHTMLBuilder(Builder): self.highlighter = PygmentsBridge('html', style, self.config.trim_doctest_flags) + def init_css_files(self): + # type: () -> None + for filename, attrs in self.app.registry.css_files: + self.add_css_file(filename, **attrs) + + for filename, attrs in self.get_builder_config('css_files', 'html'): + self.add_css_file(filename, **attrs) + + def add_css_file(self, filename, **kwargs): + # type: (unicode, **unicode) -> None + if '://' not in filename: + filename = posixpath.join('_static', filename) + + self.css_files.append(Stylesheet(filename, **kwargs)) # type: ignore + @property def default_translator_class(self): + # type: () -> nodes.NodeVisitor use_html5_writer = self.config.html_experimental_html5_writer if use_html5_writer is None: use_html5_writer = self.default_html5_translator @@ -341,7 +376,7 @@ class StandaloneHTMLBuilder(Builder): yield docname return except ValueError as exc: - logger.warning('Failed to read build info file: %r', exc) + logger.warning(__('Failed to read build info file: %r'), exc) except IOError: # ignore errors on reading pass @@ -370,7 +405,7 @@ class StandaloneHTMLBuilder(Builder): def get_asset_paths(self): # type: () -> List[unicode] - return self.config.html_extra_path + return self.config.html_extra_path + self.config.html_static_path def render_partial(self, node): # type: (nodes.Nodes) -> Dict[unicode, unicode] @@ -403,9 +438,9 @@ class StandaloneHTMLBuilder(Builder): # create the search indexer self.indexer = None if self.search: - from sphinx.search import IndexBuilder, languages + from sphinx.search import IndexBuilder lang = self.config.html_search_language or self.config.language - if not lang or lang not in languages: + if not lang: lang = 'en' self.indexer = IndexBuilder(self.env, lang, self.config.html_search_options, @@ -441,7 +476,7 @@ class StandaloneHTMLBuilder(Builder): # typically doesn't include the time of day lufmt = self.config.html_last_updated_fmt if lufmt is not None: - self.last_updated = format_date(lufmt or _('%b %d, %Y'), # type: ignore + self.last_updated = format_date(lufmt or _('%b %d, %Y'), language=self.config.language) else: self.last_updated = None @@ -453,7 +488,7 @@ class StandaloneHTMLBuilder(Builder): path.basename(self.config.html_favicon) or '' if not isinstance(self.config.html_use_opensearch, string_types): - logger.warning('html_use_opensearch config value must now be a string') + logger.warning(__('html_use_opensearch config value must now be a string')) self.relations = self.env.collect_relations() @@ -630,7 +665,7 @@ class StandaloneHTMLBuilder(Builder): def gen_indices(self): # type: () -> None - logger.info(bold('generating indices...'), nonl=1) + logger.info(bold(__('generating indices...')), nonl=1) # the global general index if self.use_index: @@ -648,7 +683,7 @@ class StandaloneHTMLBuilder(Builder): for pagename, context, template in pagelist: self.handle_page(pagename, context, template) - logger.info(bold('writing additional pages...'), nonl=1) + logger.info(bold(__('writing additional pages...')), nonl=1) # additional pages from conf.py for pagename, template in self.config.html_additional_pages.items(): @@ -714,7 +749,7 @@ class StandaloneHTMLBuilder(Builder): if self.images: stringify_func = ImageAdapter(self.app.env).get_original_image_uri ensuredir(path.join(self.outdir, self.imagedir)) - for src in status_iterator(self.images, 'copying images... ', "brown", + for src in status_iterator(self.images, __('copying images... '), "brown", len(self.images), self.app.verbosity, stringify_func=stringify_func): dest = self.images[src] @@ -722,7 +757,7 @@ class StandaloneHTMLBuilder(Builder): copyfile(path.join(self.srcdir, src), path.join(self.outdir, self.imagedir, dest)) except Exception as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) def copy_download_files(self): @@ -733,7 +768,7 @@ class StandaloneHTMLBuilder(Builder): # copy downloadable files if self.env.dlfiles: ensuredir(path.join(self.outdir, '_downloads')) - for src in status_iterator(self.env.dlfiles, 'copying downloadable files... ', + for src in status_iterator(self.env.dlfiles, __('copying downloadable files... '), "brown", len(self.env.dlfiles), self.app.verbosity, stringify_func=to_relpath): dest = self.env.dlfiles[src][1] @@ -741,13 +776,13 @@ class StandaloneHTMLBuilder(Builder): copyfile(path.join(self.srcdir, src), path.join(self.outdir, '_downloads', dest)) except Exception as err: - logger.warning('cannot copy downloadable file %r: %s', + logger.warning(__('cannot copy downloadable file %r: %s'), path.join(self.srcdir, src), err) def copy_static_files(self): # type: () -> None # copy static files - logger.info(bold('copying static files... '), nonl=True) + logger.info(bold(__('copying static files... ')), nonl=True) ensuredir(path.join(self.outdir, '_static')) # first, create pygments style file with open(path.join(self.outdir, '_static', 'pygments.css'), 'w') as f: @@ -782,7 +817,7 @@ class StandaloneHTMLBuilder(Builder): for static_path in self.config.html_static_path: entry = path.join(self.confdir, static_path) if not path.exists(entry): - logger.warning('html_static_path entry %r does not exist', entry) + logger.warning(__('html_static_path entry %r does not exist'), entry) continue copy_asset(entry, path.join(self.outdir, '_static'), excluded, context=ctx, renderer=self.templates) @@ -791,7 +826,7 @@ class StandaloneHTMLBuilder(Builder): logobase = path.basename(self.config.html_logo) logotarget = path.join(self.outdir, '_static', logobase) if not path.isfile(path.join(self.confdir, self.config.html_logo)): - logger.warning('logo file %r does not exist', self.config.html_logo) + logger.warning(__('logo file %r does not exist'), self.config.html_logo) elif not path.isfile(logotarget): copyfile(path.join(self.confdir, self.config.html_logo), logotarget) @@ -799,7 +834,7 @@ class StandaloneHTMLBuilder(Builder): iconbase = path.basename(self.config.html_favicon) icontarget = path.join(self.outdir, '_static', iconbase) if not path.isfile(path.join(self.confdir, self.config.html_favicon)): - logger.warning('favicon file %r does not exist', self.config.html_favicon) + logger.warning(__('favicon file %r does not exist'), self.config.html_favicon) elif not path.isfile(icontarget): copyfile(path.join(self.confdir, self.config.html_favicon), icontarget) @@ -808,17 +843,17 @@ class StandaloneHTMLBuilder(Builder): def copy_extra_files(self): # type: () -> None # copy html_extra_path files - logger.info(bold('copying extra files... '), nonl=True) + logger.info(bold(__('copying extra files... ')), nonl=True) excluded = Matcher(self.config.exclude_patterns) for extra_path in self.config.html_extra_path: entry = path.join(self.confdir, extra_path) if not path.exists(entry): - logger.warning('html_extra_path entry %r does not exist', entry) + logger.warning(__('html_extra_path entry %r does not exist'), entry) continue copy_asset(entry, self.outdir, excluded) - logger.info('done') + logger.info(__('done')) def write_buildinfo(self): # type: () -> None @@ -826,7 +861,7 @@ class StandaloneHTMLBuilder(Builder): with open(path.join(self.outdir, '.buildinfo'), 'w') as fp: self.build_info.dump(fp) except IOError as exc: - logger.warning('Failed to write build info file: %r', exc) + logger.warning(__('Failed to write build info file: %r'), exc) def cleanup(self): # type: () -> None @@ -873,9 +908,9 @@ class StandaloneHTMLBuilder(Builder): self.indexer.load(f, self.indexer_format) except (IOError, OSError, ValueError): if keep: - logger.warning('search index couldn\'t be loaded, but not all ' - 'documents will be built: the index will be ' - 'incomplete.') + logger.warning(__('search index couldn\'t be loaded, but not all ' + 'documents will be built: the index will be ' + 'incomplete.')) # delete all entries for files that will be rebuilt self.indexer.prune(keep) @@ -927,8 +962,8 @@ class StandaloneHTMLBuilder(Builder): if has_wildcard(pattern): # warn if both patterns contain wildcards if has_wildcard(matched): - logger.warning('page %s matches two patterns in ' - 'html_sidebars: %r and %r', + logger.warning(__('page %s matches two patterns in ' + 'html_sidebars: %r and %r'), pagename, matched, pattern) # else the already matched pattern is more specific # than the present one, because it contains no wildcard @@ -982,6 +1017,17 @@ class StandaloneHTMLBuilder(Builder): return uri ctx['pathto'] = pathto + def css_tag(css): + # type: (Stylesheet) -> unicode + attrs = [] + for key in sorted(css.attributes): + value = css.attributes[key] + if value is not None: + attrs.append('%s="%s"' % (key, htmlescape(value, True))) + attrs.append('href="%s"' % pathto(css.filename, resource=True)) + return '<link %s />' % ' '.join(attrs) + ctx['css_tag'] = css_tag + def hasdoc(name): # type: (unicode) -> bool if name in self.env.all_docs: @@ -996,6 +1042,9 @@ class StandaloneHTMLBuilder(Builder): def warn(*args, **kwargs): # type: (Any, Any) -> unicode """Simple warn() wrapper for themes.""" + warnings.warn('The template function warn() was deprecated. ' + 'Use warning() instead.', + RemovedInSphinx30Warning) self.warn(*args, **kwargs) return '' # return empty string ctx['warn'] = warn @@ -1013,9 +1062,9 @@ class StandaloneHTMLBuilder(Builder): try: output = self.templates.render(templatename, ctx) except UnicodeError: - logger.warning("a Unicode error occurred when rendering the page %s. " - "Please make sure all config values that contain " - "non-ASCII content are Unicode strings.", pagename) + logger.warning(__("a Unicode error occurred when rendering the page %s. " + "Please make sure all config values that contain " + "non-ASCII content are Unicode strings."), pagename) return except Exception as exc: raise ThemeError(__("An error happened in rendering the page %s.\nReason: %r") % @@ -1029,7 +1078,7 @@ class StandaloneHTMLBuilder(Builder): with codecs.open(outfilename, 'w', ctx['encoding'], 'xmlcharrefreplace') as f: # type: ignore # NOQA f.write(output) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", outfilename, err) + logger.warning(__("error writing file %s: %s"), outfilename, err) if self.copysource and ctx.get('sourcename'): # copy the source file for the "show source" link source_name = path.join(self.outdir, '_sources', @@ -1049,14 +1098,14 @@ class StandaloneHTMLBuilder(Builder): def dump_inventory(self): # type: () -> None - logger.info(bold('dumping object inventory... '), nonl=True) + logger.info(bold(__('dumping object inventory... ')), nonl=True) InventoryFile.dump(path.join(self.outdir, INVENTORY_FILENAME), self.env, self) - logger.info('done') + logger.info(__('done')) def dump_search_index(self): # type: () -> None logger.info( - bold('dumping search index in %s ... ' % self.indexer.label()), + bold(__('dumping search index in %s ... ') % self.indexer.label()), nonl=True) self.indexer.prune(self.env.all_docs) searchindexfn = path.join(self.outdir, self.searchindex_filename) @@ -1069,7 +1118,7 @@ class StandaloneHTMLBuilder(Builder): with f: self.indexer.dump(f, self.indexer_format) movefile(searchindexfn + '.tmp', searchindexfn) - logger.info('done') + logger.info(__('done')) class DirectoryHTMLBuilder(StandaloneHTMLBuilder): @@ -1111,7 +1160,7 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder): HTML page. """ name = 'singlehtml' - epilog = 'The HTML page is in %(outdir)s.' + epilog = __('The HTML page is in %(outdir)s.') copysource = False @@ -1241,24 +1290,24 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder): # type: (Any) -> None docnames = self.env.all_docs - logger.info(bold('preparing documents... '), nonl=True) + logger.info(bold(__('preparing documents... ')), nonl=True) self.prepare_writing(docnames) - logger.info('done') + logger.info(__('done')) - logger.info(bold('assembling single document... '), nonl=True) + logger.info(bold(__('assembling single document... ')), nonl=True) doctree = self.assemble_doctree() self.env.toc_secnumbers = self.assemble_toc_secnumbers() self.env.toc_fignumbers = self.assemble_toc_fignumbers() logger.info('') - logger.info(bold('writing... '), nonl=True) + logger.info(bold(__('writing... ')), nonl=True) self.write_doc_serialized(self.config.master_doc, doctree) self.write_doc(self.config.master_doc, doctree) - logger.info('done') + logger.info(__('done')) def finish(self): # type: () -> None # no indices or search pages are supported - logger.info(bold('writing additional files...'), nonl=1) + logger.info(bold(__('writing additional files...')), nonl=1) # additional pages from conf.py for pagename, template in self.config.html_additional_pages.items(): @@ -1307,6 +1356,7 @@ class SerializingHTMLBuilder(StandaloneHTMLBuilder): self.templates = None # no template bridge necessary self.init_templates() self.init_highlighter() + self.init_css_files() self.use_index = self.get_builder_config('use_index', 'html') def get_target_uri(self, docname, typ=None): @@ -1375,7 +1425,7 @@ class PickleHTMLBuilder(SerializingHTMLBuilder): A Builder that dumps the generated HTML into pickle files. """ name = 'pickle' - epilog = 'You can now process the pickle files in %(outdir)s.' + epilog = __('You can now process the pickle files in %(outdir)s.') implementation = pickle implementation_dumps_unicode = False @@ -1396,7 +1446,7 @@ class JSONHTMLBuilder(SerializingHTMLBuilder): A builder that dumps the generated HTML into JSON files. """ name = 'json' - epilog = 'You can now process the JSON files in %(outdir)s.' + epilog = __('You can now process the JSON files in %(outdir)s.') implementation = jsonimpl implementation_dumps_unicode = True @@ -1411,6 +1461,24 @@ class JSONHTMLBuilder(SerializingHTMLBuilder): SerializingHTMLBuilder.init(self) +def convert_html_css_files(app, config): + # type: (Sphinx, Config) -> None + """This converts string styled html_css_files to tuple styled one.""" + html_css_files = [] # type: List[Tuple[unicode, Dict]] + for entry in config.html_css_files: + if isinstance(entry, string_types): + html_css_files.append((entry, {})) + else: + try: + filename, attrs = entry + html_css_files.append((filename, attrs)) + except Exception: + logger.warning(__('invalid css_file: %r, ignored'), entry) + continue + + config.html_css_files = html_css_files # type: ignore + + def setup(app): # type: (Sphinx) -> Dict[unicode, Any] # builders @@ -1425,12 +1493,13 @@ def setup(app): app.add_config_value('html_theme_path', [], 'html') app.add_config_value('html_theme_options', {}, 'html') app.add_config_value('html_title', - lambda self: l_('%s %s documentation') % (self.project, self.release), + lambda self: _('%s %s documentation') % (self.project, self.release), 'html', string_classes) app.add_config_value('html_short_title', lambda self: self.html_title, 'html') app.add_config_value('html_style', None, 'html', string_classes) app.add_config_value('html_logo', None, 'html', string_classes) app.add_config_value('html_favicon', None, 'html', string_classes) + app.add_config_value('html_css_files', [], 'html') app.add_config_value('html_static_path', [], 'html') app.add_config_value('html_extra_path', [], 'html') app.add_config_value('html_last_updated_fmt', None, 'html', string_classes) @@ -1458,6 +1527,9 @@ def setup(app): app.add_config_value('html_scaled_image_link', True, 'html') app.add_config_value('html_experimental_html5_writer', None, 'html') + # event handlers + app.connect('config-inited', convert_html_css_files) + return { 'version': 'builtin', 'parallel_read_safe': True, diff --git a/sphinx/builders/htmlhelp.py b/sphinx/builders/htmlhelp.py index 8be51d50d..49d48361e 100644 --- a/sphinx/builders/htmlhelp.py +++ b/sphinx/builders/htmlhelp.py @@ -20,6 +20,7 @@ from docutils import nodes from sphinx import addnodes from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.environment.adapters.indexentries import IndexEntries +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import make_filename from sphinx.util.pycompat import htmlescape @@ -174,8 +175,8 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): index files. Adapted from the original Doc/tools/prechm.py. """ name = 'htmlhelp' - epilog = ('You can now run HTML Help Workshop with the .htp file in ' - '%(outdir)s.') + epilog = __('You can now run HTML Help Workshop with the .htp file in ' + '%(outdir)s.') # don't copy the reST source copysource = False @@ -228,12 +229,12 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): def build_hhx(self, outdir, outname): # type: (unicode, unicode) -> None - logger.info('dumping stopword list...') + logger.info(__('dumping stopword list...')) with self.open_file(outdir, outname + '.stp') as f: for word in sorted(stopwords): print(word, file=f) - logger.info('writing project file...') + logger.info(__('writing project file...')) with self.open_file(outdir, outname + '.hhp') as f: f.write(project_template % { 'outname': outname, @@ -256,7 +257,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): print(path.join(root, fn)[olen:].replace(os.sep, '\\'), file=f) - logger.info('writing TOC file...') + logger.info(__('writing TOC file...')) with self.open_file(outdir, outname + '.hhc') as f: f.write(contents_header) # special books @@ -298,7 +299,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): write_toc(node) f.write(contents_footer) - logger.info('writing index file...') + logger.info(__('writing index file...')) index = IndexEntries(self.env).create_index(self) with self.open_file(outdir, outname + '.hhk') as f: f.write('<UL>\n') diff --git a/sphinx/builders/latex/__init__.py b/sphinx/builders/latex/__init__.py index 3e366ba05..ef3dd7276 100644 --- a/sphinx/builders/latex/__init__.py +++ b/sphinx/builders/latex/__init__.py @@ -12,20 +12,22 @@ import os from os import path -from docutils import nodes from docutils.frontend import OptionParser from docutils.io import FileOutput from six import text_type from sphinx import package_dir, addnodes, highlighting from sphinx.builders import Builder +from sphinx.builders.latex.transforms import ( + BibliographyTransform, CitationReferenceTransform, + FootnoteDocnameUpdater, LaTeXFootnoteTransform, ShowUrlsTransform +) from sphinx.config import string_classes, ENUM from sphinx.environment import NoUri from sphinx.environment.adapters.asset import ImageAdapter from sphinx.errors import SphinxError, ConfigError -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.transforms import SphinxTransformer -from sphinx.transforms.references import SubstitutionDefinitionsRemover from sphinx.util import texescape, logging, status_iterator from sphinx.util.console import bold, darkgreen # type: ignore from sphinx.util.docutils import new_document @@ -36,6 +38,7 @@ from sphinx.writers.latex import LaTeXWriter, LaTeXTranslator if False: # For type annotation + from docutils import nodes # NOQA from typing import Any, Dict, Iterable, List, Tuple, Union # NOQA from sphinx.application import Sphinx # NOQA from sphinx.config import Config # NOQA @@ -50,11 +53,11 @@ class LaTeXBuilder(Builder): """ name = 'latex' format = 'latex' - epilog = 'The LaTeX files are in %(outdir)s.' + epilog = __('The LaTeX files are in %(outdir)s.') if os.name == 'posix': - epilog += ("\nRun 'make' in that directory to run these through " - "(pdf)latex\n" - "(use `make latexpdf' here to do that automatically).") + epilog += __("\nRun 'make' in that directory to run these through " + "(pdf)latex\n" + "(use `make latexpdf' here to do that automatically).") supported_image_types = ['application/pdf', 'image/png', 'image/jpeg'] supported_remote_images = False @@ -64,7 +67,7 @@ class LaTeXBuilder(Builder): # type: () -> None self.docnames = [] # type: Iterable[unicode] self.document_data = [] # type: List[Tuple[unicode, unicode, unicode, unicode, unicode, bool]] # NOQA - self.usepackages = [] # type: List[unicode] + self.usepackages = self.app.registry.latex_packages texescape.init() def get_outdated_docs(self): @@ -87,16 +90,16 @@ class LaTeXBuilder(Builder): # type: () -> None preliminary_document_data = [list(x) for x in self.config.latex_documents] if not preliminary_document_data: - logger.warning('no "latex_documents" config value found; no documents ' - 'will be written') + logger.warning(__('no "latex_documents" config value found; no documents ' + 'will be written')) return # assign subdirs to titles self.titles = [] # type: List[Tuple[unicode, unicode]] for entry in preliminary_document_data: docname = entry[0] if docname not in self.env.all_docs: - logger.warning('"latex_documents" config value references unknown ' - 'document %s', docname) + logger.warning(__('"latex_documents" config value references unknown ' + 'document %s'), docname) continue self.document_data.append(entry) # type: ignore if docname.endswith(SEP + 'index'): @@ -133,7 +136,7 @@ class LaTeXBuilder(Builder): destination = FileOutput( destination_path=path.join(self.outdir, targetname), encoding='utf-8') - logger.info("processing %s...", targetname, nonl=1) + logger.info(__("processing %s..."), targetname, nonl=1) toctrees = self.env.get_doctree(docname).traverse(addnodes.toctree) if toctrees: if toctrees[0].get('maxdepth') > 0: @@ -148,7 +151,7 @@ class LaTeXBuilder(Builder): doctree['tocdepth'] = tocdepth self.apply_transforms(doctree) self.post_process_images(doctree) - logger.info("writing... ", nonl=1) + logger.info(__("writing... "), nonl=1) doctree.settings = docsettings doctree.settings.author = author doctree.settings.title = title @@ -171,6 +174,7 @@ class LaTeXBuilder(Builder): def assemble_doctree(self, indexfile, toctree_only, appendices): # type: (unicode, bool, List[unicode]) -> nodes.Node + from docutils import nodes # NOQA self.docnames = set([indexfile] + appendices) logger.info(darkgreen(indexfile) + " ", nonl=1) tree = self.env.get_doctree(indexfile) @@ -194,7 +198,7 @@ class LaTeXBuilder(Builder): appendix['docname'] = docname largetree.append(appendix) logger.info('') - logger.info("resolving references...") + logger.info(__("resolving references...")) self.env.resolve_references(largetree, indexfile, self) # resolve :ref:s to distant tex files -- we can't add a cross-reference, # but append the document name @@ -217,7 +221,9 @@ class LaTeXBuilder(Builder): # type: (nodes.document) -> None transformer = SphinxTransformer(doctree) transformer.set_environment(self.env) - transformer.add_transforms([SubstitutionDefinitionsRemover]) + transformer.add_transforms([BibliographyTransform, + ShowUrlsTransform, + LaTeXFootnoteTransform]) transformer.apply_transforms() def finish(self): @@ -226,7 +232,7 @@ class LaTeXBuilder(Builder): # copy TeX support files from texinputs context = {'latex_engine': self.config.latex_engine} - logger.info(bold('copying TeX support files...')) + logger.info(bold(__('copying TeX support files...'))) staticdirname = path.join(package_dir, 'texinputs') for filename in os.listdir(staticdirname): if not filename.startswith('.'): @@ -241,7 +247,7 @@ class LaTeXBuilder(Builder): # copy additional files if self.config.latex_additional_files: - logger.info(bold('copying additional files...'), nonl=1) + logger.info(bold(__('copying additional files...')), nonl=1) for filename in self.config.latex_additional_files: logger.info(' ' + filename, nonl=1) copy_asset_file(path.join(self.confdir, filename), self.outdir) @@ -250,16 +256,16 @@ class LaTeXBuilder(Builder): # the logo is handled differently if self.config.latex_logo: if not path.isfile(path.join(self.confdir, self.config.latex_logo)): - raise SphinxError('logo file %r does not exist' % self.config.latex_logo) + raise SphinxError(__('logo file %r does not exist') % self.config.latex_logo) else: copy_asset_file(path.join(self.confdir, self.config.latex_logo), self.outdir) - logger.info('done') + logger.info(__('done')) def copy_image_files(self): # type: () -> None if self.images: stringify_func = ImageAdapter(self.app.env).get_original_image_uri - for src in status_iterator(self.images, 'copying images... ', "brown", + for src in status_iterator(self.images, __('copying images... '), "brown", len(self.images), self.app.verbosity, stringify_func=stringify_func): dest = self.images[src] @@ -267,27 +273,27 @@ class LaTeXBuilder(Builder): copy_asset_file(path.join(self.srcdir, src), path.join(self.outdir, dest)) except Exception as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) -def validate_config_values(app): - # type: (Sphinx) -> None - for document in app.config.latex_documents: +def validate_config_values(app, config): + # type: (Sphinx, Config) -> None + for document in config.latex_documents: try: text_type(document[2]) except UnicodeDecodeError: raise ConfigError( - 'Invalid latex_documents.title found (might contain non-ASCII chars. ' - 'Please use u"..." notation instead): %r' % (document,) + __('Invalid latex_documents.title found (might contain non-ASCII chars. ' + 'Please use u"..." notation instead): %r') % (document,) ) try: text_type(document[3]) except UnicodeDecodeError: raise ConfigError( - 'Invalid latex_documents.author found (might contain non-ASCII chars. ' - 'Please use u"..." notation instead): %r' % (document,) + __('Invalid latex_documents.author found (might contain non-ASCII chars. ' + 'Please use u"..." notation instead): %r') % (document,) ) @@ -313,7 +319,9 @@ def default_latex_docclass(config): def setup(app): # type: (Sphinx) -> Dict[unicode, Any] app.add_builder(LaTeXBuilder) - app.connect('builder-inited', validate_config_values) + app.add_post_transform(CitationReferenceTransform) + app.connect('config-inited', validate_config_values) + app.add_transform(FootnoteDocnameUpdater) app.add_config_value('latex_engine', default_latex_engine, None, ENUM('pdflatex', 'xelatex', 'lualatex', 'platex')) @@ -325,7 +333,7 @@ def setup(app): app.add_config_value('latex_appendices', [], None) app.add_config_value('latex_use_latex_multicolumn', False, None) app.add_config_value('latex_toplevel_sectioning', None, None, - ENUM('part', 'chapter', 'section')) + ENUM(None, 'part', 'chapter', 'section')) app.add_config_value('latex_domain_indices', True, None, [list]) app.add_config_value('latex_show_urls', 'no', None) app.add_config_value('latex_show_pagerefs', False, None) diff --git a/sphinx/builders/latex/nodes.py b/sphinx/builders/latex/nodes.py new file mode 100644 index 000000000..80734820d --- /dev/null +++ b/sphinx/builders/latex/nodes.py @@ -0,0 +1,27 @@ +# -*- coding: utf-8 -*- +""" + sphinx.builders.latex.nodes + ~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Additional nodes for LaTeX writer. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from docutils import nodes + + +class footnotemark(nodes.Inline, nodes.Referential, nodes.TextElement): + """A node represents ``\footnotemark``.""" + pass + + +class footnotetext(nodes.General, nodes.BackLinkable, nodes.Element, + nodes.Labeled, nodes.Targetable): + """A node represents ``\footnotetext``.""" + + +class thebibliography(nodes.container): + """A node for wrapping bibliographies.""" + pass diff --git a/sphinx/builders/latex/transforms.py b/sphinx/builders/latex/transforms.py index 201129989..5b5231a04 100644 --- a/sphinx/builders/latex/transforms.py +++ b/sphinx/builders/latex/transforms.py @@ -11,19 +11,33 @@ from docutils import nodes +from sphinx import addnodes +from sphinx.builders.latex.nodes import footnotemark, footnotetext, thebibliography from sphinx.transforms import SphinxTransform if False: # For type annotation - from typing import Dict, List, Set, Union # NOQA + from typing import Dict, List, Set, Tuple, Union # NOQA URI_SCHEMES = ('mailto:', 'http:', 'https:', 'ftp:') +class FootnoteDocnameUpdater(SphinxTransform): + """Add docname to footnote and footnote_reference nodes.""" + default_priority = 200 + TARGET_NODES = (nodes.footnote, nodes.footnote_reference) + + def apply(self): + for node in self.document.traverse(lambda n: isinstance(n, self.TARGET_NODES)): + node['docname'] = self.env.docname + + class ShowUrlsTransform(SphinxTransform): """Expand references to inline text or footnotes. For more information, see :confval:`latex_show_urls`. + + .. note:: This transform is used for integrated doctree """ default_priority = 400 @@ -32,16 +46,17 @@ class ShowUrlsTransform(SphinxTransform): def apply(self): # type: () -> None - # replace id_prefix temporarily - id_prefix = self.document.settings.id_prefix - self.document.settings.id_prefix = 'show_urls' - - self.expand_show_urls() - if self.expanded: - self.renumber_footnotes() - - # restore id_prefix - self.document.settings.id_prefix = id_prefix + try: + # replace id_prefix temporarily + id_prefix = self.document.settings.id_prefix + self.document.settings.id_prefix = 'show_urls' + + self.expand_show_urls() + if self.expanded: + self.renumber_footnotes() + finally: + # restore id_prefix + self.document.settings.id_prefix = id_prefix def expand_show_urls(self): # type: () -> None @@ -56,32 +71,45 @@ class ShowUrlsTransform(SphinxTransform): uri = uri[7:] if node.astext() != uri: index = node.parent.index(node) + docname = self.get_docname_for_node(node) if show_urls == 'footnote': - footnote_nodes = self.create_footnote(uri) - for i, fn in enumerate(footnote_nodes): - node.parent.insert(index + i + 1, fn) + fn, fnref = self.create_footnote(uri, docname) + node.parent.insert(index + 1, fn) + node.parent.insert(index + 2, fnref) self.expanded = True else: # all other true values (b/w compat) textnode = nodes.Text(" (%s)" % uri) node.parent.insert(index + 1, textnode) - def create_footnote(self, uri): - # type: (unicode) -> List[Union[nodes.footnote, nodes.footnote_ref]] + def get_docname_for_node(self, node): + # type: (nodes.Node) -> unicode + while node: + if isinstance(node, nodes.document): + return self.env.path2doc(node['source']) + elif isinstance(node, addnodes.start_of_file): + return node['docname'] + else: + node = node.parent + + return None # never reached here. only for type hinting + + def create_footnote(self, uri, docname): + # type: (unicode, unicode) -> Tuple[nodes.footnote, nodes.footnote_ref] label = nodes.label('', '#') para = nodes.paragraph() para.append(nodes.reference('', nodes.Text(uri), refuri=uri, nolinkurl=True)) - footnote = nodes.footnote(uri, label, para, auto=1) + footnote = nodes.footnote(uri, label, para, auto=1, docname=docname) footnote['names'].append('#') self.document.note_autofootnote(footnote) label = nodes.Text('#') footnote_ref = nodes.footnote_reference('[#]_', label, auto=1, - refid=footnote['ids'][0]) + refid=footnote['ids'][0], docname=docname) self.document.note_autofootnote_ref(footnote_ref) footnote.add_backref(footnote_ref['ids'][0]) - return [footnote, footnote_ref] + return footnote, footnote_ref def renumber_footnotes(self): # type: () -> None @@ -89,7 +117,7 @@ class ShowUrlsTransform(SphinxTransform): self.document.walkabout(collector) num = 0 - for document, footnote in collector.auto_footnotes: + for footnote in collector.auto_footnotes: # search unused footnote number while True: num += 1 @@ -104,8 +132,9 @@ class ShowUrlsTransform(SphinxTransform): footnote['names'].append(str(num)) # update footnote_references by new footnote number - for ref in collector.footnote_refs.get(document, []): - if footnote['ids'][0] == ref['refid']: + docname = footnote['docname'] + for ref in collector.footnote_refs: + if docname == ref['docname'] and footnote['ids'][0] == ref['refid']: ref.remove(ref[0]) ref += nodes.Text(str(num)) @@ -117,45 +146,399 @@ class FootnoteCollector(nodes.NodeVisitor): # type: (nodes.document) -> None self.auto_footnotes = [] # type: List[nodes.footnote] self.used_footnote_numbers = set() # type: Set[unicode] - self.footnote_refs = {} # type: Dict[nodes.Node, List[nodes.footnote_reference]] # NOQA - self.current_document = [] # type: List[nodes.Node] + self.footnote_refs = [] # type: List[nodes.footnote_reference] nodes.NodeVisitor.__init__(self, document) - def visit_document(self, node): - # type: (nodes.Node) -> None - self.current_document.append(node) - - def depart_document(self, node): - # type: (nodes.Node) -> None - self.current_document.pop() - - def visit_start_of_file(self, node): - # type: (nodes.Node) -> None - self.current_document.append(node) - - def depart_start_of_file(self, node): + def unknown_visit(self, node): # type: (nodes.Node) -> None - self.current_document.pop() + pass - def unknown_visit(self, node): + def unknown_departure(self, node): # type: (nodes.Node) -> None pass def visit_footnote(self, node): # type: (nodes.footnote) -> None - document = self.current_document[-1] if node.get('auto'): - self.auto_footnotes.append((document, node)) + self.auto_footnotes.append(node) else: for name in node['names']: self.used_footnote_numbers.add(name) def visit_footnote_reference(self, node): # type: (nodes.footnote_reference) -> None - document = self.current_document[-1] - footnote_refs = self.footnote_refs.setdefault(document, []) - footnote_refs.append(node) + self.footnote_refs.append(node) + + +class LaTeXFootnoteTransform(SphinxTransform): + """Convert footnote definitions and references to appropriate form to LaTeX. + + * Replace footnotes on restricted zone (e.g. headings) by footnotemark node. + In addition, append a footnotetext node after the zone. + + Before:: + + <section> + <title> + headings having footnotes + <footnote_reference> + 1 + <footnote ids="1"> + <label> + 1 + <paragraph> + footnote body + + After:: + + <section> + <title> + headings having footnotes + <footnotemark> + 1 + <footnotetext> + footnote body + <footnotetext> + <label> + 1 + <paragraph> + footnote body + + * Integrate footnote definitions and footnote references to single footnote node + + Before:: + + blah blah blah + <footnote_reference refid="id1"> + 1 + blah blah blah ... + + <footnote ids="1"> + <label> + 1 + <paragraph> + footnote body + + After:: + + blah blah blah + <footnote ids="1"> + <label> + 1 + <paragraph> + footnote body + blah blah blah ... + + * Replace second and subsequent footnote references which refers same footnote definition + by footnotemark node. + + Before:: + + blah blah blah + <footnote_reference refid="id1"> + 1 + blah blah blah + <footnote_reference refid="id1"> + 1 + blah blah blah ... + + <footnote ids="1"> + <label> + 1 + <paragraph> + footnote body + + After:: + + blah blah blah + <footnote ids="1"> + <label> + 1 + <paragraph> + footnote body + blah blah blah + <footnotemark> + 1 + blah blah blah ... + + * Remove unreferenced footnotes + + Before:: + + <footnote ids="1"> + <label> + 1 + <paragraph> + Unreferenced footnote! + + After:: + + <!-- nothing! --> + + * Move footnotes in a title of table or thead to head of tbody + + Before:: + + <table> + <title> + title having footnote_reference + <footnote_reference refid="1"> + 1 + <tgroup> + <thead> + <row> + <entry> + header having footnote_reference + <footnote_reference refid="2"> + 2 + <tbody> + <row> + ... + + <footnote ids="1"> + <label> + 1 + <paragraph> + footnote body + + <footnote ids="2"> + <label> + 2 + <paragraph> + footnote body + + After:: + + <table> + <title> + title having footnote_reference + <footnotemark> + 1 + <tgroup> + <thead> + <row> + <entry> + header having footnote_reference + <footnotemark> + 2 + <tbody> + <footnotetext> + <label> + 1 + <paragraph> + footnote body + + <footnotetext> + <label> + 2 + <paragraph> + footnote body + <row> + ... + """ + + default_priority = 600 + + def apply(self): + footnotes = list(self.document.traverse(nodes.footnote)) + for node in footnotes: + node.parent.remove(node) + + visitor = LaTeXFootnoteVisitor(self.document, footnotes) + self.document.walkabout(visitor) + + +class LaTeXFootnoteVisitor(nodes.NodeVisitor): + def __init__(self, document, footnotes): + # type: (nodes.document, List[nodes.footnote]) -> None + self.appeared = set() # type: Set[Tuple[unicode, nodes.footnote]] + self.footnotes = footnotes # type: List[nodes.footnote] + self.pendings = [] # type: List[nodes.Node] + self.table_footnotes = [] # type: List[nodes.Node] + self.restricted = None # type: nodes.Node + nodes.NodeVisitor.__init__(self, document) + + def unknown_visit(self, node): + # type: (nodes.Node) -> None + pass def unknown_departure(self, node): # type: (nodes.Node) -> None pass + + def restrict(self, node): + # type: (nodes.Node) -> None + if self.restricted is None: + self.restricted = node + + def unrestrict(self, node): + # type: (nodes.Node) -> None + if self.restricted == node: + self.restricted = None + pos = node.parent.index(node) + for i, footnote, in enumerate(self.pendings): + fntext = footnotetext('', *footnote.children) + node.parent.insert(pos + i + 1, fntext) + self.pendings = [] + + def visit_figure(self, node): + # type: (nodes.Node) -> None + self.restrict(node) + + def depart_figure(self, node): + # type: (nodes.Node) -> None + self.unrestrict(node) + + def visit_term(self, node): + # type: (nodes.Node) -> None + self.restrict(node) + + def depart_term(self, node): + # type: (nodes.Node) -> None + self.unrestrict(node) + + def visit_caption(self, node): + # type: (nodes.Node) -> None + self.restrict(node) + + def depart_caption(self, node): + # type: (nodes.Node) -> None + self.unrestrict(node) + + def visit_title(self, node): + # type: (nodes.Node) -> None + if isinstance(node.parent, (nodes.section, nodes.table)): + self.restrict(node) + + def depart_title(self, node): + # type: (nodes.Node) -> None + if isinstance(node.parent, nodes.section): + self.unrestrict(node) + elif isinstance(node.parent, nodes.table): + self.table_footnotes += self.pendings + self.pendings = [] + self.unrestrict(node) + + def visit_thead(self, node): + # type: (nodes.Node) -> None + self.restrict(node) + + def depart_thead(self, node): + # type: (nodes.Node) -> None + self.table_footnotes += self.pendings + self.pendings = [] + self.unrestrict(node) + + def depart_table(self, node): + # type: (nodes.Node) -> None + tbody = list(node.traverse(nodes.tbody))[0] + for footnote in reversed(self.table_footnotes): + fntext = footnotetext('', *footnote.children) + tbody.insert(0, fntext) + + self.table_footnotes = [] + + def visit_footnote(self, node): + # type: (nodes.Node) -> None + self.restrict(node) + + def depart_footnote(self, node): + # type: (nodes.Node) -> None + self.unrestrict(node) + + def visit_footnote_reference(self, node): + # type: (nodes.Node) -> None + number = node.astext().strip() + docname = node['docname'] + if self.restricted: + mark = footnotemark('', number) + node.replace_self(mark) + if (docname, number) not in self.appeared: + footnote = self.get_footnote_by_reference(node) + self.pendings.append(footnote) + elif (docname, number) in self.appeared: + mark = footnotemark('', number) + node.replace_self(mark) + else: + footnote = self.get_footnote_by_reference(node) + self.footnotes.remove(footnote) + node.replace_self(footnote) + footnote.walkabout(self) + + self.appeared.add((docname, number)) + raise nodes.SkipNode + + def get_footnote_by_reference(self, node): + # type: (nodes.Node) -> nodes.Node + docname = node['docname'] + for footnote in self.footnotes: + if docname == footnote['docname'] and footnote['ids'][0] == node['refid']: + return footnote + + return None + + +class BibliographyTransform(SphinxTransform): + """Gather bibliography entries to tail of document. + + Before:: + + <document> + <paragraph> + blah blah blah + <citation> + ... + <paragraph> + blah blah blah + <citation> + ... + ... + + After:: + + <document> + <paragraph> + blah blah blah + <paragraph> + blah blah blah + ... + <thebibliography> + <citation> + ... + <citation> + ... + """ + default_priority = 750 + + def apply(self): + # type: () -> None + citations = thebibliography() + for node in self.document.traverse(nodes.citation): + node.parent.remove(node) + citations += node + + if len(citations) > 0: + self.document += citations + + +class CitationReferenceTransform(SphinxTransform): + """Replace pending_xref nodes for citation by citation_reference. + + To handle citation reference easily on LaTeX writer, this converts + pending_xref nodes to citation_reference. + """ + default_priority = 5 # before ReferencesResolver + + def apply(self): + # type: () -> None + if self.app.builder.name != 'latex': + return + + citations = self.env.get_domain('std').data['citations'] + for node in self.document.traverse(addnodes.pending_xref): + if node['refdomain'] == 'std' and node['reftype'] == 'citation': + docname, labelid, _ = citations.get(node['reftarget'], ('', '', 0)) + if docname: + citation_ref = nodes.citation_reference('', *node.children, + docname=docname, refname=labelid) + node.replace_self(citation_ref) diff --git a/sphinx/builders/linkcheck.py b/sphinx/builders/linkcheck.py index 8a10a4f2c..22e7f01ad 100644 --- a/sphinx/builders/linkcheck.py +++ b/sphinx/builders/linkcheck.py @@ -31,6 +31,7 @@ except ImportError: pass from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import encode_uri, requests, logging from sphinx.util.console import ( # type: ignore purple, red, darkgreen, darkgray, darkred, turquoise @@ -58,6 +59,7 @@ class AnchorCheckParser(html_parser.HTMLParser): self.found = False def handle_starttag(self, tag, attrs): + # type: (Any, Any) -> None for key, value in attrs: if key in ('id', 'name') and value == self.search_anchor: self.found = True @@ -90,8 +92,8 @@ class CheckExternalLinksBuilder(Builder): Checks for broken external links. """ name = 'linkcheck' - epilog = ('Look for any errors in the above output or in ' - '%(outdir)s/output.txt') + epilog = __('Look for any errors in the above output or in ' + '%(outdir)s/output.txt') def init(self): # type: () -> None @@ -151,7 +153,7 @@ class CheckExternalLinksBuilder(Builder): found = check_anchor(response, unquote(anchor)) if not found: - raise Exception("Anchor '%s' not found" % anchor) + raise Exception(__("Anchor '%s' not found") % anchor) else: try: # try a HEAD request first, which should be easier on @@ -249,7 +251,7 @@ class CheckExternalLinksBuilder(Builder): elif status == 'broken': self.write_entry('broken', docname, lineno, uri + ': ' + info) if self.app.quiet or self.app.warningiserror: - logger.warning('broken link: %s (%s)', uri, info, + logger.warning(__('broken link: %s (%s)'), uri, info, location=(self.env.doc2path(docname), lineno)) else: logger.info(red('broken ') + uri + red(' - ' + info)) diff --git a/sphinx/builders/manpage.py b/sphinx/builders/manpage.py index 7a691ccf0..32bd9950b 100644 --- a/sphinx/builders/manpage.py +++ b/sphinx/builders/manpage.py @@ -18,6 +18,7 @@ from six import string_types from sphinx import addnodes from sphinx.builders import Builder from sphinx.environment import NoUri +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.console import bold, darkgreen # type: ignore from sphinx.util.nodes import inline_all_toctrees @@ -39,7 +40,7 @@ class ManualPageBuilder(Builder): """ name = 'man' format = 'man' - epilog = 'The manual pages are in %(outdir)s.' + epilog = __('The manual pages are in %(outdir)s.') default_translator_class = ManualPageTranslator supported_image_types = [] # type: List[unicode] @@ -47,8 +48,8 @@ class ManualPageBuilder(Builder): def init(self): # type: () -> None if not self.config.man_pages: - logger.warning('no "man_pages" config value found; no manual pages ' - 'will be written') + logger.warning(__('no "man_pages" config value found; no manual pages ' + 'will be written')) def get_outdated_docs(self): # type: () -> Union[unicode, List[unicode]] @@ -68,7 +69,7 @@ class ManualPageBuilder(Builder): components=(docwriter,), read_config_files=True).get_default_values() - logger.info(bold('writing... '), nonl=True) + logger.info(bold(__('writing... ')), nonl=True) for info in self.config.man_pages: docname, name, description, authors, section = info diff --git a/sphinx/builders/qthelp.py b/sphinx/builders/qthelp.py index 13ea256da..9afe47637 100644 --- a/sphinx/builders/qthelp.py +++ b/sphinx/builders/qthelp.py @@ -23,6 +23,7 @@ from sphinx import package_dir from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.config import string_classes from sphinx.environment.adapters.indexentries import IndexEntries +from sphinx.locale import __ from sphinx.util import force_decode, logging from sphinx.util.osutil import make_filename from sphinx.util.pycompat import htmlescape @@ -55,11 +56,11 @@ class QtHelpBuilder(StandaloneHTMLBuilder): Builder that also outputs Qt help project, contents and index files. """ name = 'qthelp' - epilog = ('You can now run "qcollectiongenerator" with the .qhcp ' - 'project file in %(outdir)s, like this:\n' - '$ qcollectiongenerator %(outdir)s/%(project)s.qhcp\n' - 'To view the help file:\n' - '$ assistant -collectionFile %(outdir)s/%(project)s.qhc') + epilog = __('You can now run "qcollectiongenerator" with the .qhcp ' + 'project file in %(outdir)s, like this:\n' + '$ qcollectiongenerator %(outdir)s/%(project)s.qhcp\n' + 'To view the help file:\n' + '$ assistant -collectionFile %(outdir)s/%(project)s.qhc') # don't copy the reST source copysource = False @@ -95,7 +96,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder): def build_qhp(self, outdir, outname): # type: (unicode, unicode) -> None - logger.info('writing project file...') + logger.info(__('writing project file...')) # sections tocdoc = self.env.get_and_resolve_doctree(self.config.master_doc, self, @@ -157,7 +158,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder): nspace, 'doc', self.get_target_uri(self.config.master_doc)) startpage = 'qthelp://' + posixpath.join(nspace, 'doc', 'index.html') - logger.info('writing collection project file...') + logger.info(__('writing collection project file...')) with codecs.open(path.join(outdir, outname + '.qhcp'), 'w', 'utf-8') as f: # type: ignore # NOQA body = render_file('project.qhcp', outname=outname, title=self.config.html_short_title, @@ -268,7 +269,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder): resourcedir = root.startswith((staticdir, imagesdir)) for fn in sorted(files): if (resourcedir and not fn.endswith('.js')) or fn.endswith('.html'): - filename = path.join(root, fn)[olen:] + filename = posixpath.join(root, fn)[olen:] project_files.append(filename) return project_files diff --git a/sphinx/builders/texinfo.py b/sphinx/builders/texinfo.py index c9a95a5fc..2a022158f 100644 --- a/sphinx/builders/texinfo.py +++ b/sphinx/builders/texinfo.py @@ -20,7 +20,7 @@ from sphinx import addnodes from sphinx.builders import Builder from sphinx.environment import NoUri from sphinx.environment.adapters.asset import ImageAdapter -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util import status_iterator from sphinx.util.console import bold, darkgreen # type: ignore @@ -98,11 +98,11 @@ class TexinfoBuilder(Builder): """ name = 'texinfo' format = 'texinfo' - epilog = 'The Texinfo files are in %(outdir)s.' + epilog = __('The Texinfo files are in %(outdir)s.') if os.name == 'posix': - epilog += ("\nRun 'make' in that directory to run these through " - "makeinfo\n" - "(use 'make info' here to do that automatically).") + epilog += __("\nRun 'make' in that directory to run these through " + "makeinfo\n" + "(use 'make info' here to do that automatically).") supported_image_types = ['image/png', 'image/jpeg', 'image/gif'] @@ -133,16 +133,16 @@ class TexinfoBuilder(Builder): # type: () -> None preliminary_document_data = [list(x) for x in self.config.texinfo_documents] if not preliminary_document_data: - logger.warning('no "texinfo_documents" config value found; no documents ' - 'will be written') + logger.warning(__('no "texinfo_documents" config value found; no documents ' + 'will be written')) return # assign subdirs to titles self.titles = [] # type: List[Tuple[unicode, unicode]] for entry in preliminary_document_data: docname = entry[0] if docname not in self.env.all_docs: - logger.warning('"texinfo_documents" config value references unknown ' - 'document %s', docname) + logger.warning(__('"texinfo_documents" config value references unknown ' + 'document %s'), docname) continue self.document_data.append(entry) # type: ignore if docname.endswith(SEP + 'index'): @@ -164,11 +164,11 @@ class TexinfoBuilder(Builder): destination = FileOutput( destination_path=path.join(self.outdir, targetname), encoding='utf-8') - logger.info("processing " + targetname + "... ", nonl=1) + logger.info(__("processing %s..."), targetname, nonl=1) doctree = self.assemble_doctree( docname, toctree_only, appendices=(self.config.texinfo_appendices or [])) - logger.info("writing... ", nonl=1) + logger.info(__("writing... "), nonl=1) self.post_process_images(doctree) docwriter = TexinfoWriter(self) settings = OptionParser( @@ -185,7 +185,7 @@ class TexinfoBuilder(Builder): settings.docname = docname doctree.settings = settings docwriter.write(doctree, destination) - logger.info("done") + logger.info(__("done")) def assemble_doctree(self, indexfile, toctree_only, appendices): # type: (unicode, bool, List[unicode]) -> nodes.Node @@ -212,7 +212,7 @@ class TexinfoBuilder(Builder): appendix['docname'] = docname largetree.append(appendix) logger.info('') - logger.info("resolving references...") + logger.info(__("resolving references...")) self.env.resolve_references(largetree, indexfile, self) # TODO: add support for external :ref:s for pendingnode in largetree.traverse(addnodes.pending_xref): @@ -234,7 +234,7 @@ class TexinfoBuilder(Builder): # type: () -> None self.copy_image_files() - logger.info(bold('copying Texinfo support files... '), nonl=True) + logger.info(bold(__('copying Texinfo support files... ')), nonl=True) # copy Makefile fn = path.join(self.outdir, 'Makefile') logger.info(fn, nonl=1) @@ -242,14 +242,14 @@ class TexinfoBuilder(Builder): with open(fn, 'w') as mkfile: mkfile.write(TEXINFO_MAKEFILE) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", fn, err) - logger.info(' done') + logger.warning(__("error writing file %s: %s"), fn, err) + logger.info(__(' done')) def copy_image_files(self): # type: () -> None if self.images: stringify_func = ImageAdapter(self.app.env).get_original_image_uri - for src in status_iterator(self.images, 'copying images... ', "brown", + for src in status_iterator(self.images, __('copying images... '), "brown", len(self.images), self.app.verbosity, stringify_func=stringify_func): dest = self.images[src] @@ -257,7 +257,7 @@ class TexinfoBuilder(Builder): copy_asset_file(path.join(self.srcdir, src), path.join(self.outdir, dest)) except Exception as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) diff --git a/sphinx/builders/text.py b/sphinx/builders/text.py index babb6ccee..81209d165 100644 --- a/sphinx/builders/text.py +++ b/sphinx/builders/text.py @@ -15,6 +15,7 @@ from os import path from docutils.io import StringOutput from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import ensuredir, os_path from sphinx.writers.text import TextWriter, TextTranslator @@ -31,7 +32,7 @@ logger = logging.getLogger(__name__) class TextBuilder(Builder): name = 'text' format = 'text' - epilog = 'The text files are in %(outdir)s.' + epilog = __('The text files are in %(outdir)s.') out_suffix = '.txt' allow_parallel = True @@ -84,7 +85,7 @@ class TextBuilder(Builder): with codecs.open(outfilename, 'w', 'utf-8') as f: # type: ignore f.write(self.writer.output) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", outfilename, err) + logger.warning(__("error writing file %s: %s"), outfilename, err) def finish(self): # type: () -> None diff --git a/sphinx/builders/xml.py b/sphinx/builders/xml.py index 80d7723aa..6198532c9 100644 --- a/sphinx/builders/xml.py +++ b/sphinx/builders/xml.py @@ -17,6 +17,7 @@ from docutils.io import StringOutput from docutils.writers.docutils_xml import XMLTranslator from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import ensuredir, os_path from sphinx.writers.xml import XMLWriter, PseudoXMLWriter @@ -35,7 +36,7 @@ class XMLBuilder(Builder): """ name = 'xml' format = 'xml' - epilog = 'The XML files are in %(outdir)s.' + epilog = __('The XML files are in %(outdir)s.') out_suffix = '.xml' allow_parallel = True @@ -97,7 +98,7 @@ class XMLBuilder(Builder): with codecs.open(outfilename, 'w', 'utf-8') as f: # type: ignore f.write(self.writer.output) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", outfilename, err) + logger.warning(__("error writing file %s: %s"), outfilename, err) def finish(self): # type: () -> None @@ -110,7 +111,7 @@ class PseudoXMLBuilder(XMLBuilder): """ name = 'pseudoxml' format = 'pseudoxml' - epilog = 'The pseudo-XML files are in %(outdir)s.' + epilog = __('The pseudo-XML files are in %(outdir)s.') out_suffix = '.pseudoxml' diff --git a/sphinx/cmd/build.py b/sphinx/cmd/build.py index c0c31ae67..53e93fb99 100644 --- a/sphinx/cmd/build.py +++ b/sphinx/cmd/build.py @@ -8,30 +8,309 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ +from __future__ import print_function +import argparse +import locale +import multiprocessing +import os import sys +import traceback + +from docutils.utils import SystemMessage +from six import text_type, binary_type + +import sphinx.locale +from sphinx import __display_version__, package_dir +from sphinx.application import Sphinx +from sphinx.errors import SphinxError +from sphinx.locale import __ +from sphinx.util import Tee, format_exception_cut_frames, save_traceback +from sphinx.util.console import red, nocolor, color_terminal # type: ignore +from sphinx.util.docutils import docutils_namespace, patch_docutils +from sphinx.util.pycompat import terminal_safe if False: # For type annotation - from typing import List # NOQA + from typing import Any, IO, List, Union # NOQA -def build_main(argv=sys.argv[1:]): - # type: (List[str]) -> int - """Sphinx build "main" command-line entry.""" - from sphinx import cmdline - return cmdline.main(argv) # type: ignore +def handle_exception(app, args, exception, stderr=sys.stderr): + # type: (Sphinx, Any, Union[Exception, KeyboardInterrupt], IO) -> None + if args.pdb: + import pdb + print(red(__('Exception occurred while building, starting debugger:')), + file=stderr) + traceback.print_exc() + pdb.post_mortem(sys.exc_info()[2]) + else: + print(file=stderr) + if args.verbosity or args.traceback: + traceback.print_exc(None, stderr) + print(file=stderr) + if isinstance(exception, KeyboardInterrupt): + print(__('interrupted!'), file=stderr) + elif isinstance(exception, SystemMessage): + print(red(__('reST markup error:')), file=stderr) + print(terminal_safe(exception.args[0]), file=stderr) + elif isinstance(exception, SphinxError): + print(red('%s:' % exception.category), file=stderr) + print(terminal_safe(text_type(exception)), file=stderr) + elif isinstance(exception, UnicodeError): + print(red(__('Encoding error:')), file=stderr) + print(terminal_safe(text_type(exception)), file=stderr) + tbpath = save_traceback(app) + print(red(__('The full traceback has been saved in %s, if you want ' + 'to report the issue to the developers.') % tbpath), + file=stderr) + elif isinstance(exception, RuntimeError) and 'recursion depth' in str(exception): + print(red(__('Recursion error:')), file=stderr) + print(terminal_safe(text_type(exception)), file=stderr) + print(file=stderr) + print(__('This can happen with very large or deeply nested source ' + 'files. You can carefully increase the default Python ' + 'recursion limit of 1000 in conf.py with e.g.:'), file=stderr) + print(__(' import sys; sys.setrecursionlimit(1500)'), file=stderr) + else: + print(red(__('Exception occurred:')), file=stderr) + print(format_exception_cut_frames().rstrip(), file=stderr) + tbpath = save_traceback(app) + print(red(__('The full traceback has been saved in %s, if you ' + 'want to report the issue to the developers.') % tbpath), + file=stderr) + print(__('Please also report this if it was a user error, so ' + 'that a better error message can be provided next time.'), + file=stderr) + print(__('A bug report can be filed in the tracker at ' + '<https://github.com/sphinx-doc/sphinx/issues>. Thanks!'), + file=stderr) + + +def jobs_argument(value): + # type: (str) -> int + """ + Special type to handle 'auto' flags passed to 'sphinx-build' via -j flag. Can + be expanded to handle other special scaling requests, such as setting job count + to cpu_count. + """ + if value == 'auto': + return multiprocessing.cpu_count() + else: + jobs = int(value) + if jobs <= 0: + raise argparse.ArgumentTypeError(__('job number should be a positive number')) + else: + return jobs + + +def get_parser(): + # type: () -> argparse.ArgumentParser + parser = argparse.ArgumentParser( + usage='%(prog)s [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]', + epilog=__('For more information, visit <http://sphinx-doc.org/>.'), + description=__(""" +Generate documentation from source files. +sphinx-build generates documentation from the files in SOURCEDIR and places it +in OUTPUTDIR. It looks for 'conf.py' in SOURCEDIR for the configuration +settings. The 'sphinx-quickstart' tool may be used to generate template files, +including 'conf.py' -def make_main(argv=sys.argv[1:]): - # type: (List[str]) -> int +sphinx-build can create documentation in different formats. A format is +selected by specifying the builder name on the command line; it defaults to +HTML. Builders can also perform other tasks related to documentation +processing. + +By default, everything that is outdated is built. Output only for selected +files can be built by specifying individual filenames. +""")) + + parser.add_argument('--version', action='version', dest='show_version', + version='%%(prog)s %s' % __display_version__) + + parser.add_argument('sourcedir', + help=__('path to documentation source files')) + parser.add_argument('outputdir', + help=__('path to output directory')) + parser.add_argument('filenames', nargs='*', + help=__('a list of specific files to rebuild. Ignored ' + 'if -a is specified')) + + group = parser.add_argument_group(__('general options')) + group.add_argument('-b', metavar='BUILDER', dest='builder', + default='html', + help=__('builder to use (default: html)')) + group.add_argument('-a', action='store_true', dest='force_all', + help=__('write all files (default: only write new and ' + 'changed files)')) + group.add_argument('-E', action='store_true', dest='freshenv', + help=__('don\'t use a saved environment, always read ' + 'all files')) + group.add_argument('-d', metavar='PATH', dest='doctreedir', + help=__('path for the cached environment and doctree ' + 'files (default: OUTPUTDIR/.doctrees)')) + group.add_argument('-j', metavar='N', default=1, type=jobs_argument, dest='jobs', + help=__('build in parallel with N processes where ' + 'possible (special value "auto" will set N to cpu-count)')) + group = parser.add_argument_group('build configuration options') + group.add_argument('-c', metavar='PATH', dest='confdir', + help=__('path where configuration file (conf.py) is ' + 'located (default: same as SOURCEDIR)')) + group.add_argument('-C', action='store_true', dest='noconfig', + help=__('use no config file at all, only -D options')) + group.add_argument('-D', metavar='setting=value', action='append', + dest='define', default=[], + help=__('override a setting in configuration file')) + group.add_argument('-A', metavar='name=value', action='append', + dest='htmldefine', default=[], + help=__('pass a value into HTML templates')) + group.add_argument('-t', metavar='TAG', action='append', + dest='tags', default=[], + help=__('define tag: include "only" blocks with TAG')) + group.add_argument('-n', action='store_true', dest='nitpicky', + help=__('nit-picky mode, warn about all missing ' + 'references')) + + group = parser.add_argument_group(__('console output options')) + group.add_argument('-v', action='count', dest='verbosity', default=0, + help=__('increase verbosity (can be repeated)')) + group.add_argument('-q', action='store_true', dest='quiet', + help=__('no output on stdout, just warnings on stderr')) + group.add_argument('-Q', action='store_true', dest='really_quiet', + help=__('no output at all, not even warnings')) + group.add_argument('--color', action='store_const', const='yes', + default='auto', + help=__('do emit colored output (default: auto-detect)')) + group.add_argument('-N', '--no-color', dest='color', action='store_const', + const='no', + help=__('do not emit colored output (default: ' + 'auto-detect)')) + group.add_argument('-w', metavar='FILE', dest='warnfile', + help=__('write warnings (and errors) to given file')) + group.add_argument('-W', action='store_true', dest='warningiserror', + help=__('turn warnings into errors')) + group.add_argument('-T', action='store_true', dest='traceback', + help=__('show full traceback on exception')) + group.add_argument('-P', action='store_true', dest='pdb', + help=__('run Pdb on exception')) + + return parser + + +def make_main(argv=sys.argv[1:]): # type: ignore + # type: (List[unicode]) -> int """Sphinx build "make mode" entry.""" from sphinx import make_mode - return make_mode.run_make_mode(argv[1:]) # type: ignore + return make_mode.run_make_mode(argv[1:]) + + +def build_main(argv=sys.argv[1:]): # type: ignore + # type: (List[unicode]) -> int + """Sphinx build "main" command-line entry.""" + + parser = get_parser() + args = parser.parse_args(argv) + + if args.noconfig: + args.confdir = None + elif not args.confdir: + args.confdir = args.sourcedir + + if not args.doctreedir: + args.doctreedir = os.path.join(args.sourcedir, '.doctrees') + + # handle remaining filename arguments + filenames = args.filenames + missing_files = [] + for filename in filenames: + if not os.path.isfile(filename): + missing_files.append(filename) + if missing_files: + parser.error(__('cannot find files %r') % missing_files) + + # likely encoding used for command-line arguments + try: + locale = __import__('locale') # due to submodule of the same name + likely_encoding = locale.getpreferredencoding() + except Exception: + likely_encoding = None + + if args.force_all and filenames: + parser.error(__('cannot combine -a option and filenames')) + + if args.color == 'no' or (args.color == 'auto' and not color_terminal()): + nocolor() + + status = sys.stdout + warning = sys.stderr + error = sys.stderr + + if args.quiet: + status = None + + if args.really_quiet: + status = warning = None + + if warning and args.warnfile: + try: + warnfp = open(args.warnfile, 'w') + except Exception as exc: + parser.error(__('cannot open warning file %r: %s') % ( + args.warnfile, exc)) + warning = Tee(warning, warnfp) # type: ignore + error = warning + + confoverrides = {} + for val in args.define: + try: + key, val = val.split('=', 1) + except ValueError: + parser.error(__('-D option argument must be in the form name=value')) + if likely_encoding and isinstance(val, binary_type): + try: + val = val.decode(likely_encoding) + except UnicodeError: + pass + confoverrides[key] = val + + for val in args.htmldefine: + try: + key, val = val.split('=') + except ValueError: + parser.error(__('-A option argument must be in the form name=value')) + try: + val = int(val) + except ValueError: + if likely_encoding and isinstance(val, binary_type): + try: + val = val.decode(likely_encoding) + except UnicodeError: + pass + confoverrides['html_context.%s' % key] = val + + if args.nitpicky: + confoverrides['nitpicky'] = True + + app = None + try: + confdir = args.confdir or args.sourcedir + with patch_docutils(confdir), docutils_namespace(): + app = Sphinx(args.sourcedir, args.confdir, args.outputdir, + args.doctreedir, args.builder, confoverrides, status, + warning, args.freshenv, args.warningiserror, + args.tags, args.verbosity, args.jobs) + app.build(args.force_all, filenames) + return app.statuscode + except (Exception, KeyboardInterrupt) as exc: + handle_exception(app, args, exc, error) + return 2 + +def main(argv=sys.argv[1:]): # type: ignore + # type: (List[unicode]) -> int + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') -def main(argv=sys.argv[1:]): - # type: (List[str]) -> int if sys.argv[1:2] == ['-M']: return make_main(argv) else: @@ -39,4 +318,4 @@ def main(argv=sys.argv[1:]): if __name__ == '__main__': - sys.exit(main(sys.argv[1:])) + sys.exit(main(sys.argv[1:])) # type: ignore diff --git a/sphinx/cmd/quickstart.py b/sphinx/cmd/quickstart.py index 68718eeaf..d6c7fa22a 100644 --- a/sphinx/cmd/quickstart.py +++ b/sphinx/cmd/quickstart.py @@ -12,6 +12,7 @@ from __future__ import absolute_import from __future__ import print_function import argparse +import locale import os import re import sys @@ -35,7 +36,9 @@ from six import PY2, PY3, text_type, binary_type from six.moves import input from six.moves.urllib.parse import quote as urlquote +import sphinx.locale from sphinx import __display_version__, package_dir +from sphinx.locale import __ from sphinx.util import texescape from sphinx.util.console import ( # type: ignore purple, bold, red, turquoise, nocolor, color_terminal @@ -50,18 +53,18 @@ if False: TERM_ENCODING = getattr(sys.stdin, 'encoding', None) EXTENSIONS = OrderedDict([ - ('autodoc', 'automatically insert docstrings from modules'), - ('doctest', 'automatically test code snippets in doctest blocks'), - ('intersphinx', 'link between Sphinx documentation of different projects'), - ('todo', 'write "todo" entries that can be shown or hidden on build'), - ('coverage', 'checks for documentation coverage'), - ('imgmath', 'include math, rendered as PNG or SVG images'), - ('mathjax', 'include math, rendered in the browser by MathJax'), - ('ifconfig', 'conditional inclusion of content based on config values'), + ('autodoc', __('automatically insert docstrings from modules')), + ('doctest', __('automatically test code snippets in doctest blocks')), + ('intersphinx', __('link between Sphinx documentation of different projects')), + ('todo', __('write "todo" entries that can be shown or hidden on build')), + ('coverage', __('checks for documentation coverage')), + ('imgmath', __('include math, rendered as PNG or SVG images')), + ('mathjax', __('include math, rendered in the browser by MathJax')), + ('ifconfig', __('conditional inclusion of content based on config values')), ('viewcode', - 'include links to the source code of documented Python objects'), + __('include links to the source code of documented Python objects')), ('githubpages', - 'create .nojekyll file to publish the document on GitHub pages'), + __('create .nojekyll file to publish the document on GitHub pages')), ]) DEFAULTS = { @@ -94,7 +97,7 @@ def is_path(x): # type: (unicode) -> unicode x = path.expanduser(x) if path.exists(x) and not path.isdir(x): - raise ValidationError("Please enter a valid path name.") + raise ValidationError(__("Please enter a valid path name.")) return x @@ -106,7 +109,7 @@ def allow_empty(x): def nonempty(x): # type: (unicode) -> unicode if not x: - raise ValidationError("Please enter some text.") + raise ValidationError(__("Please enter some text.")) return x @@ -115,7 +118,7 @@ def choice(*l): def val(x): # type: (unicode) -> unicode if x not in l: - raise ValidationError('Please enter one of %s.' % ', '.join(l)) + raise ValidationError(__('Please enter one of %s.') % ', '.join(l)) return x return val @@ -123,15 +126,15 @@ def choice(*l): def boolean(x): # type: (unicode) -> bool if x.upper() not in ('Y', 'YES', 'N', 'NO'): - raise ValidationError("Please enter either 'y' or 'n'.") + raise ValidationError(__("Please enter either 'y' or 'n'.")) return x.upper() in ('Y', 'YES') def suffix(x): # type: (unicode) -> unicode if not (x[0:1] == '.' and len(x) > 1): - raise ValidationError("Please enter a file suffix, " - "e.g. '.rst' or '.txt'.") + raise ValidationError(__("Please enter a file suffix, " + "e.g. '.rst' or '.txt'.")) return x @@ -153,9 +156,9 @@ def term_decode(text): if text.decode('ascii', 'replace').encode('ascii', 'replace') == text: return text.decode('ascii') - print(turquoise('* Note: non-ASCII characters entered ' - 'and terminal encoding unknown -- assuming ' - 'UTF-8 or Latin-1.')) + print(turquoise(__('* Note: non-ASCII characters entered ' + 'and terminal encoding unknown -- assuming ' + 'UTF-8 or Latin-1.'))) try: return text.decode('utf-8') except UnicodeDecodeError: @@ -176,9 +179,9 @@ def do_prompt(text, default=None, validator=nonempty): if TERM_ENCODING: prompt = prompt.encode(TERM_ENCODING) else: - print(turquoise('* Note: non-ASCII default value provided ' - 'and terminal encoding unknown -- assuming ' - 'UTF-8 or Latin-1.')) + print(turquoise(__('* Note: non-ASCII default value provided ' + 'and terminal encoding unknown -- assuming ' + 'UTF-8 or Latin-1.'))) try: prompt = prompt.encode('utf-8') except UnicodeEncodeError: @@ -243,110 +246,110 @@ def ask_user(d): * batchfile: make command file """ - print(bold('Welcome to the Sphinx %s quickstart utility.') % __display_version__) - print(''' + print(bold(__('Welcome to the Sphinx %s quickstart utility.')) % __display_version__) + print(__(''' Please enter values for the following settings (just press Enter to -accept a default value, if one is given in brackets).''') +accept a default value, if one is given in brackets).''')) if 'path' in d: - print(bold(''' -Selected root path: %s''' % d['path'])) + print(bold(__(''' +Selected root path: %s''') % d['path'])) else: - print(''' -Enter the root path for documentation.''') - d['path'] = do_prompt('Root path for the documentation', '.', is_path) + print(__(''' +Enter the root path for documentation.''')) + d['path'] = do_prompt(__('Root path for the documentation'), '.', is_path) while path.isfile(path.join(d['path'], 'conf.py')) or \ path.isfile(path.join(d['path'], 'source', 'conf.py')): print() - print(bold('Error: an existing conf.py has been found in the ' - 'selected root path.')) - print('sphinx-quickstart will not overwrite existing Sphinx projects.') + print(bold(__('Error: an existing conf.py has been found in the ' + 'selected root path.'))) + print(__('sphinx-quickstart will not overwrite existing Sphinx projects.')) print() - d['path'] = do_prompt('Please enter a new root path (or just Enter ' - 'to exit)', '', is_path) + d['path'] = do_prompt(__('Please enter a new root path (or just Enter ' + 'to exit)'), '', is_path) if not d['path']: sys.exit(1) if 'sep' not in d: - print(''' + print(__(''' You have two options for placing the build directory for Sphinx output. Either, you use a directory "_build" within the root path, or you separate -"source" and "build" directories within the root path.''') - d['sep'] = do_prompt('Separate source and build directories (y/n)', +"source" and "build" directories within the root path.''')) + d['sep'] = do_prompt(__('Separate source and build directories (y/n)'), 'n', boolean) if 'dot' not in d: - print(''' + print(__(''' Inside the root directory, two more directories will be created; "_templates" for custom HTML templates and "_static" for custom stylesheets and other static -files. You can enter another prefix (such as ".") to replace the underscore.''') - d['dot'] = do_prompt('Name prefix for templates and static dir', '_', ok) +files. You can enter another prefix (such as ".") to replace the underscore.''')) + d['dot'] = do_prompt(__('Name prefix for templates and static dir'), '_', ok) if 'project' not in d: - print(''' -The project name will occur in several places in the built documentation.''') - d['project'] = do_prompt('Project name') + print(__(''' +The project name will occur in several places in the built documentation.''')) + d['project'] = do_prompt(__('Project name')) if 'author' not in d: - d['author'] = do_prompt('Author name(s)') + d['author'] = do_prompt(__('Author name(s)')) if 'version' not in d: - print(''' + print(__(''' Sphinx has the notion of a "version" and a "release" for the software. Each version can have multiple releases. For example, for Python the version is something like 2.5 or 3.0, while the release is something like 2.5.1 or 3.0a1. If you don't need this dual structure, -just set both to the same value.''') - d['version'] = do_prompt('Project version', '', allow_empty) +just set both to the same value.''')) + d['version'] = do_prompt(__('Project version'), '', allow_empty) if 'release' not in d: - d['release'] = do_prompt('Project release', d['version'], allow_empty) + d['release'] = do_prompt(__('Project release'), d['version'], allow_empty) if 'language' not in d: - print(''' + print(__(''' If the documents are to be written in a language other than English, you can select a language here by its language code. Sphinx will then translate text that it generates into that language. For a list of supported codes, see -http://sphinx-doc.org/config.html#confval-language.''') - d['language'] = do_prompt('Project language', 'en') +http://sphinx-doc.org/config.html#confval-language.''')) + d['language'] = do_prompt(__('Project language'), 'en') if d['language'] == 'en': d['language'] = None if 'suffix' not in d: - print(''' + print(__(''' The file name suffix for source files. Commonly, this is either ".txt" -or ".rst". Only files with this suffix are considered documents.''') - d['suffix'] = do_prompt('Source file suffix', '.rst', suffix) +or ".rst". Only files with this suffix are considered documents.''')) + d['suffix'] = do_prompt(__('Source file suffix'), '.rst', suffix) if 'master' not in d: - print(''' + print(__(''' One document is special in that it is considered the top node of the "contents tree", that is, it is the root of the hierarchical structure of the documents. Normally, this is "index", but if your "index" -document is a custom template, you can also set this to another filename.''') - d['master'] = do_prompt('Name of your master document (without suffix)', +document is a custom template, you can also set this to another filename.''')) + d['master'] = do_prompt(__('Name of your master document (without suffix)'), 'index') while path.isfile(path.join(d['path'], d['master'] + d['suffix'])) or \ path.isfile(path.join(d['path'], 'source', d['master'] + d['suffix'])): print() - print(bold('Error: the master file %s has already been found in the ' - 'selected root path.' % (d['master'] + d['suffix']))) - print('sphinx-quickstart will not overwrite the existing file.') + print(bold(__('Error: the master file %s has already been found in the ' + 'selected root path.') % (d['master'] + d['suffix']))) + print(__('sphinx-quickstart will not overwrite the existing file.')) print() - d['master'] = do_prompt('Please enter a new file name, or rename the ' - 'existing file and press Enter', d['master']) + d['master'] = do_prompt(__('Please enter a new file name, or rename the ' + 'existing file and press Enter'), d['master']) if 'epub' not in d: - print(''' -Sphinx can also add configuration for epub output:''') - d['epub'] = do_prompt('Do you want to use the epub builder (y/n)', + print(__(''' +Sphinx can also add configuration for epub output:''')) + d['epub'] = do_prompt(__('Do you want to use the epub builder (y/n)'), 'n', boolean) if 'extensions' not in d: - print('Indicate which of the following Sphinx extensions should be ' - 'enabled:') + print(__('Indicate which of the following Sphinx extensions should be ' + 'enabled:')) d['extensions'] = [] for name, description in EXTENSIONS.items(): if do_prompt('%s: %s (y/n)' % (name, description), 'n', boolean): @@ -355,19 +358,19 @@ Sphinx can also add configuration for epub output:''') # Handle conflicting options if set(['sphinx.ext.imgmath', 'sphinx.ext.mathjax']).issubset( d['extensions']): - print('Note: imgmath and mathjax cannot be enabled at the same ' - 'time. imgmath has been deselected.') + print(__('Note: imgmath and mathjax cannot be enabled at the same ' + 'time. imgmath has been deselected.')) d['extensions'].remove('sphinx.ext.imgmath') if 'makefile' not in d: - print(''' + print(__(''' A Makefile and a Windows command file can be generated for you so that you only have to run e.g. `make html' instead of invoking sphinx-build -directly.''') - d['makefile'] = do_prompt('Create Makefile? (y/n)', 'y', boolean) +directly.''')) + d['makefile'] = do_prompt(__('Create Makefile? (y/n)'), 'y', boolean) if 'batchfile' not in d: - d['batchfile'] = do_prompt('Create Windows command file? (y/n)', + d['batchfile'] = do_prompt(__('Create Windows command file? (y/n)'), 'y', boolean) print() @@ -429,12 +432,12 @@ def generate(d, overwrite=True, silent=False, templatedir=None): # type: (unicode, unicode, unicode) -> None if overwrite or not path.isfile(fpath): if 'quiet' not in d: - print('Creating file %s.' % fpath) + print(__('Creating file %s.') % fpath) with open(fpath, 'wt', encoding='utf-8', newline=newline) as f: f.write(content) else: if 'quiet' not in d: - print('File %s already exists, skipping.' % fpath) + print(__('File %s already exists, skipping.') % fpath) conf_path = os.path.join(templatedir, 'conf.py_t') if templatedir else None if not conf_path or not path.isfile(conf_path): @@ -470,18 +473,18 @@ def generate(d, overwrite=True, silent=False, templatedir=None): if silent: return print() - print(bold('Finished: An initial directory structure has been created.')) - print(''' + print(bold(__('Finished: An initial directory structure has been created.'))) + print(__(''' You should now populate your master file %s and create other documentation -source files. ''' % masterfile + ((d['makefile'] or d['batchfile']) and '''\ +source files. ''') % masterfile + ((d['makefile'] or d['batchfile']) and __('''\ Use the Makefile to build the docs, like so: make builder -''' or '''\ +''') or __('''\ Use the sphinx-build command to build the docs, like so: sphinx-build -b builder %s %s -''' % (srcdir, builddir)) + '''\ +''') % (srcdir, builddir)) + __('''\ where "builder" is one of the supported builders, e.g. html, latex or linkcheck. -''') +''')) def valid_dir(d): @@ -518,86 +521,89 @@ def get_parser(): # type: () -> argparse.ArgumentParser parser = argparse.ArgumentParser( usage='%(prog)s [OPTIONS] <PROJECT_DIR>', - epilog="For more information, visit <http://sphinx-doc.org/>.", - description=""" + epilog=__("For more information, visit <http://sphinx-doc.org/>."), + description=__(""" Generate required files for a Sphinx project. sphinx-quickstart is an interactive tool that asks some questions about your project and then generates a complete documentation directory and sample Makefile to be used with sphinx-build. -""") +""")) parser.add_argument('-q', '--quiet', action='store_true', dest='quiet', default=False, - help='quiet mode') + help=__('quiet mode')) parser.add_argument('--version', action='version', dest='show_version', version='%%(prog)s %s' % __display_version__) parser.add_argument('path', metavar='PROJECT_DIR', default='.', nargs='?', - help='output path') + help=__('output path')) - group = parser.add_argument_group('Structure options') + group = parser.add_argument_group(__('Structure options')) group.add_argument('--sep', action='store_true', - help='if specified, separate source and build dirs') + help=__('if specified, separate source and build dirs')) group.add_argument('--dot', metavar='DOT', - help='replacement for dot in _templates etc.') + help=__('replacement for dot in _templates etc.')) - group = parser.add_argument_group('Project basic options') + group = parser.add_argument_group(__('Project basic options')) group.add_argument('-p', '--project', metavar='PROJECT', dest='project', - help='project name') + help=__('project name')) group.add_argument('-a', '--author', metavar='AUTHOR', dest='author', - help='author names') + help=__('author names')) group.add_argument('-v', metavar='VERSION', dest='version', default='', - help='version of project') + help=__('version of project')) group.add_argument('-r', '--release', metavar='RELEASE', dest='release', - help='release of project') + help=__('release of project')) group.add_argument('-l', '--language', metavar='LANGUAGE', dest='language', - help='document language') + help=__('document language')) group.add_argument('--suffix', metavar='SUFFIX', - help='source file suffix') + help=__('source file suffix')) group.add_argument('--master', metavar='MASTER', - help='master document name') + help=__('master document name')) group.add_argument('--epub', action='store_true', default=False, - help='use epub') + help=__('use epub')) - group = parser.add_argument_group('Extension options') + group = parser.add_argument_group(__('Extension options')) for ext in EXTENSIONS: group.add_argument('--ext-%s' % ext, action='append_const', const='sphinx.ext.%s' % ext, dest='extensions', - help='enable %s extension' % ext) + help=__('enable %s extension') % ext) group.add_argument('--extensions', metavar='EXTENSIONS', dest='extensions', - action='append', help='enable arbitrary extensions') + action='append', help=__('enable arbitrary extensions')) - group = parser.add_argument_group('Makefile and Batchfile creation') + group = parser.add_argument_group(__('Makefile and Batchfile creation')) group.add_argument('--makefile', action='store_true', dest='makefile', - help='create makefile') + help=__('create makefile')) group.add_argument('--no-makefile', action='store_false', dest='makefile', - help='do not create makefile') + help=__('do not create makefile')) group.add_argument('--batchfile', action='store_true', dest='batchfile', - help='create batchfile') + help=__('create batchfile')) group.add_argument('--no-batchfile', action='store_false', dest='batchfile', - help='do not create batchfile') + help=__('do not create batchfile')) group.add_argument('-m', '--use-make-mode', action='store_true', dest='make_mode', default=True, - help='use make-mode for Makefile/make.bat') + help=__('use make-mode for Makefile/make.bat')) group.add_argument('-M', '--no-use-make-mode', action='store_false', dest='make_mode', - help='do not use make-mode for Makefile/make.bat') + help=__('do not use make-mode for Makefile/make.bat')) - group = parser.add_argument_group('Project templating') + group = parser.add_argument_group(__('Project templating')) group.add_argument('-t', '--templatedir', metavar='TEMPLATEDIR', dest='templatedir', - help='template directory for template files') + help=__('template directory for template files')) group.add_argument('-d', metavar='NAME=VALUE', action='append', dest='variables', - help='define a template variable') + help=__('define a template variable')) return parser def main(argv=sys.argv[1:]): # type: (List[str]) -> int + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') + if not color_terminal(): nocolor() @@ -615,8 +621,8 @@ def main(argv=sys.argv[1:]): try: if 'quiet' in d: if not set(['project', 'author']).issubset(d): - print('''"quiet" is specified, but any of "project" or \ -"author" is not specified.''') + print(__('''"quiet" is specified, but any of "project" or \ +"author" is not specified.''')) return 1 if set(['quiet', 'project', 'author']).issubset(d): @@ -629,10 +635,10 @@ def main(argv=sys.argv[1:]): if not valid_dir(d): print() - print(bold('Error: specified path is not a directory, or sphinx' - ' files already exist.')) - print('sphinx-quickstart only generate into a empty directory.' - ' Please specify a new root path.') + print(bold(__('Error: specified path is not a directory, or sphinx' + ' files already exist.'))) + print(__('sphinx-quickstart only generate into a empty directory.' + ' Please specify a new root path.')) return 1 else: ask_user(d) @@ -658,7 +664,7 @@ def main(argv=sys.argv[1:]): name, value = variable.split('=') d[name] = value except ValueError: - print('Invalid template variable: %s' % variable) + print(__('Invalid template variable: %s') % variable) generate(d, templatedir=args.templatedir) return 0 diff --git a/sphinx/cmdline.py b/sphinx/cmdline.py index 4ef647846..10835d2e7 100644 --- a/sphinx/cmdline.py +++ b/sphinx/cmdline.py @@ -8,301 +8,45 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ +from __future__ import absolute_import from __future__ import print_function -import argparse -import multiprocessing import sys -import traceback -from os import path +import warnings -from docutils.utils import SystemMessage -from six import text_type, binary_type - -from sphinx import __display_version__ -from sphinx.application import Sphinx -from sphinx.errors import SphinxError -from sphinx.util import Tee, format_exception_cut_frames, save_traceback -from sphinx.util.console import red, nocolor, color_terminal # type: ignore -from sphinx.util.docutils import docutils_namespace, patch_docutils -from sphinx.util.osutil import abspath, fs_encoding -from sphinx.util.pycompat import terminal_safe +from sphinx.cmd import build +from sphinx.deprecation import RemovedInSphinx30Warning if False: # For type annotation + import argparse # NOQA from typing import Any, IO, List, Union # NOQA + from sphinx.application import Sphinx # NOQA def handle_exception(app, args, exception, stderr=sys.stderr): # type: (Sphinx, Any, Union[Exception, KeyboardInterrupt], IO) -> None - if args.pdb: - import pdb - print(red('Exception occurred while building, starting debugger:'), - file=stderr) - traceback.print_exc() - pdb.post_mortem(sys.exc_info()[2]) - else: - print(file=stderr) - if args.verbosity or args.traceback: - traceback.print_exc(None, stderr) - print(file=stderr) - if isinstance(exception, KeyboardInterrupt): - print('interrupted!', file=stderr) - elif isinstance(exception, SystemMessage): - print(red('reST markup error:'), file=stderr) - print(terminal_safe(exception.args[0]), file=stderr) - elif isinstance(exception, SphinxError): - print(red('%s:' % exception.category), file=stderr) - print(terminal_safe(text_type(exception)), file=stderr) - elif isinstance(exception, UnicodeError): - print(red('Encoding error:'), file=stderr) - print(terminal_safe(text_type(exception)), file=stderr) - tbpath = save_traceback(app) - print(red('The full traceback has been saved in %s, if you want ' - 'to report the issue to the developers.' % tbpath), - file=stderr) - elif isinstance(exception, RuntimeError) and 'recursion depth' in str(exception): - print(red('Recursion error:'), file=stderr) - print(terminal_safe(text_type(exception)), file=stderr) - print(file=stderr) - print('This can happen with very large or deeply nested source ' - 'files. You can carefully increase the default Python ' - 'recursion limit of 1000 in conf.py with e.g.:', file=stderr) - print(' import sys; sys.setrecursionlimit(1500)', file=stderr) - else: - print(red('Exception occurred:'), file=stderr) - print(format_exception_cut_frames().rstrip(), file=stderr) - tbpath = save_traceback(app) - print(red('The full traceback has been saved in %s, if you ' - 'want to report the issue to the developers.' % tbpath), - file=stderr) - print('Please also report this if it was a user error, so ' - 'that a better error message can be provided next time.', - file=stderr) - print('A bug report can be filed in the tracker at ' - '<https://github.com/sphinx-doc/sphinx/issues>. Thanks!', - file=stderr) + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + build.handle_exception(app, args, exception, stderr) def jobs_argument(value): # type: (str) -> int - """ - Special type to handle 'auto' flags passed to 'sphinx-build' via -j flag. Can - be expanded to handle other special scaling requests, such as setting job count - to cpu_count. - """ - if value == 'auto': - return multiprocessing.cpu_count() - else: - jobs = int(value) - if jobs <= 0: - raise argparse.ArgumentTypeError('job number should be a positive number') - else: - return jobs + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + return build.jobs_argument(value) def get_parser(): # type: () -> argparse.ArgumentParser - parser = argparse.ArgumentParser( - usage='%(prog)s [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]', - epilog='For more information, visit <http://sphinx-doc.org/>.', - description=""" -Generate documentation from source files. - -sphinx-build generates documentation from the files in SOURCEDIR and places it -in OUTPUTDIR. It looks for 'conf.py' in SOURCEDIR for the configuration -settings. The 'sphinx-quickstart' tool may be used to generate template files, -including 'conf.py' - -sphinx-build can create documentation in different formats. A format is -selected by specifying the builder name on the command line; it defaults to -HTML. Builders can also perform other tasks related to documentation -processing. - -By default, everything that is outdated is built. Output only for selected -files can be built by specifying individual filenames. -""") - - parser.add_argument('--version', action='version', dest='show_version', - version='%%(prog)s %s' % __display_version__) - - parser.add_argument('sourcedir', - help='path to documentation source files') - parser.add_argument('outputdir', - help='path to output directory') - parser.add_argument('filenames', nargs='*', - help='a list of specific files to rebuild. Ignored ' - 'if -a is specified') - - group = parser.add_argument_group('general options') - group.add_argument('-b', metavar='BUILDER', dest='builder', - default='html', - help='builder to use (default: html)') - group.add_argument('-a', action='store_true', dest='force_all', - help='write all files (default: only write new and ' - 'changed files)') - group.add_argument('-E', action='store_true', dest='freshenv', - help='don\'t use a saved environment, always read ' - 'all files') - group.add_argument('-d', metavar='PATH', dest='doctreedir', - help='path for the cached environment and doctree ' - 'files (default: OUTPUTDIR/.doctrees)') - group.add_argument('-j', metavar='N', default=1, type=jobs_argument, dest='jobs', - help='build in parallel with N processes where ' - 'possible (special value "auto" will set N to cpu-count)') - group = parser.add_argument_group('build configuration options') - group.add_argument('-c', metavar='PATH', dest='confdir', - help='path where configuration file (conf.py) is ' - 'located (default: same as SOURCEDIR)') - group.add_argument('-C', action='store_true', dest='noconfig', - help='use no config file at all, only -D options') - group.add_argument('-D', metavar='setting=value', action='append', - dest='define', default=[], - help='override a setting in configuration file') - group.add_argument('-A', metavar='name=value', action='append', - dest='htmldefine', default=[], - help='pass a value into HTML templates') - group.add_argument('-t', metavar='TAG', action='append', - dest='tags', default=[], - help='define tag: include "only" blocks with TAG') - group.add_argument('-n', action='store_true', dest='nitpicky', - help='nit-picky mode, warn about all missing ' - 'references') - - group = parser.add_argument_group('console output options') - group.add_argument('-v', action='count', dest='verbosity', default=0, - help='increase verbosity (can be repeated)') - group.add_argument('-q', action='store_true', dest='quiet', - help='no output on stdout, just warnings on stderr') - group.add_argument('-Q', action='store_true', dest='really_quiet', - help='no output at all, not even warnings') - group.add_argument('--color', action='store_const', const='yes', - default='auto', - help='do emit colored output (default: auto-detect)') - group.add_argument('-N', '--no-color', dest='color', action='store_const', - const='no', - help='do not emit colored output (default: ' - 'auto-detect)') - group.add_argument('-w', metavar='FILE', dest='warnfile', - help='write warnings (and errors) to given file') - group.add_argument('-W', action='store_true', dest='warningiserror', - help='turn warnings into errors') - group.add_argument('-T', action='store_true', dest='traceback', - help='show full traceback on exception') - group.add_argument('-P', action='store_true', dest='pdb', - help='run Pdb on exception') - - return parser + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + return build.get_parser() def main(argv=sys.argv[1:]): # type: ignore # type: (List[unicode]) -> int - - parser = get_parser() - args = parser.parse_args(argv) - - # get paths (first and second positional argument) - try: - srcdir = abspath(args.sourcedir) - confdir = abspath(args.confdir or srcdir) - if args.noconfig: - confdir = None - - if not path.isdir(srcdir): - parser.error('cannot find source directory (%s)' % srcdir) - if not args.noconfig and not path.isfile(path.join(confdir, 'conf.py')): - parser.error("config directory doesn't contain a conf.py file " - "(%s)" % confdir) - - outdir = abspath(args.outputdir) - if srcdir == outdir: - parser.error('source directory and destination directory are same') - except UnicodeError: - parser.error('multibyte filename not supported on this filesystem ' - 'encoding (%r)' % fs_encoding) - - # handle remaining filename arguments - filenames = args.filenames - missing_files = [] - for filename in filenames: - if not path.isfile(filename): - missing_files.append(filename) - if missing_files: - parser.error('cannot find files %r' % missing_files) - - # likely encoding used for command-line arguments - try: - locale = __import__('locale') # due to submodule of the same name - likely_encoding = locale.getpreferredencoding() - except Exception: - likely_encoding = None - - if args.force_all and filenames: - parser.error('cannot combine -a option and filenames') - - if args.color == 'no' or (args.color == 'auto' and not color_terminal()): - nocolor() - - doctreedir = abspath(args.doctreedir or path.join(outdir, '.doctrees')) - - status = sys.stdout - warning = sys.stderr - error = sys.stderr - - if args.quiet: - status = None - - if args.really_quiet: - status = warning = None - - if warning and args.warnfile: - try: - warnfp = open(args.warnfile, 'w') - except Exception as exc: - parser.error('cannot open warning file %r: %s' % ( - args.warnfile, exc)) - warning = Tee(warning, warnfp) # type: ignore - error = warning - - confoverrides = {} - for val in args.define: - try: - key, val = val.split('=', 1) - except ValueError: - parser.error('-D option argument must be in the form name=value') - if likely_encoding and isinstance(val, binary_type): - try: - val = val.decode(likely_encoding) - except UnicodeError: - pass - confoverrides[key] = val - - for val in args.htmldefine: - try: - key, val = val.split('=') - except ValueError: - parser.error('-A option argument must be in the form name=value') - try: - val = int(val) - except ValueError: - if likely_encoding and isinstance(val, binary_type): - try: - val = val.decode(likely_encoding) - except UnicodeError: - pass - confoverrides['html_context.%s' % key] = val - - if args.nitpicky: - confoverrides['nitpicky'] = True - - app = None - try: - with patch_docutils(), docutils_namespace(): - app = Sphinx(srcdir, confdir, outdir, doctreedir, args.builder, - confoverrides, status, warning, args.freshenv, - args.warningiserror, args.tags, args.verbosity, args.jobs) - app.build(args.force_all, filenames) - return app.statuscode - except (Exception, KeyboardInterrupt) as exc: - handle_exception(app, args, exc, error) - return 2 + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + return build.main(argv) diff --git a/sphinx/config.py b/sphinx/config.py index 1f80ab366..1184f1891 100644 --- a/sphinx/config.py +++ b/sphinx/config.py @@ -11,13 +11,19 @@ import re import traceback +import types +import warnings +from collections import OrderedDict from os import path, getenv from typing import Any, NamedTuple, Union -from six import PY2, PY3, iteritems, string_types, binary_type, text_type, integer_types +from six import ( + PY2, PY3, iteritems, string_types, binary_type, text_type, integer_types, class_types +) -from sphinx.errors import ConfigError -from sphinx.locale import l_, __ +from sphinx.deprecation import RemovedInSphinx30Warning +from sphinx.errors import ConfigError, ExtensionError +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.i18n import format_date from sphinx.util.osutil import cd @@ -25,27 +31,16 @@ from sphinx.util.pycompat import execfile_, NoneType if False: # For type annotation - from typing import Any, Callable, Dict, Iterable, Iterator, List, Tuple, Union # NOQA + from typing import Any, Callable, Dict, Generator, Iterator, List, Tuple, Union # NOQA + from sphinx.application import Sphinx # NOQA from sphinx.util.tags import Tags # NOQA logger = logging.getLogger(__name__) -nonascii_re = re.compile(br'[\x80-\xff]') +CONFIG_FILENAME = 'conf.py' +UNSERIALIZEABLE_TYPES = class_types + (types.ModuleType, types.FunctionType) copyright_year_re = re.compile(r'^((\d{4}-)?)(\d{4})(?=[ ,])') -CONFIG_SYNTAX_ERROR = "There is a syntax error in your configuration file: %s" -if PY3: - CONFIG_SYNTAX_ERROR += "\nDid you change the syntax from 2.x to 3.x?" -CONFIG_ERROR = "There is a programable error in your configuration file:\n\n%s" -CONFIG_EXIT_ERROR = "The configuration file (or one of the modules it imports) " \ - "called sys.exit()" -CONFIG_ENUM_WARNING = "The config value `{name}` has to be a one of {candidates}, " \ - "but `{current}` is given." -CONFIG_PERMITTED_TYPE_WARNING = "The config value `{name}' has type `{current.__name__}', " \ - "expected to {permitted}." -CONFIG_TYPE_WARNING = "The config value `{name}' has type `{current.__name__}', " \ - "defaults to `{default.__name__}'." - if PY3: unicode = str # special alias for static typing... @@ -54,6 +49,10 @@ ConfigValue = NamedTuple('ConfigValue', [('name', str), ('rebuild', Union[bool, unicode])]) +#: represents the config value accepts any type of value. +Any = object() + + class ENUM(object): """represents the config value should be a one of candidates. @@ -78,8 +77,15 @@ if PY2: class Config(object): - """ - Configuration file abstraction. + """Configuration file abstraction. + + The config object makes the values of all config values available as + attributes. + + It is exposed via the :py:attr:`sphinx.application.Application.config` and + :py:attr:`sphinx.environment.Environment.config` attributes. For example, + to get the value of :confval:`language`, use either ``app.config.language`` + or ``env.config.language``. """ # the values are: (default, what needs to be rebuilt if changed) @@ -89,86 +95,83 @@ class Config(object): config_values = dict( # general options - project = ('Python', 'env'), - copyright = ('', 'html'), - version = ('', 'env'), - release = ('', 'env'), - today = ('', 'env'), + project = ('Python', 'env', []), + author = ('unknown', 'env', []), + copyright = ('', 'html', []), + version = ('', 'env', []), + release = ('', 'env', []), + today = ('', 'env', []), # the real default is locale-dependent today_fmt = (None, 'env', string_classes), language = (None, 'env', string_classes), - locale_dirs = (['locales'], 'env'), + locale_dirs = (['locales'], 'env', []), figure_language_filename = (u'{root}.{language}{ext}', 'env', [str]), - master_doc = ('contents', 'env'), - source_suffix = (['.rst'], 'env'), - source_encoding = ('utf-8-sig', 'env'), - source_parsers = ({}, 'env'), - exclude_patterns = ([], 'env'), + master_doc = ('contents', 'env', []), + source_suffix = ({'.rst': 'restructuredtext'}, 'env', Any), + source_encoding = ('utf-8-sig', 'env', []), + source_parsers = ({}, 'env', []), + exclude_patterns = ([], 'env', []), default_role = (None, 'env', string_classes), - add_function_parentheses = (True, 'env'), - add_module_names = (True, 'env'), - trim_footnote_reference_space = (False, 'env'), - show_authors = (False, 'env'), + add_function_parentheses = (True, 'env', []), + add_module_names = (True, 'env', []), + trim_footnote_reference_space = (False, 'env', []), + show_authors = (False, 'env', []), pygments_style = (None, 'html', string_classes), - highlight_language = ('default', 'env'), - highlight_options = ({}, 'env'), - templates_path = ([], 'html'), + highlight_language = ('default', 'env', []), + highlight_options = ({}, 'env', []), + templates_path = ([], 'html', []), template_bridge = (None, 'html', string_classes), - keep_warnings = (False, 'env'), - suppress_warnings = ([], 'env'), - modindex_common_prefix = ([], 'html'), + keep_warnings = (False, 'env', []), + suppress_warnings = ([], 'env', []), + modindex_common_prefix = ([], 'html', []), rst_epilog = (None, 'env', string_classes), rst_prolog = (None, 'env', string_classes), - trim_doctest_flags = (True, 'env'), + trim_doctest_flags = (True, 'env', []), primary_domain = ('py', 'env', [NoneType]), needs_sphinx = (None, None, string_classes), - needs_extensions = ({}, None), - manpages_url = (None, 'env'), - nitpicky = (False, None), - nitpick_ignore = ([], None), - numfig = (False, 'env'), - numfig_secnum_depth = (1, 'env'), - numfig_format = ({'section': l_('Section %s'), - 'figure': l_('Fig. %s'), - 'table': l_('Table %s'), - 'code-block': l_('Listing %s')}, - 'env'), - - tls_verify = (True, 'env'), - tls_cacerts = (None, 'env'), - smartquotes = (True, 'env'), - smartquotes_action = ('qDe', 'env'), + needs_extensions = ({}, None, []), + manpages_url = (None, 'env', []), + nitpicky = (False, None, []), + nitpick_ignore = ([], None, []), + numfig = (False, 'env', []), + numfig_secnum_depth = (1, 'env', []), + numfig_format = ({}, 'env', []), # will be initialized in init_numfig_format() + + tls_verify = (True, 'env', []), + tls_cacerts = (None, 'env', []), + smartquotes = (True, 'env', []), + smartquotes_action = ('qDe', 'env', []), smartquotes_excludes = ({'languages': ['ja'], 'builders': ['man', 'text']}, - 'env'), + 'env', []), ) # type: Dict[unicode, Tuple] - def __init__(self, dirname, filename, overrides, tags): - # type: (unicode, unicode, Dict, Tags) -> None + def __init__(self, *args): + # type: (Any) -> None + if len(args) == 4: + # old style arguments: (dirname, filename, overrides, tags) + warnings.warn('The argument of Config() class has been changed. ' + 'Use Config.read() to read configuration from conf.py.', + RemovedInSphinx30Warning) + dirname, filename, overrides, tags = args + if dirname is None: + config = {} # type: Dict[unicode, Any] + else: + config = eval_config_file(path.join(dirname, filename), tags) + else: + # new style arguments: (config={}, overrides={}) + if len(args) == 0: + config, overrides = {}, {} + elif len(args) == 1: + config, overrides = args[0], {} + else: + config, overrides = args[:2] + self.overrides = overrides self.values = Config.config_values.copy() - config = {} # type: Dict[unicode, Any] - if dirname is not None: - config_file = path.join(dirname, filename) - config['__file__'] = config_file - config['tags'] = tags - with cd(dirname): - # we promise to have the config dir as current dir while the - # config file is executed - try: - execfile_(filename, config) - except SyntaxError as err: - raise ConfigError(CONFIG_SYNTAX_ERROR % err) - except SystemExit: - raise ConfigError(CONFIG_EXIT_ERROR) - except Exception: - raise ConfigError(CONFIG_ERROR % traceback.format_exc()) - self._raw_config = config - # these two must be preinitialized because extensions can add their - # own config values self.setup = config.get('setup', None) # type: Callable if 'extensions' in overrides: @@ -178,66 +181,25 @@ class Config(object): config['extensions'] = overrides.pop('extensions') self.extensions = config.get('extensions', []) # type: List[unicode] - # correct values of copyright year that are not coherent with - # the SOURCE_DATE_EPOCH environment variable (if set) - # See https://reproducible-builds.org/specs/source-date-epoch/ - if getenv('SOURCE_DATE_EPOCH') is not None: - for k in ('copyright', 'epub_copyright'): - if k in config: - config[k] = copyright_year_re.sub(r'\g<1>%s' % format_date('%Y'), - config[k]) + @classmethod + def read(cls, confdir, overrides=None, tags=None): + # type: (unicode, Dict, Tags) -> Config + """Create a Config object from configuration file.""" + filename = path.join(confdir, CONFIG_FILENAME) + namespace = eval_config_file(filename, tags) + return cls(namespace, overrides or {}) def check_types(self): # type: () -> None - # check all values for deviation from the default value's type, since - # that can result in TypeErrors all over the place - # NB. since config values might use l_() we have to wait with calling - # this method until i18n is initialized - for name in self._raw_config: - if name not in self.values: - continue # we don't know a default value - settings = self.values[name] - default, dummy_rebuild = settings[:2] - permitted = settings[2] if len(settings) == 3 else () - - if hasattr(default, '__call__'): - default = default(self) # could invoke l_() - if default is None and not permitted: - continue # neither inferrable nor expliclitly permitted types - current = self[name] - if isinstance(permitted, ENUM): - if not permitted.match(current): - logger.warning(CONFIG_ENUM_WARNING.format( - name=name, current=current, candidates=permitted.candidates)) - else: - if type(current) is type(default): - continue - if type(current) in permitted: - continue - - common_bases = (set(type(current).__bases__ + (type(current),)) & - set(type(default).__bases__)) - common_bases.discard(object) - if common_bases: - continue # at least we share a non-trivial base class - - if permitted: - logger.warning(CONFIG_PERMITTED_TYPE_WARNING.format( - name=name, current=type(current), - permitted=str([cls.__name__ for cls in permitted]))) - else: - logger.warning(CONFIG_TYPE_WARNING.format( - name=name, current=type(current), default=type(default))) + warnings.warn('Config.check_types() is deprecated. Use check_confval_types() instead.', + RemovedInSphinx30Warning) + check_confval_types(None, self) def check_unicode(self): # type: () -> None - # check all string values for non-ASCII characters in bytestrings, - # since that can result in UnicodeErrors all over the place - for name, value in iteritems(self._raw_config): - if isinstance(value, binary_type) and nonascii_re.search(value): - logger.warning('the config value %r is set to a string with non-ASCII ' - 'characters; this can lead to Unicode errors occurring. ' - 'Please use Unicode strings, e.g. %r.', name, u'Content') + warnings.warn('Config.check_unicode() is deprecated. Use check_unicode() instead.', + RemovedInSphinx30Warning) + check_unicode(self) def convert_overrides(self, name, value): # type: (unicode, Any) -> Any @@ -245,7 +207,9 @@ class Config(object): return value else: defvalue = self.values[name][0] - if isinstance(defvalue, dict): + if self.values[name][2] == Any: + return value + elif isinstance(defvalue, dict): raise ValueError(__('cannot override dictionary config setting %r, ' 'ignoring (use %r to set individual elements)') % (name, name + '.key=value')) @@ -302,8 +266,6 @@ class Config(object): for name in config: if name in self.values: self.__dict__[name] = config[name] # type: ignore - if isinstance(self.source_suffix, string_types): # type: ignore - self.source_suffix = [self.source_suffix] # type: ignore def __getattr__(self, name): # type: (unicode) -> Any @@ -333,16 +295,204 @@ class Config(object): return name in self.values def __iter__(self): - # type: () -> Iterable[ConfigValue] + # type: () -> Generator[ConfigValue, None, None] for name, value in iteritems(self.values): yield ConfigValue(name, getattr(self, name), value[1]) # type: ignore def add(self, name, default, rebuild, types): # type: (unicode, Any, Union[bool, unicode], Any) -> None - self.values[name] = (default, rebuild, types) + if name in self.values: + raise ExtensionError(__('Config value %r already present') % name) + else: + self.values[name] = (default, rebuild, types) def filter(self, rebuild): # type: (Union[unicode, List[unicode]]) -> Iterator[ConfigValue] if isinstance(rebuild, string_types): rebuild = [rebuild] - return (value for value in self if value.rebuild in rebuild) # type: ignore + return (value for value in self if value.rebuild in rebuild) + + def __getstate__(self): + # type: () -> Dict + """Obtains serializable data for pickling.""" + # remove potentially pickling-problematic values from config + __dict__ = {} + for key, value in iteritems(self.__dict__): + if key.startswith('_') or isinstance(value, UNSERIALIZEABLE_TYPES): + pass + else: + __dict__[key] = value + + # create a picklable copy of values list + __dict__['values'] = {} + for key, value in iteritems(self.values): # type: ignore + real_value = getattr(self, key) + if isinstance(real_value, UNSERIALIZEABLE_TYPES): + # omit unserializable value + real_value = None + + # types column is also omitted + __dict__['values'][key] = (real_value, value[1], None) + + return __dict__ + + def __setstate__(self, state): + # type: (Dict) -> None + self.__dict__.update(state) + + +def eval_config_file(filename, tags): + # type: (unicode, Tags) -> Dict[unicode, Any] + """Evaluate a config file.""" + namespace = {} # type: Dict[unicode, Any] + namespace['__file__'] = filename + namespace['tags'] = tags + + with cd(path.dirname(filename)): + # during executing config file, current dir is changed to ``confdir``. + try: + execfile_(filename, namespace) + except SyntaxError as err: + msg = __("There is a syntax error in your configuration file: %s") + if PY3: + msg += __("\nDid you change the syntax from 2.x to 3.x?") + raise ConfigError(msg % err) + except SystemExit: + msg = __("The configuration file (or one of the modules it imports) " + "called sys.exit()") + raise ConfigError(msg) + except Exception: + msg = __("There is a programable error in your configuration file:\n\n%s") + raise ConfigError(msg % traceback.format_exc()) + + return namespace + + +def convert_source_suffix(app, config): + # type: (Sphinx, Config) -> None + """This converts old styled source_suffix to new styled one. + + * old style: str or list + * new style: a dict which maps from fileext to filetype + """ + source_suffix = config.source_suffix + if isinstance(source_suffix, string_types): + # if str, considers as default filetype (None) + # + # The default filetype is determined on later step. + # By default, it is considered as restructuredtext. + config.source_suffix = OrderedDict({source_suffix: None}) # type: ignore + elif isinstance(source_suffix, (list, tuple)): + # if list, considers as all of them are default filetype + config.source_suffix = OrderedDict([(s, None) for s in source_suffix]) # type: ignore # NOQA + elif isinstance(source_suffix, dict): + # if dict, convert it to OrderedDict + config.source_suffix = OrderedDict(config.source_suffix) # type: ignore + else: + logger.warning(__("The config value `source_suffix' expected to " + "a string, list of strings or dictionary. " + "But `%r' is given." % source_suffix)) + + +def init_numfig_format(app, config): + # type: (Sphinx, Config) -> None + """Initialize :confval:`numfig_format`.""" + numfig_format = {'section': _('Section %s'), + 'figure': _('Fig. %s'), + 'table': _('Table %s'), + 'code-block': _('Listing %s')} + + # override default labels by configuration + numfig_format.update(config.numfig_format) + config.numfig_format = numfig_format # type: ignore + + +def correct_copyright_year(app, config): + # type: (Sphinx, Config) -> None + """correct values of copyright year that are not coherent with + the SOURCE_DATE_EPOCH environment variable (if set) + + See https://reproducible-builds.org/specs/source-date-epoch/ + """ + if getenv('SOURCE_DATE_EPOCH') is not None: + for k in ('copyright', 'epub_copyright'): + if k in config: + replace = r'\g<1>%s' % format_date('%Y') + config[k] = copyright_year_re.sub(replace, config[k]) # type: ignore + + +def check_confval_types(app, config): + # type: (Sphinx, Config) -> None + """check all values for deviation from the default value's type, since + that can result in TypeErrors all over the place NB. + """ + for confval in config: + default, rebuild, annotations = config.values[confval.name] + + if hasattr(default, '__call__'): + default = default(config) # evaluate default value + if default is None and not annotations: + continue # neither inferrable nor expliclitly annotated types + + if annotations is Any: + # any type of value is accepted + pass + elif isinstance(annotations, ENUM): + if not annotations.match(confval.value): + msg = __("The config value `{name}` has to be a one of {candidates}, " + "but `{current}` is given.") + logger.warning(msg.format(name=confval.name, + current=confval.value, + candidates=annotations.candidates)) + else: + if type(confval.value) is type(default): + continue + if type(confval.value) in annotations: + continue + + common_bases = (set(type(confval.value).__bases__ + (type(confval.value),)) & + set(type(default).__bases__)) + common_bases.discard(object) + if common_bases: + continue # at least we share a non-trivial base class + + if annotations: + msg = __("The config value `{name}' has type `{current.__name__}', " + "expected to {permitted}.") + logger.warning(msg.format(name=confval.name, + current=type(confval.value), + permitted=str([c.__name__ for c in annotations]))) + else: + msg = __("The config value `{name}' has type `{current.__name__}', " + "defaults to `{default.__name__}'.") + logger.warning(msg.format(name=confval.name, + current=type(confval.value), + default=type(default))) + + +def check_unicode(config): + # type: (Config) -> None + """check all string values for non-ASCII characters in bytestrings, + since that can result in UnicodeErrors all over the place + """ + nonascii_re = re.compile(br'[\x80-\xff]') + + for name, value in iteritems(config._raw_config): + if isinstance(value, binary_type) and nonascii_re.search(value): + logger.warning(__('the config value %r is set to a string with non-ASCII ' + 'characters; this can lead to Unicode errors occurring. ' + 'Please use Unicode strings, e.g. %r.'), name, u'Content') + + +def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] + app.connect('config-inited', convert_source_suffix) + app.connect('config-inited', init_numfig_format) + app.connect('config-inited', correct_copyright_year) + app.connect('config-inited', check_confval_types) + + return { + 'version': 'builtin', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/deprecation.py b/sphinx/deprecation.py index e28e0f916..d4067055a 100644 --- a/sphinx/deprecation.py +++ b/sphinx/deprecation.py @@ -9,6 +9,13 @@ :license: BSD, see LICENSE for details. """ +import warnings + +if False: + # For type annotation + # note: Don't use typing.TYPE_CHECK here (for py27 and py34). + from typing import Any, Dict, Type # NOQA + class RemovedInSphinx18Warning(DeprecationWarning): pass @@ -22,4 +29,47 @@ class RemovedInSphinx20Warning(PendingDeprecationWarning): pass +class RemovedInSphinx30Warning(PendingDeprecationWarning): + pass + + +class RemovedInSphinx40Warning(PendingDeprecationWarning): + pass + + RemovedInNextVersionWarning = RemovedInSphinx18Warning + + +class DeprecatedDict(dict): + """A deprecated dict which warns on each access.""" + + def __init__(self, data, message, warning): + # type: (Dict, str, Type[Warning]) -> None + self.message = message + self.warning = warning + super(DeprecatedDict, self).__init__(data) + + def __setitem__(self, key, value): + # type: (unicode, Any) -> None + warnings.warn(self.message, self.warning) + super(DeprecatedDict, self).__setitem__(key, value) + + def setdefault(self, key, default=None): + # type: (unicode, Any) -> None + warnings.warn(self.message, self.warning) + return super(DeprecatedDict, self).setdefault(key, default) + + def __getitem__(self, key): + # type: (unicode) -> None + warnings.warn(self.message, self.warning) + return super(DeprecatedDict, self).__getitem__(key) + + def get(self, key, default=None): + # type: (unicode, Any) -> None + warnings.warn(self.message, self.warning) + return super(DeprecatedDict, self).get(key, default) + + def update(self, other=None): # type: ignore + # type: (Dict) -> None + warnings.warn(self.message, self.warning) + super(DeprecatedDict, self).update(other) diff --git a/sphinx/directives/__init__.py b/sphinx/directives/__init__.py index dc51810d3..1177a258e 100644 --- a/sphinx/directives/__init__.py +++ b/sphinx/directives/__init__.py @@ -12,10 +12,11 @@ import re from docutils import nodes -from docutils.parsers.rst import Directive, directives, roles +from docutils.parsers.rst import directives, roles from sphinx import addnodes from sphinx.util.docfields import DocFieldTransformer +from sphinx.util.docutils import SphinxDirective # import all directives sphinx provides from sphinx.directives.code import ( # noqa @@ -33,6 +34,7 @@ if False: # For type annotation from typing import Any, Dict, List # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA from sphinx.environment import BuildEnvironment # NOQA @@ -41,7 +43,7 @@ nl_escape_re = re.compile(r'\\\n') strip_backslash_re = re.compile(r'\\(.)') -class ObjectDescription(Directive): +class ObjectDescription(SphinxDirective): """ Directive to describe a class, function or similar object. Not used directly, but subclassed (in domain-specific directives) to add custom @@ -135,7 +137,6 @@ class ObjectDescription(Directive): self.domain, self.objtype = self.name.split(':', 1) else: self.domain, self.objtype = '', self.name - self.env = self.state.document.settings.env # type: BuildEnvironment self.indexnode = addnodes.index(entries=[]) node = addnodes.desc() @@ -187,7 +188,7 @@ class ObjectDescription(Directive): DescDirective = ObjectDescription -class DefaultRole(Directive): +class DefaultRole(SphinxDirective): """ Set the default interpreted text role. Overridden from docutils. """ @@ -212,11 +213,11 @@ class DefaultRole(Directive): line=self.lineno) return messages + [error] roles._roles[''] = role - self.state.document.settings.env.temp_data['default_role'] = role_name + self.env.temp_data['default_role'] = role_name return messages -class DefaultDomain(Directive): +class DefaultDomain(SphinxDirective): """ Directive to (re-)set the default domain for this source file. """ @@ -229,7 +230,6 @@ class DefaultDomain(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env domain_name = self.arguments[0].lower() # if domain_name not in env.domains: # # try searching by label @@ -237,7 +237,7 @@ class DefaultDomain(Directive): # if domain.label.lower() == domain_name: # domain_name = domain.name # break - env.temp_data['default_domain'] = env.domains.get(domain_name) + self.env.temp_data['default_domain'] = self.env.domains.get(domain_name) return [] diff --git a/sphinx/directives/code.py b/sphinx/directives/code.py index ae67cf4c6..713e5ad0c 100644 --- a/sphinx/directives/code.py +++ b/sphinx/directives/code.py @@ -9,16 +9,20 @@ import codecs import sys +import warnings from difflib import unified_diff from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from docutils.statemachine import ViewList +from six import text_type from sphinx import addnodes +from sphinx.deprecation import RemovedInSphinx40Warning from sphinx.locale import __ from sphinx.util import logging from sphinx.util import parselinenos +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import set_source_info if False: @@ -30,7 +34,7 @@ if False: logger = logging.getLogger(__name__) -class Highlight(Directive): +class Highlight(SphinxDirective): """ Directive to set the highlighting language for code blocks, as well as the threshold for line numbers. @@ -57,6 +61,17 @@ class Highlight(Directive): linenothreshold=linenothreshold)] +class HighlightLang(Highlight): + """highlightlang directive (deprecated)""" + + def run(self): + # type: () -> List[nodes.Node] + warnings.warn('highlightlang directive is deprecated. ' + 'Please use highlight directive instead.', + RemovedInSphinx40Warning) + return Highlight.run(self) + + def dedent_lines(lines, dedent, location=None): # type: (List[unicode], int, Any) -> List[unicode] if not dedent: @@ -76,7 +91,7 @@ def dedent_lines(lines, dedent, location=None): def container_wrapper(directive, literal_node, caption): - # type: (Directive, nodes.Node, unicode) -> nodes.container + # type: (SphinxDirective, nodes.Node, unicode) -> nodes.container container_node = nodes.container('', literal_block=True, classes=['literal-block-wrapper']) parsed = nodes.Element() @@ -94,7 +109,7 @@ def container_wrapper(directive, literal_node, caption): return container_node -class CodeBlock(Directive): +class CodeBlock(SphinxDirective): """ Directive for a code block with special highlighting or line numbering settings. @@ -126,7 +141,7 @@ class CodeBlock(Directive): nlines = len(self.content) hl_lines = parselinenos(linespec, nlines) if any(i >= nlines for i in hl_lines): - logger.warning('line number spec is out of range(1-%d): %r' % + logger.warning(__('line number spec is out of range(1-%d): %r') % (nlines, self.options['emphasize-lines']), location=location) @@ -159,7 +174,7 @@ class CodeBlock(Directive): try: literal = container_wrapper(self, literal, caption) except ValueError as exc: - return [document.reporter.warning(str(exc), line=self.lineno)] + return [document.reporter.warning(text_type(exc), line=self.lineno)] # literal will be note_implicit_target that is linked from caption and numref. # when options['name'] is provided, it should be primary ID. @@ -268,7 +283,7 @@ class LiteralIncludeReader(object): if linespec: linelist = parselinenos(linespec, len(lines)) if any(i >= len(lines) for i in linelist): - logger.warning('line number spec is out of range(1-%d): %r' % + logger.warning(__('line number spec is out of range(1-%d): %r') % (len(lines), linespec), location=location) if 'lineno-match' in self.options: @@ -364,13 +379,14 @@ class LiteralIncludeReader(object): return lines def dedent_filter(self, lines, location=None): + # type: (List[unicode], Any) -> List[unicode] if 'dedent' in self.options: return dedent_lines(lines, self.options.get('dedent'), location=location) else: return lines -class LiteralInclude(Directive): +class LiteralInclude(SphinxDirective): """ Like ``.. include:: :literal:``, but only warns if the include file is not found, and does not raise errors. Also has several options for @@ -410,19 +426,17 @@ class LiteralInclude(Directive): if not document.settings.file_insertion_enabled: return [document.reporter.warning('File insertion disabled', line=self.lineno)] - env = document.settings.env - # convert options['diff'] to absolute path if 'diff' in self.options: - _, path = env.relfn2path(self.options['diff']) + _, path = self.env.relfn2path(self.options['diff']) self.options['diff'] = path try: location = self.state_machine.get_source_and_line(self.lineno) - rel_filename, filename = env.relfn2path(self.arguments[0]) - env.note_dependency(rel_filename) + rel_filename, filename = self.env.relfn2path(self.arguments[0]) + self.env.note_dependency(rel_filename) - reader = LiteralIncludeReader(filename, self.options, env.config) + reader = LiteralIncludeReader(filename, self.options, self.config) text, lines = reader.read(location=location) retnode = nodes.literal_block(text, text, source=filename) @@ -439,7 +453,7 @@ class LiteralInclude(Directive): if 'emphasize-lines' in self.options: hl_lines = parselinenos(self.options['emphasize-lines'], lines) if any(i >= lines for i in hl_lines): - logger.warning('line number spec is out of range(1-%d): %r' % + logger.warning(__('line number spec is out of range(1-%d): %r') % (lines, self.options['emphasize-lines']), location=location) extra_args['hl_lines'] = [x + 1 for x in hl_lines if x < lines] @@ -455,13 +469,13 @@ class LiteralInclude(Directive): return [retnode] except Exception as exc: - return [document.reporter.warning(str(exc), line=self.lineno)] + return [document.reporter.warning(text_type(exc), line=self.lineno)] def setup(app): # type: (Sphinx) -> Dict[unicode, Any] directives.register_directive('highlight', Highlight) - directives.register_directive('highlightlang', Highlight) # old + directives.register_directive('highlightlang', HighlightLang) directives.register_directive('code-block', CodeBlock) directives.register_directive('sourcecode', CodeBlock) directives.register_directive('literalinclude', LiteralInclude) diff --git a/sphinx/directives/other.py b/sphinx/directives/other.py index e9a3864ab..41b21f917 100644 --- a/sphinx/directives/other.py +++ b/sphinx/directives/other.py @@ -8,17 +8,20 @@ """ import re +from contextlib import contextmanager from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from docutils.parsers.rst.directives.admonitions import BaseAdmonition from docutils.parsers.rst.directives.misc import Class from docutils.parsers.rst.directives.misc import Include as BaseInclude from six.moves import range -from sphinx import addnodes -from sphinx.locale import versionlabels, _ +from sphinx import addnodes, locale +from sphinx.deprecation import DeprecatedDict, RemovedInSphinx30Warning +from sphinx.locale import _ from sphinx.util import url_re, docname_join +from sphinx.util.docutils import SphinxDirective from sphinx.util.matching import patfilter from sphinx.util.nodes import explicit_title_re, set_source_info, \ process_index_entry @@ -29,6 +32,19 @@ if False: from sphinx.application import Sphinx # NOQA +versionlabels = { + 'versionadded': _('New in version %s'), + 'versionchanged': _('Changed in version %s'), + 'deprecated': _('Deprecated since version %s'), +} # type: Dict[unicode, unicode] + +locale.versionlabels = DeprecatedDict( + versionlabels, + 'sphinx.locale.versionlabels is deprecated. ' + 'Please use sphinx.directives.other.versionlabels instead.', + RemovedInSphinx30Warning +) + glob_re = re.compile('.*[*?\[].*') @@ -39,7 +55,7 @@ def int_or_nothing(argument): return int(argument) -class TocTree(Directive): +class TocTree(SphinxDirective): """ Directive to notify Sphinx about the hierarchical structure of the docs, and to include a table-of-contents like tree in the current document. @@ -63,7 +79,7 @@ class TocTree(Directive): def run(self): # type: () -> List[nodes.Node] subnode = addnodes.toctree() - subnode['parent'] = self.state.document.settings.env.docname + subnode['parent'] = self.env.docname # (title, ref) pairs, where ref may be a document, or an external link, # and title may be None if the document's title is to be used @@ -86,12 +102,11 @@ class TocTree(Directive): return ret def parse_content(self, toctree): - env = self.state.document.settings.env - suffixes = env.config.source_suffix + suffixes = self.config.source_suffix # glob target documents - all_docnames = env.found_docs.copy() - all_docnames.remove(env.docname) # remove current document + all_docnames = self.env.found_docs.copy() + all_docnames.remove(self.env.docname) # remove current document ret = [] for entry in self.content: @@ -101,7 +116,7 @@ class TocTree(Directive): explicit = explicit_title_re.match(entry) if (toctree['glob'] and glob_re.match(entry) and not explicit and not url_re.match(entry)): - patname = docname_join(env.docname, entry) + patname = docname_join(self.env.docname, entry) docnames = sorted(patfilter(all_docnames, patname)) for docname in docnames: all_docnames.remove(docname) # don't include it again @@ -125,14 +140,14 @@ class TocTree(Directive): docname = docname[:-len(suffix)] break # absolutize filenames - docname = docname_join(env.docname, docname) + docname = docname_join(self.env.docname, docname) if url_re.match(ref) or ref == 'self': toctree['entries'].append((title, ref)) - elif docname not in env.found_docs: + elif docname not in self.env.found_docs: ret.append(self.state.document.reporter.warning( 'toctree contains reference to nonexisting ' 'document %r' % docname, line=self.lineno)) - env.note_reread() + self.env.note_reread() else: all_docnames.discard(docname) toctree['entries'].append((title, docname)) @@ -145,7 +160,7 @@ class TocTree(Directive): return ret -class Author(Directive): +class Author(SphinxDirective): """ Directive to give the name of the author of the current document or section. Shown in the output only if the show_authors option is on. @@ -158,8 +173,7 @@ class Author(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env - if not env.config.show_authors: + if not self.config.show_authors: return [] para = nodes.paragraph(translatable=False) emph = nodes.emphasis() @@ -179,7 +193,7 @@ class Author(Directive): return [para] + messages -class Index(Directive): +class Index(SphinxDirective): """ Directive to add entries to the index. """ @@ -192,8 +206,7 @@ class Index(Directive): def run(self): # type: () -> List[nodes.Node] arguments = self.arguments[0].split('\n') - env = self.state.document.settings.env - targetid = 'index-%s' % env.new_serialno('index') + targetid = 'index-%s' % self.env.new_serialno('index') targetnode = nodes.target('', '', ids=[targetid]) self.state.document.note_explicit_target(targetnode) indexnode = addnodes.index() @@ -205,7 +218,7 @@ class Index(Directive): return [indexnode, targetnode] -class VersionChange(Directive): +class VersionChange(SphinxDirective): """ Directive to describe a change/addition/deprecation in a specific version. """ @@ -248,9 +261,8 @@ class VersionChange(Directive): classes=['versionmodified']), translatable=False) node.append(para) - env = self.state.document.settings.env # XXX should record node.source as well - env.note_versionchange(node['type'], node['version'], node, node.line) + self.env.note_versionchange(node['type'], node['version'], node, node.line) return [node] + messages @@ -261,7 +273,7 @@ class SeeAlso(BaseAdmonition): node_class = addnodes.seealso -class TabularColumns(Directive): +class TabularColumns(SphinxDirective): """ Directive to give an explicit tabulary column definition to LaTeX. """ @@ -279,7 +291,7 @@ class TabularColumns(Directive): return [node] -class Centered(Directive): +class Centered(SphinxDirective): """ Directive to create a centered line of bold text. """ @@ -300,7 +312,7 @@ class Centered(Directive): return [subnode] + messages -class Acks(Directive): +class Acks(SphinxDirective): """ Directive for a list of names. """ @@ -322,7 +334,7 @@ class Acks(Directive): return [node] -class HList(Directive): +class HList(SphinxDirective): """ Directive for a list that gets compacted horizontally. """ @@ -359,7 +371,7 @@ class HList(Directive): return [newnode] -class Only(Directive): +class Only(SphinxDirective): """ Directive to only include text if the given tag(s) are enabled. """ @@ -417,7 +429,7 @@ class Only(Directive): self.state.memo.section_level = surrounding_section_level -class Include(BaseInclude): +class Include(BaseInclude, SphinxDirective): """ Like the standard "Include" directive, but interprets absolute paths "correctly", i.e. relative to source directory. @@ -425,15 +437,35 @@ class Include(BaseInclude): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env + current_filename = self.env.doc2path(self.env.docname) if self.arguments[0].startswith('<') and \ self.arguments[0].endswith('>'): # docutils "standard" includes, do not do path processing return BaseInclude.run(self) - rel_filename, filename = env.relfn2path(self.arguments[0]) + rel_filename, filename = self.env.relfn2path(self.arguments[0]) self.arguments[0] = filename - env.note_included(filename) - return BaseInclude.run(self) + self.env.note_included(filename) + with patched_warnings(self, current_filename): + return BaseInclude.run(self) + + +@contextmanager +def patched_warnings(directive, parent_filename): + # type: (BaseInclude, unicode) -> Generator[None, None, None] + """Add includee filename to the warnings during inclusion.""" + try: + original = directive.state_machine.insert_input + + def insert_input(input_lines, source): + # type: (Any, unicode) -> None + source += ' <included from %s>' % parent_filename + original(input_lines, source) + + # patch insert_input() temporarily + directive.state_machine.insert_input = insert_input + yield + finally: + directive.state_machine.insert_input = original def setup(app): diff --git a/sphinx/directives/patches.py b/sphinx/directives/patches.py index c97340a81..00be5584d 100644 --- a/sphinx/directives/patches.py +++ b/sphinx/directives/patches.py @@ -12,11 +12,12 @@ from docutils.parsers.rst import directives from docutils.parsers.rst.directives import images, html, tables from sphinx import addnodes +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import set_source_info if False: # For type annotation - from typing import Dict, List # NOQA + from typing import Dict, List, Tuple # NOQA from sphinx.application import Sphinx # NOQA @@ -44,16 +45,15 @@ class Figure(images.Figure): return [figure_node] -class Meta(html.Meta): +class Meta(html.Meta, SphinxDirective): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env result = html.Meta.run(self) for node in result: if (isinstance(node, nodes.pending) and isinstance(node.details['nodes'][0], html.MetaBody.meta)): meta = node.details['nodes'][0] - meta.source = env.doc2path(env.docname) + meta.source = self.env.doc2path(self.env.docname) meta.line = self.lineno meta.rawcontent = meta['content'] @@ -69,6 +69,7 @@ class RSTTable(tables.RSTTable): Only for docutils-0.13 or older version.""" def make_title(self): + # type: () -> Tuple[nodes.Node, unicode] title, message = tables.RSTTable.make_title(self) if title: set_source_info(self, title) @@ -82,6 +83,7 @@ class CSVTable(tables.CSVTable): Only for docutils-0.13 or older version.""" def make_title(self): + # type: () -> Tuple[nodes.Node, unicode] title, message = tables.CSVTable.make_title(self) if title: set_source_info(self, title) @@ -95,6 +97,7 @@ class ListTable(tables.ListTable): Only for docutils-0.13 or older version.""" def make_title(self): + # type: () -> Tuple[nodes.Node, unicode] title, message = tables.ListTable.make_title(self) if title: set_source_info(self, title) diff --git a/sphinx/domains/__init__.py b/sphinx/domains/__init__.py index c68d37472..41db13cb6 100644 --- a/sphinx/domains/__init__.py +++ b/sphinx/domains/__init__.py @@ -150,6 +150,8 @@ class Domain(object): indices = [] # type: List[Type[Index]] #: role name -> a warning message if reference is missing dangling_warnings = {} # type: Dict[unicode, unicode] + #: node_class -> (enum_node_type, title_getter) + enumerable_nodes = {} # type: Dict[nodes.Node, Tuple[unicode, Callable]] #: data value for a fresh environment initial_data = {} # type: Dict @@ -281,11 +283,11 @@ class Domain(object): cross-reference. If no resolution can be found, None can be returned; the xref node will - then given to the 'missing-reference' event, and if that yields no + then given to the :event:`missing-reference` event, and if that yields no resolution, replaced by *contnode*. The method can also raise :exc:`sphinx.environment.NoUri` to suppress - the 'missing-reference' event being emitted. + the :event:`missing-reference` event being emitted. """ pass @@ -333,6 +335,12 @@ class Domain(object): return type.lname return _('%s %s') % (self.label, type.lname) + def get_enumerable_node_type(self, node): + # type: (nodes.Node) -> unicode + """Get type of enumerable nodes (experimental).""" + enum_node_type, _ = self.enumerable_nodes.get(node.__class__, (None, None)) + return enum_node_type + def get_full_qualified_name(self, node): # type: (nodes.Node) -> unicode """Return full qualified name for given node.""" diff --git a/sphinx/domains/c.py b/sphinx/domains/c.py index f0c81db7b..50e9b2e5e 100644 --- a/sphinx/domains/c.py +++ b/sphinx/domains/c.py @@ -17,7 +17,7 @@ from docutils import nodes from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import l_, _ +from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util.docfields import Field, TypedField from sphinx.util.nodes import make_refnode @@ -62,12 +62,12 @@ class CObject(ObjectDescription): """ doc_field_types = [ - TypedField('parameter', label=l_('Parameters'), + TypedField('parameter', label=_('Parameters'), names=('param', 'parameter', 'arg', 'argument'), typerolename='type', typenames=('type',)), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), - Field('returntype', label=l_('Return type'), has_arg=False, + Field('returntype', label=_('Return type'), has_arg=False, names=('rtype',)), ] @@ -254,11 +254,11 @@ class CDomain(Domain): name = 'c' label = 'C' object_types = { - 'function': ObjType(l_('function'), 'func'), - 'member': ObjType(l_('member'), 'member'), - 'macro': ObjType(l_('macro'), 'macro'), - 'type': ObjType(l_('type'), 'type'), - 'var': ObjType(l_('variable'), 'data'), + 'function': ObjType(_('function'), 'func'), + 'member': ObjType(_('member'), 'member'), + 'macro': ObjType(_('macro'), 'macro'), + 'type': ObjType(_('type'), 'type'), + 'var': ObjType(_('variable'), 'data'), } directives = { @@ -330,6 +330,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/cpp.py b/sphinx/domains/cpp.py index fe339b153..98eabb956 100644 --- a/sphinx/domains/cpp.py +++ b/sphinx/domains/cpp.py @@ -13,17 +13,18 @@ import re from copy import deepcopy from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from six import iteritems, text_type from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType from sphinx.environment import NoUri -from sphinx.locale import l_, _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import logging from sphinx.util.docfields import Field, GroupedField +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_refnode from sphinx.util.pycompat import UnicodeMixin @@ -48,7 +49,7 @@ logger = logging.getLogger(__name__) It is not the actual old code, but a replication of the behaviour. - v2: 1.3 <= version < now Standardised mangling scheme from - http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling + https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling though not completely implemented. All versions are generated and attached to elements. The newest is used for the index. All of the versions should work as permalinks. @@ -615,6 +616,7 @@ class ASTBase(UnicodeMixin): raise NotImplementedError(repr(self)) def __repr__(self): + # type: () -> str return '<%s %s>' % (self.__class__.__name__, self) @@ -3655,8 +3657,8 @@ class Symbol(object): ourChild._fill_empty(otherChild.declaration, otherChild.docname) elif ourChild.docname != otherChild.docname: name = text_type(ourChild.declaration) - msg = "Duplicate declaration, also defined in '%s'.\n" - msg += "Declaration is '%s'." + msg = __("Duplicate declaration, also defined in '%s'.\n" + "Declaration is '%s'.") msg = msg % (ourChild.docname, name) logger.warning(msg, location=otherChild.docname) else: @@ -5562,16 +5564,16 @@ class CPPObject(ObjectDescription): """Description of a C++ language object.""" doc_field_types = [ - GroupedField('parameter', label=l_('Parameters'), + GroupedField('parameter', label=_('Parameters'), names=('param', 'parameter', 'arg', 'argument'), can_collapse=True), - GroupedField('template parameter', label=l_('Template Parameters'), + GroupedField('template parameter', label=_('Template Parameters'), names=('tparam', 'template parameter'), can_collapse=True), - GroupedField('exceptions', label=l_('Throws'), rolename='cpp:class', + GroupedField('exceptions', label=_('Throws'), rolename='cpp:class', names=('throws', 'throw', 'exception'), can_collapse=True), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), ] @@ -5827,7 +5829,7 @@ class CPPEnumeratorObject(CPPObject): return parser.parse_declaration("enumerator") -class CPPNamespaceObject(Directive): +class CPPNamespaceObject(SphinxDirective): """ This directive is just to tell Sphinx that we're documenting stuff in namespace foo. @@ -5845,13 +5847,12 @@ class CPPNamespaceObject(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env - rootSymbol = env.domaindata['cpp']['root_symbol'] + rootSymbol = self.env.domaindata['cpp']['root_symbol'] if self.arguments[0].strip() in ('NULL', '0', 'nullptr'): symbol = rootSymbol stack = [] # type: List[Symbol] else: - parser = DefinitionParser(self.arguments[0], self, env.config) + parser = DefinitionParser(self.arguments[0], self, self.config) try: ast = parser.parse_namespace_object() parser.assert_end() @@ -5861,13 +5862,13 @@ class CPPNamespaceObject(Directive): ast = ASTNamespace(name, None) symbol = rootSymbol.add_name(ast.nestedName, ast.templatePrefix) stack = [symbol] - env.temp_data['cpp:parent_symbol'] = symbol - env.temp_data['cpp:namespace_stack'] = stack - env.ref_context['cpp:parent_key'] = symbol.get_lookup_key() + self.env.temp_data['cpp:parent_symbol'] = symbol + self.env.temp_data['cpp:namespace_stack'] = stack + self.env.ref_context['cpp:parent_key'] = symbol.get_lookup_key() return [] -class CPPNamespacePushObject(Directive): +class CPPNamespacePushObject(SphinxDirective): has_content = False required_arguments = 1 optional_arguments = 0 @@ -5880,10 +5881,9 @@ class CPPNamespacePushObject(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env if self.arguments[0].strip() in ('NULL', '0', 'nullptr'): return [] - parser = DefinitionParser(self.arguments[0], self, env.config) + parser = DefinitionParser(self.arguments[0], self, self.config) try: ast = parser.parse_namespace_object() parser.assert_end() @@ -5891,19 +5891,19 @@ class CPPNamespacePushObject(Directive): self.warn(e.description) name = _make_phony_error_name() ast = ASTNamespace(name, None) - oldParent = env.temp_data.get('cpp:parent_symbol', None) + oldParent = self.env.temp_data.get('cpp:parent_symbol', None) if not oldParent: - oldParent = env.domaindata['cpp']['root_symbol'] + oldParent = self.env.domaindata['cpp']['root_symbol'] symbol = oldParent.add_name(ast.nestedName, ast.templatePrefix) - stack = env.temp_data.get('cpp:namespace_stack', []) + stack = self.env.temp_data.get('cpp:namespace_stack', []) stack.append(symbol) - env.temp_data['cpp:parent_symbol'] = symbol - env.temp_data['cpp:namespace_stack'] = stack - env.ref_context['cpp:parent_key'] = symbol.get_lookup_key() + self.env.temp_data['cpp:parent_symbol'] = symbol + self.env.temp_data['cpp:namespace_stack'] = stack + self.env.ref_context['cpp:parent_key'] = symbol.get_lookup_key() return [] -class CPPNamespacePopObject(Directive): +class CPPNamespacePopObject(SphinxDirective): has_content = False required_arguments = 0 optional_arguments = 0 @@ -5916,8 +5916,7 @@ class CPPNamespacePopObject(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env - stack = env.temp_data.get('cpp:namespace_stack', None) + stack = self.env.temp_data.get('cpp:namespace_stack', None) if not stack or len(stack) == 0: self.warn("C++ namespace pop on empty stack. Defaulting to gobal scope.") stack = [] @@ -5926,10 +5925,10 @@ class CPPNamespacePopObject(Directive): if len(stack) > 0: symbol = stack[-1] else: - symbol = env.domaindata['cpp']['root_symbol'] - env.temp_data['cpp:parent_symbol'] = symbol - env.temp_data['cpp:namespace_stack'] = stack - env.ref_context['cpp:parent_key'] = symbol.get_lookup_key() + symbol = self.env.domaindata['cpp']['root_symbol'] + self.env.temp_data['cpp:parent_symbol'] = symbol + self.env.temp_data['cpp:namespace_stack'] = stack + self.env.ref_context['cpp:parent_key'] = symbol.get_lookup_key() return [] @@ -5984,13 +5983,13 @@ class CPPDomain(Domain): name = 'cpp' label = 'C++' object_types = { - 'class': ObjType(l_('class'), 'class', 'type', 'identifier'), - 'function': ObjType(l_('function'), 'function', 'func', 'type', 'identifier'), - 'member': ObjType(l_('member'), 'member', 'var'), - 'type': ObjType(l_('type'), 'type', 'identifier'), - 'concept': ObjType(l_('concept'), 'concept', 'identifier'), - 'enum': ObjType(l_('enum'), 'enum', 'type', 'identifier'), - 'enumerator': ObjType(l_('enumerator'), 'enumerator') + 'class': ObjType(_('class'), 'class', 'type', 'identifier'), + 'function': ObjType(_('function'), 'function', 'func', 'type', 'identifier'), + 'member': ObjType(_('member'), 'member', 'var'), + 'type': ObjType(_('type'), 'type', 'identifier'), + 'concept': ObjType(_('concept'), 'concept', 'identifier'), + 'enum': ObjType(_('enum'), 'enum', 'type', 'identifier'), + 'enumerator': ObjType(_('enumerator'), 'enumerator') } directives = { @@ -6052,8 +6051,8 @@ class CPPDomain(Domain): for name, docname in otherdata['names'].items(): if docname in docnames: if name in ourNames: - msg = "Duplicate declaration, also defined in '%s'.\n" - msg += "Name of declaration is '%s'." + msg = __("Duplicate declaration, also defined in '%s'.\n" + "Name of declaration is '%s'.") msg = msg % (ourNames[name], name) logger.warning(msg, location=docname) else: @@ -6223,6 +6222,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/javascript.py b/sphinx/domains/javascript.py index 734969fc1..511a058c8 100644 --- a/sphinx/domains/javascript.py +++ b/sphinx/domains/javascript.py @@ -10,15 +10,16 @@ """ from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType from sphinx.domains.python import _pseudo_parse_arglist -from sphinx.locale import l_, _ +from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util.docfields import Field, GroupedField, TypedField +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_refnode if False: @@ -201,15 +202,15 @@ class JSCallable(JSObject): has_arguments = True doc_field_types = [ - TypedField('arguments', label=l_('Arguments'), + TypedField('arguments', label=_('Arguments'), names=('argument', 'arg', 'parameter', 'param'), typerolename='func', typenames=('paramtype', 'type')), - GroupedField('errors', label=l_('Throws'), rolename='err', + GroupedField('errors', label=_('Throws'), rolename='err', names=('throws', ), can_collapse=True), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), - Field('returntype', label=l_('Return type'), has_arg=False, + Field('returntype', label=_('Return type'), has_arg=False, names=('rtype',)), ] @@ -220,7 +221,7 @@ class JSConstructor(JSCallable): allow_nesting = True -class JSModule(Directive): +class JSModule(SphinxDirective): """ Directive to mark description of a new JavaScript module. @@ -249,16 +250,15 @@ class JSModule(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env mod_name = self.arguments[0].strip() - env.ref_context['js:module'] = mod_name + self.env.ref_context['js:module'] = mod_name noindex = 'noindex' in self.options ret = [] if not noindex: - env.domaindata['js']['modules'][mod_name] = env.docname + self.env.domaindata['js']['modules'][mod_name] = self.env.docname # Make a duplicate entry in 'objects' to facilitate searching for # the module in JavaScriptDomain.find_obj() - env.domaindata['js']['objects'][mod_name] = (env.docname, 'module') + self.env.domaindata['js']['objects'][mod_name] = (self.env.docname, 'module') targetnode = nodes.target('', '', ids=['module-' + mod_name], ismod=True) self.state.document.note_explicit_target(targetnode) @@ -296,12 +296,12 @@ class JavaScriptDomain(Domain): label = 'JavaScript' # if you add a new object type make sure to edit JSObject.get_index_string object_types = { - 'function': ObjType(l_('function'), 'func'), - 'method': ObjType(l_('method'), 'meth'), - 'class': ObjType(l_('class'), 'class'), - 'data': ObjType(l_('data'), 'data'), - 'attribute': ObjType(l_('attribute'), 'attr'), - 'module': ObjType(l_('module'), 'mod'), + 'function': ObjType(_('function'), 'func'), + 'method': ObjType(_('method'), 'meth'), + 'class': ObjType(_('class'), 'class'), + 'data': ObjType(_('data'), 'data'), + 'attribute': ObjType(_('attribute'), 'attr'), + 'module': ObjType(_('module'), 'mod'), } directives = { 'function': JSCallable, @@ -415,6 +415,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/python.py b/sphinx/domains/python.py index 2b46596af..0fe4b5eb4 100644 --- a/sphinx/domains/python.py +++ b/sphinx/domains/python.py @@ -12,16 +12,18 @@ import re from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from six import iteritems -from sphinx import addnodes +from sphinx import addnodes, locale +from sphinx.deprecation import DeprecatedDict, RemovedInSphinx30Warning from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType, Index -from sphinx.locale import l_, _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import logging from sphinx.util.docfields import Field, GroupedField, TypedField +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_refnode if False: @@ -44,6 +46,24 @@ py_sig_re = re.compile( ''', re.VERBOSE) +pairindextypes = { + 'module': _('module'), + 'keyword': _('keyword'), + 'operator': _('operator'), + 'object': _('object'), + 'exception': _('exception'), + 'statement': _('statement'), + 'builtin': _('built-in function'), +} # Dict[unicode, unicode] + +locale.pairindextypes = DeprecatedDict( + pairindextypes, + 'sphinx.locale.pairindextypes is deprecated. ' + 'Please use sphinx.domains.python.pairindextypes instead.', + RemovedInSphinx30Warning +) + + def _pseudo_parse_arglist(signode, arglist): # type: (addnodes.desc_signature, unicode) -> None """"Parse" a list of arguments separated by commas. @@ -173,21 +193,21 @@ class PyObject(ObjectDescription): } doc_field_types = [ - PyTypedField('parameter', label=l_('Parameters'), + PyTypedField('parameter', label=_('Parameters'), names=('param', 'parameter', 'arg', 'argument', 'keyword', 'kwarg', 'kwparam'), typerolename='class', typenames=('paramtype', 'type'), can_collapse=True), - PyTypedField('variable', label=l_('Variables'), rolename='obj', + PyTypedField('variable', label=_('Variables'), rolename='obj', names=('var', 'ivar', 'cvar'), typerolename='class', typenames=('vartype',), can_collapse=True), - PyGroupedField('exceptions', label=l_('Raises'), rolename='exc', + PyGroupedField('exceptions', label=_('Raises'), rolename='exc', names=('raises', 'raise', 'exception', 'except'), can_collapse=True), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), - PyField('returntype', label=l_('Return type'), has_arg=False, + PyField('returntype', label=_('Return type'), has_arg=False, names=('rtype',), bodyrolename='class'), ] @@ -536,7 +556,7 @@ class PyDecoratorMethod(PyDecoratorMixin, PyClassmember): return PyClassmember.run(self) -class PyModule(Directive): +class PyModule(SphinxDirective): """ Directive to mark description of a new module. """ @@ -554,19 +574,18 @@ class PyModule(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env modname = self.arguments[0].strip() noindex = 'noindex' in self.options - env.ref_context['py:module'] = modname + self.env.ref_context['py:module'] = modname ret = [] if not noindex: - env.domaindata['py']['modules'][modname] = (env.docname, - self.options.get('synopsis', ''), - self.options.get('platform', ''), - 'deprecated' in self.options) + self.env.domaindata['py']['modules'][modname] = (self.env.docname, + self.options.get('synopsis', ''), + self.options.get('platform', ''), + 'deprecated' in self.options) # make a duplicate entry in 'objects' to facilitate searching for # the module in PythonDomain.find_obj() - env.domaindata['py']['objects'][modname] = (env.docname, 'module') + self.env.domaindata['py']['objects'][modname] = (self.env.docname, 'module') targetnode = nodes.target('', '', ids=['module-' + modname], ismod=True) self.state.document.note_explicit_target(targetnode) @@ -580,7 +599,7 @@ class PyModule(Directive): return ret -class PyCurrentModule(Directive): +class PyCurrentModule(SphinxDirective): """ This directive is just to tell Sphinx that we're documenting stuff in module foo, but links to module foo won't lead here. @@ -594,12 +613,11 @@ class PyCurrentModule(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env modname = self.arguments[0].strip() if modname == 'None': - env.ref_context.pop('py:module', None) + self.env.ref_context.pop('py:module', None) else: - env.ref_context['py:module'] = modname + self.env.ref_context['py:module'] = modname return [] @@ -632,8 +650,8 @@ class PythonModuleIndex(Index): """ name = 'modindex' - localname = l_('Python Module Index') - shortname = l_('modules') + localname = _('Python Module Index') + shortname = _('modules') def generate(self, docnames=None): # type: (Iterable[unicode]) -> Tuple[List[Tuple[unicode, List[List[Union[unicode, int]]]]], bool] # NOQA @@ -703,15 +721,15 @@ class PythonDomain(Domain): name = 'py' label = 'Python' object_types = { - 'function': ObjType(l_('function'), 'func', 'obj'), - 'data': ObjType(l_('data'), 'data', 'obj'), - 'class': ObjType(l_('class'), 'class', 'exc', 'obj'), - 'exception': ObjType(l_('exception'), 'exc', 'class', 'obj'), - 'method': ObjType(l_('method'), 'meth', 'obj'), - 'classmethod': ObjType(l_('class method'), 'meth', 'obj'), - 'staticmethod': ObjType(l_('static method'), 'meth', 'obj'), - 'attribute': ObjType(l_('attribute'), 'attr', 'obj'), - 'module': ObjType(l_('module'), 'mod', 'obj'), + 'function': ObjType(_('function'), 'func', 'obj'), + 'data': ObjType(_('data'), 'data', 'obj'), + 'class': ObjType(_('class'), 'class', 'exc', 'obj'), + 'exception': ObjType(_('exception'), 'exc', 'class', 'obj'), + 'method': ObjType(_('method'), 'meth', 'obj'), + 'classmethod': ObjType(_('class method'), 'meth', 'obj'), + 'staticmethod': ObjType(_('static method'), 'meth', 'obj'), + 'attribute': ObjType(_('attribute'), 'attr', 'obj'), + 'module': ObjType(_('module'), 'mod', 'obj'), } # type: Dict[unicode, ObjType] directives = { @@ -841,7 +859,7 @@ class PythonDomain(Domain): if not matches: return None elif len(matches) > 1: - logger.warning('more than one target found for cross-reference %r: %s', + logger.warning(__('more than one target found for cross-reference %r: %s'), target, ', '.join(match[0] for match in matches), type='ref', subtype='python', location=node) name, obj = matches[0] @@ -912,6 +930,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/rst.py b/sphinx/domains/rst.py index c14eb180f..fcb461484 100644 --- a/sphinx/domains/rst.py +++ b/sphinx/domains/rst.py @@ -16,7 +16,7 @@ from six import iteritems from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import l_, _ +from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util.nodes import make_refnode @@ -116,8 +116,8 @@ class ReSTDomain(Domain): label = 'reStructuredText' object_types = { - 'directive': ObjType(l_('directive'), 'dir'), - 'role': ObjType(l_('role'), 'role'), + 'directive': ObjType(_('directive'), 'dir'), + 'role': ObjType(_('role'), 'role'), } directives = { 'directive': ReSTDirective, @@ -182,6 +182,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/std.py b/sphinx/domains/std.py index 937760c45..bc6b4de6c 100644 --- a/sphinx/domains/std.py +++ b/sphinx/domains/std.py @@ -11,23 +11,28 @@ import re import unicodedata +import warnings +from copy import copy from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from docutils.statemachine import ViewList from six import iteritems from sphinx import addnodes +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import l_, _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import ws_re, logging, docname_join +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import clean_astext, make_refnode if False: # For type annotation from typing import Any, Callable, Dict, Iterator, List, Tuple, Type, Union # NOQA + from docutils.parsers.rst import Directive # NOQA from sphinx.application import Sphinx # NOQA from sphinx.builders import Builder # NOQA from sphinx.environment import BuildEnvironment # NOQA @@ -80,7 +85,7 @@ class GenericObject(ObjectDescription): class EnvVar(GenericObject): - indextemplate = l_('environment variable; %s') + indextemplate = _('environment variable; %s') class EnvVarXRefRole(XRefRole): @@ -104,7 +109,7 @@ class EnvVarXRefRole(XRefRole): return [indexnode, targetnode, node], [] -class Target(Directive): +class Target(SphinxDirective): """ Generic target for user-defined cross-reference types. """ @@ -118,7 +123,6 @@ class Target(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env # normalize whitespace in fullname like XRefRole does fullname = ws_re.sub(' ', self.arguments[0].strip()) targetname = '%s-%s' % (self.name, fullname) @@ -138,8 +142,8 @@ class Target(Directive): name = self.name if ':' in self.name: _, name = self.name.split(':', 1) - env.domaindata['std']['objects'][name, fullname] = \ - env.docname, targetname + self.env.domaindata['std']['objects'][name, fullname] = \ + self.env.docname, targetname return ret @@ -157,9 +161,9 @@ class Cmdoption(ObjectDescription): potential_option = potential_option.strip() m = option_desc_re.match(potential_option) # type: ignore if not m: - logger.warning('Malformed option description %r, should ' - 'look like "opt", "-opt args", "--opt args", ' - '"/opt args" or "+opt args"', potential_option, + logger.warning(__('Malformed option description %r, should ' + 'look like "opt", "-opt args", "--opt args", ' + '"/opt args" or "+opt args"'), potential_option, location=(self.env.docname, self.lineno)) continue optname, args = m.groups() @@ -201,7 +205,7 @@ class Cmdoption(ObjectDescription): signode['ids'][0], '', None)) -class Program(Directive): +class Program(SphinxDirective): """ Directive to name the program for which options are documented. """ @@ -214,12 +218,11 @@ class Program(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env program = ws_re.sub('-', self.arguments[0].strip()) if program == 'None': - env.ref_context.pop('std:program', None) + self.env.ref_context.pop('std:program', None) else: - env.ref_context['std:program'] = program + self.env.ref_context['std:program'] = program return [] @@ -267,7 +270,7 @@ def make_glossary_term(env, textnodes, index_key, source, lineno, new_id=None): return term -class Glossary(Directive): +class Glossary(SphinxDirective): """ Directive to create a glossary with cross-reference targets for :term: roles. @@ -283,7 +286,6 @@ class Glossary(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env node = addnodes.glossary() node.document = self.state.document @@ -355,7 +357,7 @@ class Glossary(Directive): textnodes, sysmsg = self.state.inline_text(parts[0], lineno) # use first classifier as a index key - term = make_glossary_term(env, textnodes, parts[1], source, lineno) + term = make_glossary_term(self.env, textnodes, parts[1], source, lineno) term.rawsource = line system_messages.extend(sysmsg) termtexts.append(term.astext()) @@ -400,7 +402,7 @@ def token_xrefs(text): return retnodes -class ProductionList(Directive): +class ProductionList(SphinxDirective): """ Directive to list grammar productions. """ @@ -413,8 +415,7 @@ class ProductionList(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env - objects = env.domaindata['std']['objects'] + objects = self.env.domaindata['std']['objects'] node = addnodes.productionlist() messages = [] # type: List[nodes.Node] i = 0 @@ -435,7 +436,7 @@ class ProductionList(Directive): if idname not in self.state.document.ids: subnode['ids'].append(idname) self.state.document.note_implicit_target(subnode, subnode) - objects['token', subnode['tokenname']] = env.docname, idname + objects['token', subnode['tokenname']] = self.env.docname, idname subnode.extend(token_xrefs(tokens)) node.append(subnode) return [node] + messages @@ -451,13 +452,13 @@ class StandardDomain(Domain): label = 'Default' object_types = { - 'term': ObjType(l_('glossary term'), 'term', searchprio=-1), - 'token': ObjType(l_('grammar token'), 'token', searchprio=-1), - 'label': ObjType(l_('reference label'), 'ref', 'keyword', + 'term': ObjType(_('glossary term'), 'term', searchprio=-1), + 'token': ObjType(_('grammar token'), 'token', searchprio=-1), + 'label': ObjType(_('reference label'), 'ref', 'keyword', searchprio=-1), - 'envvar': ObjType(l_('environment variable'), 'envvar'), - 'cmdoption': ObjType(l_('program option'), 'option'), - 'doc': ObjType(l_('document'), 'doc', searchprio=-1) + 'envvar': ObjType(_('environment variable'), 'envvar'), + 'cmdoption': ObjType(_('program option'), 'option'), + 'doc': ObjType(_('document'), 'doc', searchprio=-1) } # type: Dict[unicode, ObjType] directives = { @@ -494,9 +495,9 @@ class StandardDomain(Domain): 'citations': {}, # citation_name -> docname, labelid, lineno 'citation_refs': {}, # citation_name -> list of docnames 'labels': { # labelname -> docname, labelid, sectionname - 'genindex': ('genindex', '', l_('Index')), - 'modindex': ('py-modindex', '', l_('Module Index')), - 'search': ('search', '', l_('Search Page')), + 'genindex': ('genindex', '', _('Index')), + 'modindex': ('py-modindex', '', _('Module Index')), + 'search': ('search', '', _('Search Page')), }, 'anonlabels': { # labelname -> docname, labelid 'genindex': ('genindex', ''), @@ -522,6 +523,15 @@ class StandardDomain(Domain): nodes.container: ('code-block', None), } # type: Dict[nodes.Node, Tuple[unicode, Callable]] + def __init__(self, env): + # type: (BuildEnvironment) -> None + super(StandardDomain, self).__init__(env) + + # set up enumerable nodes + self.enumerable_nodes = copy(self.enumerable_nodes) # create a copy for this instance + for node, settings in iteritems(env.app.registry.enumerable_nodes): + self.enumerable_nodes[node] = settings + def clear_doc(self, docname): # type: (unicode) -> None for key, (fn, _l) in list(self.data['progoptions'].items()): @@ -578,10 +588,11 @@ class StandardDomain(Domain): def note_citations(self, env, docname, document): # type: (BuildEnvironment, unicode, nodes.Node) -> None for node in document.traverse(nodes.citation): + node['docname'] = docname label = node[0].astext() if label in self.data['citations']: path = env.doc2path(self.data['citations'][label][0]) - logger.warning('duplicate citation %s, other instance in %s', label, path, + logger.warning(__('duplicate citation %s, other instance in %s'), label, path, location=node, type='ref', subtype='citation') self.data['citations'][label] = (docname, node['ids'][0], node.line) @@ -613,8 +624,8 @@ class StandardDomain(Domain): # link and object descriptions continue if name in labels: - logger.warning('duplicate label %s, ' % name + 'other instance ' - 'in ' + env.doc2path(labels[name][0]), + logger.warning(__('duplicate label %s, other instance in %s'), + name, env.doc2path(labels[name][0]), location=node) anonlabels[name] = docname, labelid if node.tagname in ('section', 'rubric'): @@ -638,7 +649,7 @@ class StandardDomain(Domain): # type: () -> None for name, (docname, labelid, lineno) in iteritems(self.data['citations']): if name not in self.data['citation_refs']: - logger.warning('Citation [%s] is not referenced.', name, + logger.warning(__('Citation [%s] is not referenced.'), name, type='ref', subtype='citation', location=(docname, lineno)) @@ -716,12 +727,12 @@ class StandardDomain(Domain): return None target_node = env.get_doctree(docname).ids.get(labelid) - figtype = self.get_figtype(target_node) + figtype = self.get_enumerable_node_type(target_node) if figtype is None: return None if figtype != 'section' and env.config.numfig is False: - logger.warning('numfig is disabled. :numref: is ignored.', location=node) + logger.warning(__('numfig is disabled. :numref: is ignored.'), location=node) return contnode try: @@ -729,7 +740,7 @@ class StandardDomain(Domain): if fignumber is None: return contnode except ValueError: - logger.warning("no number is assigned for %s: %s", figtype, labelid, + logger.warning(__("no number is assigned for %s: %s"), figtype, labelid, location=node) return contnode @@ -740,7 +751,7 @@ class StandardDomain(Domain): title = env.config.numfig_format.get(figtype, '') if figname is None and '{name}' in title: - logger.warning('the link has no caption: %s', title, location=node) + logger.warning(__('the link has no caption: %s'), title, location=node) return contnode else: fignum = '.'.join(map(str, fignumber)) @@ -754,10 +765,10 @@ class StandardDomain(Domain): # old style format (cf. "Fig.%s") newtitle = title % fignum except KeyError as exc: - logger.warning('invalid numfig_format: %s (%r)', title, exc, location=node) + logger.warning(__('invalid numfig_format: %s (%r)'), title, exc, location=node) return contnode except TypeError: - logger.warning('invalid numfig_format: %s', title, location=node) + logger.warning(__('invalid numfig_format: %s'), title, location=node) return contnode return self.build_reference_node(fromdocname, builder, @@ -916,9 +927,9 @@ class StandardDomain(Domain): return None - def get_figtype(self, node): + def get_enumerable_node_type(self, node): # type: (nodes.Node) -> unicode - """Get figure type of nodes.""" + """Get type of enumerable nodes.""" def has_child(node, cls): # type: (nodes.Node, Type) -> bool return any(isinstance(child, cls) for child in node) @@ -934,6 +945,17 @@ class StandardDomain(Domain): figtype, _ = self.enumerable_nodes.get(node.__class__, (None, None)) return figtype + def get_figtype(self, node): + # type: (nodes.Node) -> unicode + """Get figure type of nodes. + + .. deprecated:: 1.8 + """ + warnings.warn('StandardDomain.get_figtype() is deprecated. ' + 'Please use get_enumerable_node_type() instead.', + RemovedInSphinx30Warning) + return self.get_enumerable_node_type(node) + def get_fignumber(self, env, builder, figtype, docname, target_node): # type: (BuildEnvironment, Builder, unicode, unicode, nodes.Node) -> Tuple[int, ...] if figtype == 'section': @@ -979,6 +1001,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/environment/__init__.py b/sphinx/environment/__init__.py index 86d69cc3a..cafe5306e 100644 --- a/sphinx/environment/__init__.py +++ b/sphinx/environment/__init__.py @@ -12,34 +12,29 @@ import os import re import sys -import time -import types import warnings from collections import defaultdict from copy import copy from os import path -from docutils.frontend import OptionParser -from docutils.utils import Reporter, get_source_line -from six import BytesIO, itervalues, class_types, next +from docutils.utils import get_source_line +from six import BytesIO, next from six.moves import cPickle as pickle -from sphinx import addnodes, versioning -from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx import addnodes +from sphinx.deprecation import RemovedInSphinx20Warning, RemovedInSphinx30Warning from sphinx.environment.adapters.indexentries import IndexEntries from sphinx.environment.adapters.toctree import TocTree from sphinx.errors import SphinxError, ExtensionError -from sphinx.io import read_doc +from sphinx.locale import __ from sphinx.transforms import SphinxTransformer -from sphinx.util import get_matching_docs, FilenameUniqDict, status_iterator -from sphinx.util import logging, rst -from sphinx.util.console import bold # type: ignore -from sphinx.util.docutils import sphinx_domains, WarningStream +from sphinx.util import get_matching_docs, FilenameUniqDict +from sphinx.util import logging +from sphinx.util.docutils import LoggingReporter from sphinx.util.i18n import find_catalog_files from sphinx.util.matching import compile_matchers from sphinx.util.nodes import is_translatable -from sphinx.util.osutil import SEP, ensuredir, relpath -from sphinx.util.parallel import ParallelTasks, parallel_available, make_chunks +from sphinx.util.osutil import SEP, relpath from sphinx.util.websupport import is_commentable if False: @@ -72,7 +67,7 @@ default_settings = { # or changed to properly invalidate pickle files. # # NOTE: increase base version by 2 to have distinct numbers for Py2 and 3 -ENV_VERSION = 52 + (sys.version_info[0] - 2) +ENV_VERSION = 53 + (sys.version_info[0] - 2) versioning_conditions = { @@ -96,73 +91,6 @@ class BuildEnvironment(object): domains = None # type: Dict[unicode, Domain] - # --------- ENVIRONMENT PERSISTENCE ---------------------------------------- - - @staticmethod - def load(f, app=None): - # type: (IO, Sphinx) -> BuildEnvironment - try: - env = pickle.load(f) - except Exception as exc: - # This can happen for example when the pickle is from a - # different version of Sphinx. - raise IOError(exc) - if env.version != ENV_VERSION: - raise IOError('build environment version not current') - if app: - env.app = app - env.config.values = app.config.values - if env.srcdir != app.srcdir: - raise IOError('source directory has changed') - return env - - @classmethod - def loads(cls, string, app=None): - # type: (unicode, Sphinx) -> BuildEnvironment - io = BytesIO(string) - return cls.load(io, app) - - @classmethod - def frompickle(cls, filename, app): - # type: (unicode, Sphinx) -> BuildEnvironment - with open(filename, 'rb') as f: - return cls.load(f, app) - - @staticmethod - def dump(env, f): - # type: (BuildEnvironment, IO) -> None - # remove unpicklable attributes - app = env.app - del env.app - values = env.config.values - del env.config.values - domains = env.domains - del env.domains - # remove potentially pickling-problematic values from config - for key, val in list(vars(env.config).items()): - if key.startswith('_') or \ - isinstance(val, types.ModuleType) or \ - isinstance(val, types.FunctionType) or \ - isinstance(val, class_types): - del env.config[key] - pickle.dump(env, f, pickle.HIGHEST_PROTOCOL) - # reset attributes - env.domains = domains - env.config.values = values - env.app = app - - @classmethod - def dumps(cls, env): - # type: (BuildEnvironment) -> unicode - io = BytesIO() - cls.dump(env, io) - return io.getvalue() - - def topickle(self, filename): - # type: (unicode) -> None - with open(filename, 'wb') as f: - self.dump(self, f) - # --------- ENVIRONMENT INITIALIZATION ------------------------------------- def __init__(self, app): @@ -187,7 +115,7 @@ class BuildEnvironment(object): self._warnfunc = None # type: Callable # this is to invalidate old pickles - self.version = ENV_VERSION # type: int + self.version = app.registry.get_envversion(app) # type: Dict[unicode, unicode] # All "docnames" here are /-separated and relative and exclude # the source suffix. @@ -265,6 +193,17 @@ class BuildEnvironment(object): # attributes of "any" cross references self.ref_context = {} # type: Dict[unicode, Any] + def __getstate__(self): + # type: () -> Dict + """Obtains serializable data for pickling.""" + __dict__ = self.__dict__.copy() + __dict__.update(app=None, domains={}) # clear unpickable attributes + return __dict__ + + def __setstate__(self, state): + # type: (Dict) -> None + self.__dict__.update(state) + def set_warnfunc(self, func): # type: (Callable) -> None warnings.warn('env.set_warnfunc() is now deprecated. Use sphinx.util.logging instead.', @@ -283,9 +222,9 @@ class BuildEnvironment(object): raise ValueError('invalid versioning method: %r' % method) condition = versioning_conditions[method] if self.versioning_condition not in (None, condition): - raise SphinxError('This environment is incompatible with the ' - 'selected builder, please choose another ' - 'doctree directory.') + raise SphinxError(__('This environment is incompatible with the ' + 'selected builder, please choose another ' + 'doctree directory.')) self.versioning_condition = condition self.versioning_compare = compare @@ -304,6 +243,19 @@ class BuildEnvironment(object): """Like :meth:`warn`, but with source information taken from *node*.""" self._warnfunc(msg, '%s:%s' % get_source_line(node), **kwargs) + def need_refresh(self, app): + # type: (Sphinx) -> Tuple[bool, unicode] + """Check refresh environment is needed. + + If needed, this method returns the reason for refresh. + """ + if self.version != app.registry.get_envversion(app): + return True, __('build environment version not current') + elif self.srcdir != app.srcdir: + return True, __('source directory has changed') + else: + return False, None + def clear_doc(self, docname): # type: (unicode) -> None """Remove all traces of a source file in the inventory.""" @@ -419,7 +371,7 @@ class BuildEnvironment(object): if os.access(self.doc2path(docname), os.R_OK): self.found_docs.add(docname) else: - logger.warning("document not readable. Ignored.", location=docname) + logger.warning(__("document not readable. Ignored."), location=docname) # Current implementation is applying translated messages in the reading # phase.Therefore, in order to apply the updated message catalog, it is @@ -490,138 +442,52 @@ class BuildEnvironment(object): return added, changed, removed - def update(self, config, srcdir, doctreedir): - # type: (Config, unicode, unicode) -> List[unicode] - """(Re-)read all files new or changed since last update. + def check_dependents(self, app, already): + # type: (Sphinx, Set[unicode]) -> Iterator[unicode] + to_rewrite = [] # type: List[unicode] + for docnames in app.emit('env-get-updated', self): + to_rewrite.extend(docnames) + for docname in set(to_rewrite): + if docname not in already: + yield docname - Store all environment docnames in the canonical format (ie using SEP as - a separator in place of os.path.sep). - """ - config_changed = False + def update_config(self, config, srcdir, doctreedir): + # type: (Config, unicode, unicode) -> Tuple[bool, unicode] + """Update configurations by new one.""" + changed_reason = '' if self.config is None: - msg = '[new config] ' - config_changed = True + changed_reason = __('new config') else: # check if a config value was changed that affects how # doctrees are read for confval in config.filter('env'): if self.config[confval.name] != confval.value: - msg = '[config changed] ' - config_changed = True + changed_reason = __('config changed') break - else: - msg = '' + # this value is not covered by the above loop because it is handled # specially by the config class if self.config.extensions != config.extensions: - msg = '[extensions changed] ' - config_changed = True + changed_reason = __('extensions changed') + # the source and doctree directories may have been relocated self.srcdir = srcdir self.doctreedir = doctreedir - self.find_files(config, self.app.builder) self.config = config + self._update_settings(config) - # this cache also needs to be updated every time - self._nitpick_ignore = set(self.config.nitpick_ignore) + # return tuple of (changed, reason) + return bool(changed_reason), changed_reason - logger.info(bold('updating environment: '), nonl=True) + def _update_settings(self, config): + # type: (Config) -> None + """Update settings by new config.""" + self.settings['input_encoding'] = config.source_encoding + self.settings['trim_footnote_reference_space'] = config.trim_footnote_reference_space + self.settings['language_code'] = config.language or 'en' - added, changed, removed = self.get_outdated_files(config_changed) - - # allow user intervention as well - for docs in self.app.emit('env-get-outdated', self, added, changed, removed): - changed.update(set(docs) & self.found_docs) - - # if files were added or removed, all documents with globbed toctrees - # must be reread - if added or removed: - # ... but not those that already were removed - changed.update(self.glob_toctrees & self.found_docs) - - msg += '%s added, %s changed, %s removed' % (len(added), len(changed), - len(removed)) - logger.info(msg) - - # clear all files no longer present - for docname in removed: - self.app.emit('env-purge-doc', self, docname) - self.clear_doc(docname) - - # read all new and changed files - docnames = sorted(added | changed) - # allow changing and reordering the list of docs to read - self.app.emit('env-before-read-docs', self, docnames) - - # check if we should do parallel or serial read - if parallel_available and len(docnames) > 5 and self.app.parallel > 1: - par_ok = self.app.is_parallel_allowed('read') - else: - par_ok = False - - if par_ok: - self._read_parallel(docnames, self.app, nproc=self.app.parallel) - else: - self._read_serial(docnames, self.app) - - if config.master_doc not in self.all_docs: - raise SphinxError('master file %s not found' % - self.doc2path(config.master_doc)) - - for retval in self.app.emit('env-updated', self): - if retval is not None: - docnames.extend(retval) - - return sorted(docnames) - - def _read_serial(self, docnames, app): - # type: (List[unicode], Sphinx) -> None - for docname in status_iterator(docnames, 'reading sources... ', "purple", - len(docnames), self.app.verbosity): - # remove all inventory entries for that file - app.emit('env-purge-doc', self, docname) - self.clear_doc(docname) - self.read_doc(docname, app) - - def _read_parallel(self, docnames, app, nproc): - # type: (List[unicode], Sphinx, int) -> None - # clear all outdated docs at once - for docname in docnames: - app.emit('env-purge-doc', self, docname) - self.clear_doc(docname) - - def read_process(docs): - # type: (List[unicode]) -> unicode - self.app = app - for docname in docs: - self.read_doc(docname, app) - # allow pickling self to send it back - return BuildEnvironment.dumps(self) - - def merge(docs, otherenv): - # type: (List[unicode], unicode) -> None - env = BuildEnvironment.loads(otherenv) - self.merge_info_from(docs, env, app) - - tasks = ParallelTasks(nproc) - chunks = make_chunks(docnames, nproc) - - for chunk in status_iterator(chunks, 'reading sources... ', "purple", - len(chunks), self.app.verbosity): - tasks.add_task(read_process, chunk, merge) - - # make sure all threads have finished - logger.info(bold('waiting for workers...')) - tasks.join() - - def check_dependents(self, app, already): - # type: (Sphinx, Set[unicode]) -> Iterator[unicode] - to_rewrite = [] # type: List[unicode] - for docnames in app.emit('env-get-updated', self): - to_rewrite.extend(docnames) - for docname in set(to_rewrite): - if docname not in already: - yield docname + # Allow to disable by 3rd party extension (workaround) + self.settings.setdefault('smart_quotes', True) # --------- SINGLE FILE READING -------------------------------------------- @@ -634,55 +500,6 @@ class BuildEnvironment(object): self.temp_data['default_domain'] = \ self.domains.get(self.config.primary_domain) - self.settings['input_encoding'] = self.config.source_encoding - self.settings['trim_footnote_reference_space'] = \ - self.config.trim_footnote_reference_space - self.settings['gettext_compact'] = self.config.gettext_compact - - self.settings['language_code'] = self.config.language or 'en' - - # Allow to disable by 3rd party extension (workaround) - self.settings.setdefault('smart_quotes', True) - - def read_doc(self, docname, app=None): - # type: (unicode, Sphinx) -> None - """Parse a file and add/update inventory entries for the doctree.""" - self.prepare_settings(docname) - - docutilsconf = path.join(self.srcdir, 'docutils.conf') - # read docutils.conf from source dir, not from current dir - OptionParser.standard_config_files[1] = docutilsconf - if path.isfile(docutilsconf): - self.note_dependency(docutilsconf) - - with sphinx_domains(self), rst.default_role(docname, self.config.default_role): - doctree = read_doc(self.app, self, self.doc2path(docname)) - - # post-processing - for domain in itervalues(self.domains): - domain.process_doc(self, docname, doctree) - - # allow extension-specific post-processing - if app: - app.emit('doctree-read', doctree) - - # store time of reading, for outdated files detection - # (Some filesystems have coarse timestamp resolution; - # therefore time.time() can be older than filesystem's timestamp. - # For example, FAT32 has 2sec timestamp resolution.) - self.all_docs[docname] = max( - time.time(), path.getmtime(self.doc2path(docname))) - - if self.versioning_condition: - # add uids for versioning - versioning.prepare(doctree) - - # cleanup - self.temp_data.clear() - self.ref_context.clear() - - self.write_doctree(docname, doctree) - # utilities to use while reading a document @property @@ -771,7 +588,7 @@ class BuildEnvironment(object): try: return self.domains[domainname] except KeyError: - raise ExtensionError('Domain %r is not registered' % domainname) + raise ExtensionError(__('Domain %r is not registered') % domainname) # --------- RESOLVING REFERENCES AND TOCTREES ------------------------------ @@ -782,24 +599,9 @@ class BuildEnvironment(object): with open(doctree_filename, 'rb') as f: doctree = pickle.load(f) doctree.settings.env = self - doctree.reporter = Reporter(self.doc2path(docname), 2, 5, stream=WarningStream()) + doctree.reporter = LoggingReporter(self.doc2path(docname)) return doctree - def write_doctree(self, docname, doctree): - # type: (unicode, nodes.Node) -> None - """Write the doctree to a file.""" - # make it picklable - doctree.reporter = None - doctree.transformer = None - doctree.settings.warning_stream = None - doctree.settings.env = None - doctree.settings.record_dependencies = None - - doctree_filename = self.doc2path(docname, self.doctreedir, '.doctree') - ensuredir(path.dirname(doctree_filename)) - with open(doctree_filename, 'wb') as f: - pickle.dump(doctree, f, pickle.HIGHEST_PROTOCOL) - def get_and_resolve_doctree(self, docname, builder, doctree=None, prune_toctrees=True, includehidden=False): # type: (unicode, Builder, nodes.Node, bool, bool) -> nodes.Node @@ -881,7 +683,7 @@ class BuildEnvironment(object): def traverse_toctree(parent, docname): # type: (unicode, unicode) -> Iterator[Tuple[unicode, unicode]] if parent == docname: - logger.warning('self referenced toctree found. Ignored.', location=docname) + logger.warning(__('self referenced toctree found. Ignored.'), location=docname) return # traverse toctree by pre-order @@ -921,10 +723,112 @@ class BuildEnvironment(object): continue if 'orphan' in self.metadata[docname]: continue - logger.warning('document isn\'t included in any toctree', + logger.warning(__('document isn\'t included in any toctree'), location=docname) # call check-consistency for all extensions for domain in self.domains.values(): domain.check_consistency() self.app.emit('env-check-consistency', self) + + # --------- METHODS FOR COMPATIBILITY -------------------------------------- + + def update(self, config, srcdir, doctreedir): + # type: (Config, unicode, unicode) -> List[unicode] + warnings.warn('env.update() is deprecated. Please use builder.read() instead.', + RemovedInSphinx30Warning) + return self.app.builder.read() + + def _read_serial(self, docnames, app): + # type: (List[unicode], Sphinx) -> None + warnings.warn('env._read_serial() is deprecated. Please use builder.read() instead.', + RemovedInSphinx30Warning) + return self.app.builder._read_serial(docnames) + + def _read_parallel(self, docnames, app, nproc): + # type: (List[unicode], Sphinx, int) -> None + warnings.warn('env._read_parallel() is deprecated. Please use builder.read() instead.', + RemovedInSphinx30Warning) + return self.app.builder._read_parallel(docnames, nproc) + + def read_doc(self, docname, app=None): + # type: (unicode, Sphinx) -> None + warnings.warn('env.read_doc() is deprecated. Please use builder.read_doc() instead.', + RemovedInSphinx30Warning) + self.app.builder.read_doc(docname) + + def write_doctree(self, docname, doctree): + # type: (unicode, nodes.Node) -> None + warnings.warn('env.write_doctree() is deprecated. ' + 'Please use builder.write_doctree() instead.', + RemovedInSphinx30Warning) + self.app.builder.write_doctree(docname, doctree) + + @property + def _nitpick_ignore(self): + # type: () -> List[unicode] + warnings.warn('env._nitpick_ignore is deprecated. ' + 'Please use config.nitpick_ignore instead.', + RemovedInSphinx30Warning) + return self.config.nitpick_ignore + + @staticmethod + def load(f, app=None): + # type: (IO, Sphinx) -> BuildEnvironment + warnings.warn('BuildEnvironment.load() is deprecated. ' + 'Please use pickle.load() instead.', + RemovedInSphinx30Warning) + try: + env = pickle.load(f) + except Exception as exc: + # This can happen for example when the pickle is from a + # different version of Sphinx. + raise IOError(exc) + if app: + env.app = app + env.config.values = app.config.values + return env + + @classmethod + def loads(cls, string, app=None): + # type: (unicode, Sphinx) -> BuildEnvironment + warnings.warn('BuildEnvironment.loads() is deprecated. ' + 'Please use pickle.loads() instead.', + RemovedInSphinx30Warning) + io = BytesIO(string) + return cls.load(io, app) + + @classmethod + def frompickle(cls, filename, app): + # type: (unicode, Sphinx) -> BuildEnvironment + warnings.warn('BuildEnvironment.frompickle() is deprecated. ' + 'Please use pickle.load() instead.', + RemovedInSphinx30Warning) + with open(filename, 'rb') as f: + return cls.load(f, app) + + @staticmethod + def dump(env, f): + # type: (BuildEnvironment, IO) -> None + warnings.warn('BuildEnvironment.dump() is deprecated. ' + 'Please use pickle.dump() instead.', + RemovedInSphinx30Warning) + pickle.dump(env, f, pickle.HIGHEST_PROTOCOL) + + @classmethod + def dumps(cls, env): + # type: (BuildEnvironment) -> unicode + warnings.warn('BuildEnvironment.dumps() is deprecated. ' + 'Please use pickle.dumps() instead.', + RemovedInSphinx30Warning) + io = BytesIO() + cls.dump(env, io) + return io.getvalue() + + def topickle(self, filename): + # type: (unicode) -> None + warnings.warn('env.topickle() is deprecated. ' + 'Please use pickle.dump() instead.', + RemovedInSphinx30Warning) + with open(filename, 'wb') as f: + self.dump(self, f) diff --git a/sphinx/environment/adapters/indexentries.py b/sphinx/environment/adapters/indexentries.py index 5ac8ff1d6..7c31fc3d5 100644 --- a/sphinx/environment/adapters/indexentries.py +++ b/sphinx/environment/adapters/indexentries.py @@ -15,7 +15,7 @@ from itertools import groupby from six import text_type, iteritems -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import split_into, logging if False: @@ -89,7 +89,7 @@ class IndexEntries(object): add_entry(first, _('see also %s') % second, None, link=False, key=index_key) else: - logger.warning('unknown index entry type %r', type, location=fn) + logger.warning(__('unknown index entry type %r'), type, location=fn) except ValueError as err: logger.warning(str(err), location=fn) diff --git a/sphinx/environment/adapters/toctree.py b/sphinx/environment/adapters/toctree.py index af5d85f23..f1261bcdc 100644 --- a/sphinx/environment/adapters/toctree.py +++ b/sphinx/environment/adapters/toctree.py @@ -13,6 +13,7 @@ from docutils import nodes from six import iteritems from sphinx import addnodes +from sphinx.locale import __ from sphinx.util import url_re, logging from sphinx.util.nodes import clean_astext, process_only_nodes @@ -147,8 +148,8 @@ class TocTree(object): toc = nodes.bullet_list('', item) else: if ref in parents: - logger.warning('circular toctree references ' - 'detected, ignoring: %s <- %s', + logger.warning(__('circular toctree references ' + 'detected, ignoring: %s <- %s'), ref, ' <- '.join(parents), location=ref) continue @@ -166,12 +167,12 @@ class TocTree(object): refnode.children = [nodes.Text(title)] if not toc.children: # empty toc means: no titles will show up in the toctree - logger.warning('toctree contains reference to document %r that ' - 'doesn\'t have a title: no link will be generated', + logger.warning(__('toctree contains reference to document %r that ' + 'doesn\'t have a title: no link will be generated'), ref, location=toctreenode) except KeyError: # this is raised if the included file does not exist - logger.warning('toctree contains reference to nonexisting document %r', + logger.warning(__('toctree contains reference to nonexisting document %r'), ref, location=toctreenode) else: # if titles_only is given, only keep the main title and diff --git a/sphinx/environment/collectors/asset.py b/sphinx/environment/collectors/asset.py index 2504a7ea4..70faf108c 100644 --- a/sphinx/environment/collectors/asset.py +++ b/sphinx/environment/collectors/asset.py @@ -19,6 +19,7 @@ from six import iteritems, itervalues from sphinx import addnodes from sphinx.environment.collectors import EnvironmentCollector +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.i18n import get_image_filename_for_language, search_image_for_language from sphinx.util.images import guess_mimetype @@ -89,7 +90,7 @@ class ImageCollector(EnvironmentCollector): for imgpath in itervalues(candidates): app.env.dependencies[docname].add(imgpath) if not os.access(path.join(app.srcdir, imgpath), os.R_OK): - logger.warning('image file not readable: %s' % imgpath, + logger.warning(__('image file not readable: %s') % imgpath, location=node, type='image', subtype='not_readable') continue app.env.images.add_file(docname, imgpath) @@ -105,7 +106,7 @@ class ImageCollector(EnvironmentCollector): if mimetype not in candidates: globbed.setdefault(mimetype, []).append(new_imgpath) except (OSError, IOError) as err: - logger.warning('image file %s not readable: %s' % (filename, err), + logger.warning(__('image file %s not readable: %s') % (filename, err), location=node, type='image', subtype='not_readable') for key, files in iteritems(globbed): candidates[key] = sorted(files, key=len)[0] # select by similarity @@ -130,7 +131,7 @@ class DownloadFileCollector(EnvironmentCollector): rel_filename, filename = app.env.relfn2path(targetname, app.env.docname) app.env.dependencies[app.env.docname].add(rel_filename) if not os.access(filename, os.R_OK): - logger.warning('download file not readable: %s' % filename, + logger.warning(__('download file not readable: %s') % filename, location=node, type='download', subtype='not_readable') continue node['filename'] = app.env.dlfiles.add_file(app.env.docname, filename) diff --git a/sphinx/environment/collectors/indexentries.py b/sphinx/environment/collectors/indexentries.py index dec5dbc75..a9ba897d0 100644 --- a/sphinx/environment/collectors/indexentries.py +++ b/sphinx/environment/collectors/indexentries.py @@ -9,6 +9,8 @@ :license: BSD, see LICENSE for details. """ +from six import text_type + from sphinx import addnodes from sphinx.environment.collectors import EnvironmentCollector from sphinx.util import split_index_msg, logging @@ -44,7 +46,7 @@ class IndexEntriesCollector(EnvironmentCollector): for entry in node['entries']: split_index_msg(entry[0], entry[1]) except ValueError as exc: - logger.warning(str(exc), location=node) + logger.warning(text_type(exc), location=node) node.parent.remove(node) else: for entry in node['entries']: diff --git a/sphinx/environment/collectors/toctree.py b/sphinx/environment/collectors/toctree.py index a7556eadd..dcbee07f7 100644 --- a/sphinx/environment/collectors/toctree.py +++ b/sphinx/environment/collectors/toctree.py @@ -15,6 +15,7 @@ from six import iteritems from sphinx import addnodes from sphinx.environment.adapters.toctree import TocTree from sphinx.environment.collectors import EnvironmentCollector +from sphinx.locale import __ from sphinx.transforms import SphinxContentsFilter from sphinx.util import url_re, logging @@ -66,6 +67,7 @@ class TocTreeCollector(EnvironmentCollector): numentries = [0] # nonlocal again... def traverse_in_section(node, cls): + # type: (nodes.Node, Any) -> List[nodes.Node] """Like traverse(), but stay within the same section.""" result = [] if isinstance(node, cls): @@ -77,6 +79,7 @@ class TocTreeCollector(EnvironmentCollector): return result def build_toc(node, depth=1): + # type: (nodes.Node, int) -> List[nodes.Node] entries = [] for sectionnode in node: # find all toctree nodes in this section and add them @@ -86,7 +89,7 @@ class TocTreeCollector(EnvironmentCollector): onlynode = addnodes.only(expr=sectionnode['expr']) blist = build_toc(sectionnode, depth) if blist: - onlynode += blist.children + onlynode += blist.children # type: ignore entries.append(onlynode) continue if not isinstance(sectionnode, nodes.section): @@ -145,6 +148,7 @@ class TocTreeCollector(EnvironmentCollector): env.toc_secnumbers = {} def _walk_toc(node, secnums, depth, titlenode=None): + # type: (nodes.Node, Dict, int, nodes.Node) -> None # titlenode is the title of the document, it will get assigned a # secnumber too, so that it shows up in next/prev/parent rellinks for subnode in node.children: @@ -177,6 +181,7 @@ class TocTreeCollector(EnvironmentCollector): _walk_toctree(subnode, depth) def _walk_toctree(toctreenode, depth): + # type: (nodes.Node, int) -> None if depth == 0: return for (title, ref) in toctreenode['entries']: @@ -184,8 +189,8 @@ class TocTreeCollector(EnvironmentCollector): # don't mess with those continue elif ref in assigned: - logger.warning('%s is already assigned section numbers ' - '(nested numbered toctree?)', ref, + logger.warning(__('%s is already assigned section numbers ' + '(nested numbered toctree?)'), ref, location=toctreenode, type='toc', subtype='secnum') elif ref in env.tocs: secnums = env.toc_secnumbers[ref] = {} @@ -216,9 +221,19 @@ class TocTreeCollector(EnvironmentCollector): assigned = set() # type: Set[unicode] old_fignumbers = env.toc_fignumbers env.toc_fignumbers = {} - fignum_counter = {} # type: Dict[unicode, Dict[Tuple[int], int]] + fignum_counter = {} # type: Dict[unicode, Dict[Tuple[int, ...], int]] + + def get_figtype(node): + # type: (nodes.Node) -> unicode + for domain in env.domains.values(): + figtype = domain.get_enumerable_node_type(node) + if figtype: + return figtype + + return None def get_section_number(docname, section): + # type: (unicode, nodes.Node) -> Tuple[int, ...] anchorname = '#' + section['ids'][0] secnumbers = env.toc_secnumbers.get(docname, {}) if anchorname in secnumbers: @@ -229,6 +244,7 @@ class TocTreeCollector(EnvironmentCollector): return secnum or tuple() def get_next_fignumber(figtype, secnum): + # type: (unicode, Tuple[int, ...]) -> Tuple[int, ...] counter = fignum_counter.setdefault(figtype, {}) secnum = secnum[:env.config.numfig_secnum_depth] @@ -236,6 +252,7 @@ class TocTreeCollector(EnvironmentCollector): return secnum + (counter[secnum],) def register_fignumber(docname, secnum, figtype, fignode): + # type: (unicode, Tuple[int], unicode, nodes.Node) -> None env.toc_fignumbers.setdefault(docname, {}) fignumbers = env.toc_fignumbers[docname].setdefault(figtype, {}) figure_id = fignode['ids'][0] @@ -243,6 +260,7 @@ class TocTreeCollector(EnvironmentCollector): fignumbers[figure_id] = get_next_fignumber(figtype, secnum) def _walk_doctree(docname, doctree, secnum): + # type: (unicode, nodes.Node, Tuple[int, ...]) -> None for subnode in doctree.children: if isinstance(subnode, nodes.section): next_secnum = get_section_number(docname, subnode) @@ -261,20 +279,21 @@ class TocTreeCollector(EnvironmentCollector): continue - figtype = env.get_domain('std').get_figtype(subnode) + figtype = get_figtype(subnode) if figtype and subnode['ids']: register_fignumber(docname, secnum, figtype, subnode) _walk_doctree(docname, subnode, secnum) def _walk_doc(docname, secnum): + # type: (unicode, Tuple[int]) -> None if docname not in assigned: assigned.add(docname) doctree = env.get_doctree(docname) _walk_doctree(docname, doctree, secnum) if env.config.numfig: - _walk_doc(env.config.master_doc, tuple()) + _walk_doc(env.config.master_doc, tuple()) # type: ignore for docname, fignums in iteritems(env.toc_fignumbers): if fignums != old_fignumbers.get(docname): rewrite_needed.append(docname) diff --git a/sphinx/errors.py b/sphinx/errors.py index eef1a157a..7652e93cb 100644 --- a/sphinx/errors.py +++ b/sphinx/errors.py @@ -16,20 +16,40 @@ if False: class SphinxError(Exception): - """ - Base class for Sphinx errors that are shown to the user in a nicer - way than normal exceptions. + """Base class for Sphinx errors. + + This is the base class for "nice" exceptions. When such an exception is + raised, Sphinx will abort the build and present the exception category and + message to the user. + + Extensions are encouraged to derive from this exception for their custom + errors. + + Exceptions *not* derived from :exc:`SphinxError` are treated as unexpected + and shown to the user with a part of the traceback (and the full traceback + saved in a temporary file). + + .. attribute:: category + + Description of the exception "category", used in converting the + exception to a string ("category: message"). Should be set accordingly + in subclasses. """ category = 'Sphinx error' class SphinxWarning(SphinxError): - """Raised for warnings if warnings are treated as errors.""" + """Warning, treated as error.""" category = 'Warning, treated as error' +class ApplicationError(SphinxError): + """Application initialization error.""" + category = 'Application error' + + class ExtensionError(SphinxError): - """Raised if something's wrong with the configuration.""" + """Extension error.""" category = 'Extension error' def __init__(self, message, orig_exc=None): @@ -53,27 +73,22 @@ class ExtensionError(SphinxError): class ConfigError(SphinxError): + """Configuration error.""" category = 'Configuration error' class ThemeError(SphinxError): + """Theme error.""" category = 'Theme error' class VersionRequirementError(SphinxError): + """Incompatible Sphinx version error.""" category = 'Sphinx version error' -class PycodeError(Exception): - def __str__(self): - # type: () -> str - res = self.args[0] - if len(self.args) > 1: - res += ' (exception was: %r)' % self.args[1] - return res - - class SphinxParallelError(SphinxError): + """Sphinx parallel build error.""" category = 'Sphinx parallel build error' @@ -85,3 +100,14 @@ class SphinxParallelError(SphinxError): def __str__(self): # type: () -> str return self.message + + +class PycodeError(Exception): + """Pycode Python source code analyser error.""" + + def __str__(self): + # type: () -> str + res = self.args[0] + if len(self.args) > 1: + res += ' (exception was: %r)' % self.args[1] + return res diff --git a/sphinx/events.py b/sphinx/events.py index 097f61fc6..fb62d1776 100644 --- a/sphinx/events.py +++ b/sphinx/events.py @@ -27,6 +27,7 @@ if False: # List of all known core events. Maps name to arguments description. core_events = { 'builder-inited': '', + 'config-inited': 'config', 'env-get-outdated': 'env, added, changed, removed', 'env-get-updated': 'env', 'env-purge-doc': 'env, docname', diff --git a/sphinx/ext/apidoc.py b/sphinx/ext/apidoc.py index a01439121..8a7c68402 100644 --- a/sphinx/ext/apidoc.py +++ b/sphinx/ext/apidoc.py @@ -9,7 +9,7 @@ This is derived from the "sphinx-autopackage" script, which is: Copyright 2008 Société des arts technologiques (SAT), - http://www.sat.qc.ca/ + https://sat.qc.ca/ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. @@ -19,6 +19,7 @@ from __future__ import print_function import argparse import glob +import locale import os import sys from fnmatch import fnmatch @@ -26,8 +27,10 @@ from os import path from six import binary_type -from sphinx import __display_version__ +import sphinx.locale +from sphinx import __display_version__, package_dir from sphinx.cmd.quickstart import EXTENSIONS +from sphinx.locale import __ from sphinx.util import rst from sphinx.util.osutil import FileAvoidWrite, ensuredir, walk @@ -68,12 +71,12 @@ def write_file(name, text, opts): """Write the output file for module/package <name>.""" fname = path.join(opts.destdir, '%s.%s' % (name, opts.suffix)) if opts.dryrun: - print('Would create file %s.' % fname) + print(__('Would create file %s.') % fname) return if not opts.force and path.isfile(fname): - print('File %s already exists, skipping.' % fname) + print(__('File %s already exists, skipping.') % fname) else: - print('Creating file %s.' % fname) + print(__('Creating file %s.') % fname) with FileAvoidWrite(fname) as f: f.write(text) @@ -284,7 +287,7 @@ def is_excluded(root, excludes): """Check if the directory is in the exclude list. Note: by having trailing slashes, we avoid common prefix issues, like - e.g. an exlude "foo" also accidentally excluding "foobar". + e.g. an exclude "foo" also accidentally excluding "foobar". """ for exclude in excludes: if fnmatch(root, exclude): @@ -297,84 +300,84 @@ def get_parser(): parser = argparse.ArgumentParser( usage='%(prog)s [OPTIONS] -o <OUTPUT_PATH> <MODULE_PATH> ' '[EXCLUDE_PATTERN, ...]', - epilog='For more information, visit <http://sphinx-doc.org/>.', - description=""" + epilog=__('For more information, visit <http://sphinx-doc.org/>.'), + description=__(""" Look recursively in <MODULE_PATH> for Python modules and packages and create one reST file with automodule directives per package in the <OUTPUT_PATH>. The <EXCLUDE_PATTERN>s can be file and/or directory patterns that will be excluded from generation. -Note: By default this script will not overwrite already created files.""") +Note: By default this script will not overwrite already created files.""")) parser.add_argument('--version', action='version', dest='show_version', version='%%(prog)s %s' % __display_version__) parser.add_argument('module_path', - help='path to module to document') + help=__('path to module to document')) parser.add_argument('exclude_pattern', nargs='*', - help='fnmatch-style file and/or directory patterns ' - 'to exclude from generation') + help=__('fnmatch-style file and/or directory patterns ' + 'to exclude from generation')) parser.add_argument('-o', '--output-dir', action='store', dest='destdir', required=True, - help='directory to place all output') + help=__('directory to place all output')) parser.add_argument('-d', '--maxdepth', action='store', dest='maxdepth', type=int, default=4, - help='maximum depth of submodules to show in the TOC ' - '(default: 4)') + help=__('maximum depth of submodules to show in the TOC ' + '(default: 4)')) parser.add_argument('-f', '--force', action='store_true', dest='force', - help='overwrite existing files') + help=__('overwrite existing files')) parser.add_argument('-l', '--follow-links', action='store_true', dest='followlinks', default=False, - help='follow symbolic links. Powerful when combined ' - 'with collective.recipe.omelette.') + help=__('follow symbolic links. Powerful when combined ' + 'with collective.recipe.omelette.')) parser.add_argument('-n', '--dry-run', action='store_true', dest='dryrun', - help='run the script without creating files') + help=__('run the script without creating files')) parser.add_argument('-e', '--separate', action='store_true', dest='separatemodules', - help='put documentation for each module on its own page') + help=__('put documentation for each module on its own page')) parser.add_argument('-P', '--private', action='store_true', dest='includeprivate', - help='include "_private" modules') + help=__('include "_private" modules')) parser.add_argument('-T', '--no-toc', action='store_true', dest='notoc', - help="don't create a table of contents file") + help=__("don't create a table of contents file")) parser.add_argument('-E', '--no-headings', action='store_true', dest='noheadings', - help="don't create headings for the module/package " - "packages (e.g. when the docstrings already " - "contain them)") + help=__("don't create headings for the module/package " + "packages (e.g. when the docstrings already " + "contain them)")) parser.add_argument('-M', '--module-first', action='store_true', dest='modulefirst', - help='put module documentation before submodule ' - 'documentation') + help=__('put module documentation before submodule ' + 'documentation')) parser.add_argument('--implicit-namespaces', action='store_true', dest='implicit_namespaces', - help='interpret module paths according to PEP-0420 ' - 'implicit namespaces specification') + help=__('interpret module paths according to PEP-0420 ' + 'implicit namespaces specification')) parser.add_argument('-s', '--suffix', action='store', dest='suffix', default='rst', - help='file suffix (default: rst)') + help=__('file suffix (default: rst)')) parser.add_argument('-F', '--full', action='store_true', dest='full', - help='generate a full project with sphinx-quickstart') + help=__('generate a full project with sphinx-quickstart')) parser.add_argument('-a', '--append-syspath', action='store_true', dest='append_syspath', - help='append module_path to sys.path, used when --full is given') + help=__('append module_path to sys.path, used when --full is given')) parser.add_argument('-H', '--doc-project', action='store', dest='header', - help='project name (default: root module name)') + help=__('project name (default: root module name)')) parser.add_argument('-A', '--doc-author', action='store', dest='author', - help='project author(s), used when --full is given') + help=__('project author(s), used when --full is given')) parser.add_argument('-V', '--doc-version', action='store', dest='version', - help='project version, used when --full is given') + help=__('project version, used when --full is given')) parser.add_argument('-R', '--doc-release', action='store', dest='release', - help='project release, used when --full is given, ' - 'defaults to --doc-version') + help=__('project release, used when --full is given, ' + 'defaults to --doc-version')) - group = parser.add_argument_group('extension options') + group = parser.add_argument_group(__('extension options')) for ext in EXTENSIONS: group.add_argument('--ext-%s' % ext, action='append_const', const='sphinx.ext.%s' % ext, dest='extensions', - help='enable %s extension' % ext) + help=__('enable %s extension') % ext) return parser @@ -382,6 +385,9 @@ Note: By default this script will not overwrite already created files.""") def main(argv=sys.argv[1:]): # type: (List[str]) -> int """Parse and check the command line arguments.""" + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') + parser = get_parser() args = parser.parse_args(argv) @@ -394,7 +400,7 @@ def main(argv=sys.argv[1:]): if args.suffix.startswith('.'): args.suffix = args.suffix[1:] if not path.isdir(rootpath): - print('%s is not a directory.' % rootpath, file=sys.stderr) + print(__('%s is not a directory.') % rootpath, file=sys.stderr) sys.exit(1) if not args.dryrun: ensuredir(args.destdir) diff --git a/sphinx/ext/autodoc/__init__.py b/sphinx/ext/autodoc/__init__.py index d516146bf..1ebf6399b 100644 --- a/sphinx/ext/autodoc/__init__.py +++ b/sphinx/ext/autodoc/__init__.py @@ -20,12 +20,12 @@ from docutils.statemachine import ViewList from six import iteritems, itervalues, text_type, class_types, string_types import sphinx -from sphinx.application import ExtensionError from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx.errors import ExtensionError from sphinx.ext.autodoc.importer import mock, import_object, get_object_members from sphinx.ext.autodoc.importer import _MockImporter # to keep compatibility # NOQA from sphinx.ext.autodoc.inspector import format_annotation, formatargspec # to keep compatibility # NOQA -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.pycode import ModuleAnalyzer, PycodeError from sphinx.util import logging from sphinx.util import rpartition, force_decode @@ -340,7 +340,7 @@ class Documenter(object): explicit_modname, path, base, args, retann = \ py_ext_sig_re.match(self.name).groups() # type: ignore except AttributeError: - logger.warning('invalid signature for auto%s (%r)' % (self.objtype, self.name)) + logger.warning(__('invalid signature for auto%s (%r)') % (self.objtype, self.name)) return False # support explicit module and class name separation via :: @@ -437,7 +437,7 @@ class Documenter(object): try: args = self.format_args() except Exception as err: - logger.warning('error while formatting arguments for %s: %s' % + logger.warning(__('error while formatting arguments for %s: %s') % (self.fullname, err)) args = None @@ -560,7 +560,7 @@ class Documenter(object): if name in members: selected.append((name, members[name].value)) else: - logger.warning('missing attribute %s in object %s' % + logger.warning(__('missing attribute %s in object %s') % (name, self.fullname)) return False, sorted(selected) elif self.options.inherited_members: @@ -731,9 +731,9 @@ class Documenter(object): if not self.parse_name(): # need a module to import logger.warning( - 'don\'t know which module to import for autodocumenting ' - '%r (try placing a "module" or "currentmodule" directive ' - 'in the document, or giving an explicit module name)' % + __('don\'t know which module to import for autodocumenting ' + '%r (try placing a "module" or "currentmodule" directive ' + 'in the document, or giving an explicit module name)') % self.name) return @@ -820,15 +820,15 @@ class ModuleDocumenter(Documenter): def resolve_name(self, modname, parents, path, base): # type: (str, Any, str, Any) -> Tuple[str, List[unicode]] if modname is not None: - logger.warning('"::" in automodule name doesn\'t make sense') + logger.warning(__('"::" in automodule name doesn\'t make sense')) return (path or '') + base, [] def parse_name(self): # type: () -> bool ret = Documenter.parse_name(self) if self.args or self.retann: - logger.warning('signature arguments or return annotation ' - 'given for automodule %s' % self.fullname) + logger.warning(__('signature arguments or return annotation ' + 'given for automodule %s') % self.fullname) return ret def add_directive_header(self, sig): @@ -861,8 +861,8 @@ class ModuleDocumenter(Documenter): if not isinstance(memberlist, (list, tuple)) or not \ all(isinstance(entry, string_types) for entry in memberlist): logger.warning( - '__all__ should be a list of strings, not %r ' - '(in module %s) -- ignoring __all__' % + __('__all__ should be a list of strings, not %r ' + '(in module %s) -- ignoring __all__') % (memberlist, self.fullname)) # fall back to all members return True, safe_getmembers(self.object) @@ -874,8 +874,8 @@ class ModuleDocumenter(Documenter): ret.append((mname, safe_getattr(self.object, mname))) except AttributeError: logger.warning( - 'missing attribute mentioned in :members: or __all__: ' - 'module %s, attribute %s' % + __('missing attribute mentioned in :members: or __all__: ' + 'module %s, attribute %s') % (safe_getattr(self.object, '__name__', '???'), mname)) return False, ret @@ -1193,6 +1193,7 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # type: def generate(self, more_content=None, real_modname=None, check_module=False, all_members=False): + # type: (Any, str, bool, bool) -> None # Do not pass real_modname and use the name from the __module__ # attribute of the class. # If a class gets imported into the module real_modname @@ -1328,6 +1329,7 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): @staticmethod def is_function_or_method(obj): + # type: (Any) -> bool return inspect.isfunction(obj) or inspect.isbuiltin(obj) or inspect.ismethod(obj) @classmethod @@ -1428,18 +1430,22 @@ class InstanceAttributeDocumenter(AttributeDocumenter): class DeprecatedDict(dict): def __init__(self, message): + # type: (str) -> None self.message = message super(DeprecatedDict, self).__init__() def __setitem__(self, key, value): + # type: (unicode, Any) -> None warnings.warn(self.message, RemovedInSphinx20Warning) super(DeprecatedDict, self).__setitem__(key, value) def setdefault(self, key, default=None): + # type: (unicode, Any) -> None warnings.warn(self.message, RemovedInSphinx20Warning) super(DeprecatedDict, self).setdefault(key, default) - def update(self, other=None): + def update(self, other=None): # type: ignore + # type: (Dict) -> None warnings.warn(self.message, RemovedInSphinx20Warning) super(DeprecatedDict, self).update(other) @@ -1532,15 +1538,3 @@ def setup(app): app.add_event('autodoc-skip-member') return {'version': sphinx.__display_version__, 'parallel_read_safe': True} - - -class testcls: - """test doc string""" - - def __getattr__(self, x): - # type: (Any) -> Any - return x - - def __setattr__(self, x, y): - # type: (Any, Any) -> None - """Attr setter.""" diff --git a/sphinx/ext/autodoc/directive.py b/sphinx/ext/autodoc/directive.py index 77b3f7aaf..64d19fcc7 100644 --- a/sphinx/ext/autodoc/directive.py +++ b/sphinx/ext/autodoc/directive.py @@ -8,13 +8,12 @@ """ from docutils import nodes -from docutils.parsers.rst import Directive from docutils.statemachine import ViewList from docutils.utils import assemble_option_dict from sphinx.ext.autodoc import Options, get_documenters from sphinx.util import logging -from sphinx.util.docutils import switch_source_input +from sphinx.util.docutils import SphinxDirective, switch_source_input from sphinx.util.nodes import nested_parse_with_titles if False: @@ -91,7 +90,7 @@ def parse_generated_content(state, content, documenter): return node.children -class AutodocDirective(Directive): +class AutodocDirective(SphinxDirective): """A directive class for all autodoc directives. It works as a dispatcher of Documenters. It invokes a Documenter on running. After the processing, it parses and returns @@ -105,7 +104,6 @@ class AutodocDirective(Directive): def run(self): # type: () -> List[nodes.Node] - env = self.state.document.settings.env reporter = self.state.document.reporter try: @@ -116,11 +114,11 @@ class AutodocDirective(Directive): # look up target Documenter objtype = self.name[4:] # strip prefix (auto-). - doccls = get_documenters(env.app)[objtype] + doccls = get_documenters(self.env.app)[objtype] # process the options with the selected documenter's option_spec try: - documenter_options = process_documenter_options(doccls, env.config, self.options) + documenter_options = process_documenter_options(doccls, self.config, self.options) except (KeyError, ValueError, TypeError) as exc: # an option is either unknown or has a wrong type logger.error('An option to %s is either unknown or has an invalid value: %s' % @@ -128,7 +126,7 @@ class AutodocDirective(Directive): return [] # generate the output - params = DocumenterBridge(env, reporter, documenter_options, lineno) + params = DocumenterBridge(self.env, reporter, documenter_options, lineno) documenter = doccls(params, self.arguments[0]) documenter.generate(more_content=self.content) if not params.result: diff --git a/sphinx/ext/autodoc/importer.py b/sphinx/ext/autodoc/importer.py index 00a55c4c5..2cc8cbd1f 100644 --- a/sphinx/ext/autodoc/importer.py +++ b/sphinx/ext/autodoc/importer.py @@ -90,6 +90,7 @@ class _MockImporter(object): sys.meta_path.insert(0, self) def disable(self): + # type: () -> None # remove `self` from `sys.meta_path` to disable import hook sys.meta_path = [i for i in sys.meta_path if i is not self] # remove mocked modules from sys.modules to avoid side effects after @@ -130,6 +131,7 @@ def mock(names): def import_module(modname, warningiserror=False): + # type: (str, bool) -> Any """ Call __import__(modname), convert exceptions to ImportError """ diff --git a/sphinx/ext/autosectionlabel.py b/sphinx/ext/autosectionlabel.py index 6a3ff1422..5713828a4 100644 --- a/sphinx/ext/autosectionlabel.py +++ b/sphinx/ext/autosectionlabel.py @@ -11,9 +11,16 @@ from docutils import nodes +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.nodes import clean_astext +if False: + # For type annotation + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + + logger = logging.getLogger(__name__) if False: @@ -37,8 +44,8 @@ def register_sections_as_label(app, document): sectname = clean_astext(node[0]) if name in labels: - logger.warning('duplicate label %s, ' % name + 'other instance ' - 'in ' + app.env.doc2path(labels[name][0]), + logger.warning(__('duplicate label %s, other instance in %s'), + name, app.env.doc2path(labels[name][0]), location=node) anonlabels[name] = docname, labelid diff --git a/sphinx/ext/autosummary/__init__.py b/sphinx/ext/autosummary/__init__.py index a1b468a69..8c6bafbd9 100644 --- a/sphinx/ext/autosummary/__init__.py +++ b/sphinx/ext/autosummary/__init__.py @@ -62,7 +62,7 @@ import warnings from types import ModuleType from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from docutils.parsers.rst.states import RSTStateMachine, state_classes from docutils.statemachine import ViewList from six import string_types @@ -75,9 +75,10 @@ from sphinx.environment.adapters.toctree import TocTree from sphinx.ext.autodoc import get_documenters from sphinx.ext.autodoc.directive import DocumenterBridge, Options from sphinx.ext.autodoc.importer import import_module +from sphinx.locale import __ from sphinx.pycode import ModuleAnalyzer, PycodeError from sphinx.util import import_object, rst, logging -from sphinx.util.docutils import NullReporter, new_document +from sphinx.util.docutils import NullReporter, SphinxDirective, new_document if False: # For type annotation @@ -108,6 +109,7 @@ def process_autosummary_toc(app, doctree): crawled = {} def crawl_toc(node, depth=1): + # type: (nodes.Node, int) -> None crawled[node] = True for j, subnode in enumerate(node): try: @@ -166,6 +168,7 @@ _app = None # type: Sphinx class FakeDirective(DocumenterBridge): def __init__(self): + # type: () -> None super(FakeDirective, self).__init__({}, None, Options(), 0) # type: ignore @@ -217,7 +220,7 @@ def get_documenter(*args): # -- .. autosummary:: ---------------------------------------------------------- -class Autosummary(Directive): +class Autosummary(SphinxDirective): """ Pretty table containing short signatures and summaries of functions etc. @@ -241,7 +244,6 @@ class Autosummary(Directive): def run(self): # type: () -> List[nodes.Node] - self.env = env = self.state.document.settings.env self.genopt = Options() self.warnings = [] # type: List[nodes.Node] self.result = ViewList() @@ -252,14 +254,14 @@ class Autosummary(Directive): nodes = self.get_table(items) if 'toctree' in self.options: - dirname = posixpath.dirname(env.docname) + dirname = posixpath.dirname(self.env.docname) tree_prefix = self.options['toctree'].strip() docnames = [] for name, sig, summary, real_name in items: docname = posixpath.join(tree_prefix, real_name) docname = posixpath.normpath(posixpath.join(dirname, docname)) - if docname not in env.found_docs: + if docname not in self.env.found_docs: self.warn('toctree references unknown document %r' % docname) docnames.append(docname) @@ -280,9 +282,7 @@ class Autosummary(Directive): """Try to import the given names, and return a list of ``[(name, signature, summary_string, real_name), ...]``. """ - env = self.state.document.settings.env - - prefixes = get_import_prefixes_from_env(env) + prefixes = get_import_prefixes_from_env(self.env) items = [] # type: List[Tuple[unicode, unicode, unicode, unicode]] @@ -655,14 +655,14 @@ def process_generate_options(app): from sphinx.ext.autosummary.generate import generate_autosummary_docs - ext = app.config.source_suffix + ext = list(app.config.source_suffix) genfiles = [genfile + (not genfile.endswith(tuple(ext)) and ext[0] or '') for genfile in genfiles] suffix = get_rst_suffix(app) if suffix is None: - logger.warning('autosummary generats .rst files internally. ' - 'But your source_suffix does not contain .rst. Skipped.') + logger.warning(__('autosummary generats .rst files internally. ' + 'But your source_suffix does not contain .rst. Skipped.')) return generate_autosummary_docs(genfiles, builder=app.builder, diff --git a/sphinx/ext/autosummary/generate.py b/sphinx/ext/autosummary/generate.py index 2818ab3c7..4c9175a5d 100644 --- a/sphinx/ext/autosummary/generate.py +++ b/sphinx/ext/autosummary/generate.py @@ -21,6 +21,7 @@ from __future__ import print_function import argparse import codecs +import locale import os import pydoc import re @@ -29,10 +30,12 @@ import sys from jinja2 import FileSystemLoader, TemplateNotFound from jinja2.sandbox import SandboxedEnvironment +import sphinx.locale from sphinx import __display_version__ from sphinx import package_dir from sphinx.ext.autosummary import import_by_name, get_documenter from sphinx.jinja2glue import BuiltinTemplateLoader +from sphinx.locale import __ from sphinx.registry import SphinxComponentRegistry from sphinx.util.inspect import safe_getattr from sphinx.util.osutil import ensuredir @@ -82,6 +85,7 @@ def _simple_warn(msg): def _underline(title, line='='): + # type: (unicode, unicode) -> unicode if '\n' in title: raise ValueError('Can only underline single lines') return title + '\n' + line * len(title) @@ -98,11 +102,11 @@ def generate_autosummary_docs(sources, output_dir=None, suffix='.rst', showed_sources = list(sorted(sources)) if len(showed_sources) > 20: showed_sources = showed_sources[:10] + ['...'] + showed_sources[-10:] - info('[autosummary] generating autosummary for: %s' % + info(__('[autosummary] generating autosummary for: %s') % ', '.join(showed_sources)) if output_dir: - info('[autosummary] writing to %s' % output_dir) + info(__('[autosummary] writing to %s') % output_dir) if base_path is not None: sources = [os.path.join(base_path, filename) for filename in sources] @@ -360,8 +364,8 @@ def get_parser(): # type: () -> argparse.ArgumentParser parser = argparse.ArgumentParser( usage='%(prog)s [OPTIONS] <SOURCE_FILE>...', - epilog='For more information, visit <http://sphinx-doc.org/>.', - description=""" + epilog=__('For more information, visit <http://sphinx-doc.org/>.'), + description=__(""" Generate ReStructuredText using autosummary directives. sphinx-autogen is a frontend to sphinx.ext.autosummary.generate. It generates @@ -372,35 +376,38 @@ The format of the autosummary directive is documented in the ``sphinx.ext.autosummary`` Python module and can be read using:: pydoc sphinx.ext.autosummary -""") +""")) parser.add_argument('--version', action='version', dest='show_version', version='%%(prog)s %s' % __display_version__) parser.add_argument('source_file', nargs='+', - help='source files to generate rST files for') + help=__('source files to generate rST files for')) parser.add_argument('-o', '--output-dir', action='store', dest='output_dir', - help='directory to place all output in') + help=__('directory to place all output in')) parser.add_argument('-s', '--suffix', action='store', dest='suffix', default='rst', - help='default suffix for files (default: ' - '%(default)s)') + help=__('default suffix for files (default: ' + '%(default)s)')) parser.add_argument('-t', '--templates', action='store', dest='templates', default=None, - help='custom template directory (default: ' - '%(default)s)') + help=__('custom template directory (default: ' + '%(default)s)')) parser.add_argument('-i', '--imported-members', action='store_true', dest='imported_members', default=False, - help='document imported members (default: ' - '%(default)s)') + help=__('document imported members (default: ' + '%(default)s)')) return parser def main(argv=sys.argv[1:]): # type: (List[str]) -> None + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') + app = DummyApplication() setup_documenters(app) args = get_parser().parse_args(argv) diff --git a/sphinx/ext/coverage.py b/sphinx/ext/coverage.py index 6c9acfc7d..351108faa 100644 --- a/sphinx/ext/coverage.py +++ b/sphinx/ext/coverage.py @@ -20,6 +20,7 @@ from six.moves import cPickle as pickle import sphinx from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.inspect import safe_getattr @@ -45,7 +46,7 @@ def compile_regex_list(name, exps): try: lst.append(re.compile(exp)) except Exception: - logger.warning('invalid regex %r in %s', exp, name) + logger.warning(__('invalid regex %r in %s'), exp, name) return lst @@ -54,8 +55,8 @@ class CoverageBuilder(Builder): Evaluates coverage of code in the documentation. """ name = 'coverage' - epilog = ('Testing of coverage in the sources finished, look at the ' - 'results in %(outdir)s/python.txt.') + epilog = __('Testing of coverage in the sources finished, look at the ' + 'results in %(outdir)s/python.txt.') def init(self): # type: () -> None @@ -69,7 +70,7 @@ class CoverageBuilder(Builder): try: self.c_regexes.append((name, re.compile(exp))) except Exception: - logger.warning('invalid regex %r in coverage_c_regexes', exp) + logger.warning(__('invalid regex %r in coverage_c_regexes'), exp) self.c_ignorexps = {} # type: Dict[unicode, List[Pattern]] for (name, exps) in iteritems(self.config.coverage_ignore_c_items): @@ -151,7 +152,7 @@ class CoverageBuilder(Builder): try: mod = __import__(mod_name, fromlist=['foo']) except ImportError as err: - logger.warning('module %s could not be imported: %s', mod_name, err) + logger.warning(__('module %s could not be imported: %s'), mod_name, err) self.py_undoc[mod_name] = {'error': err} continue diff --git a/sphinx/ext/doctest.py b/sphinx/ext/doctest.py index 6ed20febc..68877371c 100644 --- a/sphinx/ext/doctest.py +++ b/sphinx/ext/doctest.py @@ -19,16 +19,17 @@ import time from os import path from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from packaging.specifiers import SpecifierSet, InvalidSpecifier from packaging.version import Version from six import itervalues, StringIO, binary_type, text_type, PY2 import sphinx from sphinx.builders import Builder -from sphinx.locale import _ +from sphinx.locale import __ from sphinx.util import force_decode, logging from sphinx.util.console import bold # type: ignore +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import set_source_info from sphinx.util.osutil import fs_encoding, relpath @@ -77,7 +78,7 @@ def is_allowed_version(spec, version): # set up the necessary directives -class TestDirective(Directive): +class TestDirective(SphinxDirective): """ Base class for doctest-related directives. """ @@ -129,12 +130,12 @@ class TestDirective(Directive): prefix, option_name = option[0], option[1:] if prefix not in '+-': self.state.document.reporter.warning( - _("missing '+' or '-' in '%s' option.") % option, + __("missing '+' or '-' in '%s' option.") % option, line=self.lineno) continue if option_name not in doctest.OPTIONFLAGS_BY_NAME: self.state.document.reporter.warning( - _("'%s' is not a valid option.") % option_name, + __("'%s' is not a valid option.") % option_name, line=self.lineno) continue flag = doctest.OPTIONFLAGS_BY_NAME[option[1:]] @@ -148,7 +149,7 @@ class TestDirective(Directive): node['options'][flag] = True # Skip the test except InvalidSpecifier: self.state.document.reporter.warning( - _("'%s' is not a valid pyversion option") % spec, + __("'%s' is not a valid pyversion option") % spec, line=self.lineno) return [node] @@ -214,7 +215,7 @@ class TestGroup(object): if self.tests and len(self.tests[-1]) == 2: self.tests[-1][1] = code else: - raise RuntimeError('invalid TestCode type') + raise RuntimeError(__('invalid TestCode type')) def __repr__(self): # type: ignore # type: () -> unicode @@ -275,8 +276,8 @@ class DocTestBuilder(Builder): Runs test snippets in the documentation. """ name = 'doctest' - epilog = ('Testing of doctests in the sources finished, look at the ' - 'results in %(outdir)s/output.txt.') + epilog = __('Testing of doctests in the sources finished, look at the ' + 'results in %(outdir)s/output.txt.') def init(self): # type: () -> None @@ -427,7 +428,7 @@ Doctest summary filename = self.get_filename_for_node(node, docname) line_number = self.get_line_number(node) if not source: - logger.warning('no code/output in %s block at %s:%s', + logger.warning(__('no code/output in %s block at %s:%s'), node.get('testnodetype', 'doctest'), filename, line_number) code = TestCode(source, type=node.get('testnodetype', 'doctest'), @@ -518,7 +519,7 @@ Doctest summary doctest_encode(code[0].code, self.env.config.source_encoding), {}, # type: ignore # NOQA group.name, code[0].filename, code[0].lineno) except Exception: - logger.warning('ignoring invalid doctest code: %r', code[0].code, + logger.warning(__('ignoring invalid doctest code: %r'), code[0].code, location=(code[0].filename, code[0].lineno)) continue if not test.examples: diff --git a/sphinx/ext/extlinks.py b/sphinx/ext/extlinks.py index dbd55e4b4..29bfe928f 100644 --- a/sphinx/ext/extlinks.py +++ b/sphinx/ext/extlinks.py @@ -8,11 +8,11 @@ This adds a new config value called ``extlinks`` that is created like this:: - extlinks = {'exmpl': ('http://example.com/%s.html', prefix), ...} + extlinks = {'exmpl': ('https://example.invalid/%s.html', prefix), ...} Now you can use e.g. :exmpl:`foo` in your documents. This will create a - link to ``http://example.com/foo.html``. The link caption depends on the - *prefix* value given: + link to ``https://example.invalid/foo.html``. The link caption depends on + the *prefix* value given: - If it is ``None``, the caption will be the full URL. - If it is a string (empty or not), the caption will be the prefix prepended @@ -30,9 +30,18 @@ from six import iteritems import sphinx from sphinx.util.nodes import split_explicit_title +if False: + # For type annotation + from typing import Any, Dict, List, Tuple # NOQA + from docutils.parsers.rst.states import Inliner # NOQA + from sphinx.application import Sphinx # NOQA + from sphinx.util.typing import RoleFunction # NOQA + def make_link_role(base_url, prefix): + # type: (unicode, unicode) -> RoleFunction def role(typ, rawtext, text, lineno, inliner, options={}, content=[]): + # type: (unicode, unicode, unicode, int, Inliner, Dict, List[unicode]) -> Tuple[List[nodes.Node], List[nodes.Node]] # NOQA text = utils.unescape(text) has_explicit_title, title, part = split_explicit_title(text) try: @@ -54,11 +63,13 @@ def make_link_role(base_url, prefix): def setup_link_roles(app): + # type: (Sphinx) -> None for name, (base_url, prefix) in iteritems(app.config.extlinks): app.add_role(name, make_link_role(base_url, prefix)) def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] app.add_config_value('extlinks', {}, 'env') app.connect('builder-inited', setup_link_roles) return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/githubpages.py b/sphinx/ext/githubpages.py index b267b1740..bd7061fb7 100644 --- a/sphinx/ext/githubpages.py +++ b/sphinx/ext/githubpages.py @@ -13,13 +13,21 @@ import os import sphinx +if False: + # For type annotation + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + from sphinx.environment import BuildEnvironment # NOQA + def create_nojekyll(app, env): + # type: (Sphinx, BuildEnvironment) -> None if app.builder.format == 'html': path = os.path.join(app.builder.outdir, '.nojekyll') open(path, 'wt').close() def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] app.connect('env-updated', create_nojekyll) return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/graphviz.py b/sphinx/ext/graphviz.py index 08707b733..11d9e54a0 100644 --- a/sphinx/ext/graphviz.py +++ b/sphinx/ext/graphviz.py @@ -18,7 +18,7 @@ from os import path from subprocess import Popen, PIPE from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from docutils.statemachine import ViewList from six import text_type @@ -26,11 +26,13 @@ import sphinx from sphinx.errors import SphinxError from sphinx.locale import _, __ from sphinx.util import logging +from sphinx.util.docutils import SphinxDirective from sphinx.util.i18n import search_image_for_language from sphinx.util.osutil import ensuredir, ENOENT, EPIPE, EINVAL if False: # For type annotation + from docutils.parsers.rst import Directive # NOQA from typing import Any, Dict, List, Tuple # NOQA from sphinx.application import Sphinx # NOQA @@ -111,7 +113,7 @@ def align_spec(argument): return directives.choice(argument, ('left', 'center', 'right')) -class Graphviz(Directive): +class Graphviz(SphinxDirective): """ Directive to insert arbitrary dot markup. """ @@ -135,12 +137,11 @@ class Graphviz(Directive): return [document.reporter.warning( __('Graphviz directive cannot have both content and ' 'a filename argument'), line=self.lineno)] - env = self.state.document.settings.env - argument = search_image_for_language(self.arguments[0], env) - rel_filename, filename = env.relfn2path(argument) - env.note_dependency(rel_filename) + argument = search_image_for_language(self.arguments[0], self.env) + rel_filename, filename = self.env.relfn2path(argument) + self.env.note_dependency(rel_filename) try: - with codecs.open(filename, 'r', 'utf-8') as fp: + with codecs.open(filename, 'r', 'utf-8') as fp: # type: ignore dotcode = fp.read() except (IOError, OSError): return [document.reporter.warning( @@ -170,7 +171,7 @@ class Graphviz(Directive): return [node] -class GraphvizSimple(Directive): +class GraphvizSimple(SphinxDirective): """ Directive to insert arbitrary dot markup. """ @@ -277,7 +278,7 @@ def render_dot_html(self, node, code, options, prefix='graphviz', "'svg', but is %r") % format) fname, outfn = render_dot(self, code, options, format, prefix) except GraphvizError as exc: - logger.warning('dot code %r: ' % code + str(exc)) + logger.warning(__('dot code %r: %s'), code, text_type(exc)) raise nodes.SkipNode if fname is None: @@ -290,21 +291,27 @@ def render_dot_html(self, node, code, options, prefix='graphviz', self.body.append('<div align="%s" class="align-%s">' % (node['align'], node['align'])) if format == 'svg': - svgtag = '''<object data="%s" type="image/svg+xml"> - <p class="warning">%s</p></object>\n''' % (fname, alt) - self.body.append(svgtag) + self.body.append('<div class="graphviz">') + self.body.append('<object data="%s" type="image/svg+xml" %s>\n' % + (fname, imgcss)) + self.body.append('<p class="warning">%s</p>' % alt) + self.body.append('</object></div>\n') else: with codecs.open(outfn + '.map', 'r', encoding='utf-8') as mapfile: # type: ignore imgmap = ClickableMapDefinition(outfn + '.map', mapfile.read(), dot=code) if imgmap.clickable: # has a map - self.body.append('<img src="%s" alt="%s" usemap="#%s" %s/>\n' % + self.body.append('<div class="graphviz">') + self.body.append('<img src="%s" alt="%s" usemap="#%s" %s/>' % (fname, alt, imgmap.id, imgcss)) + self.body.append('</div>\n') self.body.append(imgmap.generate_clickable_map()) else: # nothing in image map - self.body.append('<img src="%s" alt="%s" %s/>\n' % + self.body.append('<div class="graphviz">') + self.body.append('<img src="%s" alt="%s" %s/>' % (fname, alt, imgcss)) + self.body.append('</div>\n') if 'align' in node: self.body.append('</div>\n') @@ -321,7 +328,7 @@ def render_dot_latex(self, node, code, options, prefix='graphviz'): try: fname, outfn = render_dot(self, code, options, 'pdf', prefix) except GraphvizError as exc: - logger.warning('dot code %r: ' % code + str(exc)) + logger.warning(__('dot code %r: %s'), code, text_type(exc)) raise nodes.SkipNode is_inline = self.is_inline(node) @@ -359,7 +366,7 @@ def render_dot_texinfo(self, node, code, options, prefix='graphviz'): try: fname, outfn = render_dot(self, code, options, 'png', prefix) except GraphvizError as exc: - logger.warning('dot code %r: ' % code + str(exc)) + logger.warning(__('dot code %r: %s'), code, text_type(exc)) raise nodes.SkipNode if fname is not None: self.body.append('@image{%s,,,[graphviz],png}\n' % fname[:-4]) diff --git a/sphinx/ext/ifconfig.py b/sphinx/ext/ifconfig.py index 16042ac3f..f22a37e92 100644 --- a/sphinx/ext/ifconfig.py +++ b/sphinx/ext/ifconfig.py @@ -21,9 +21,9 @@ """ from docutils import nodes -from docutils.parsers.rst import Directive import sphinx +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import set_source_info if False: @@ -36,7 +36,7 @@ class ifconfig(nodes.Element): pass -class IfConfig(Directive): +class IfConfig(SphinxDirective): has_content = True required_arguments = 1 @@ -57,7 +57,7 @@ class IfConfig(Directive): def process_ifconfig_nodes(app, doctree, docname): # type: (Sphinx, nodes.Node, unicode) -> None - ns = dict((confval.name, confval.value) for confval in app.config) # type: ignore + ns = dict((confval.name, confval.value) for confval in app.config) ns.update(app.config.__dict__.copy()) ns['builder'] = app.builder.name for node in doctree.traverse(ifconfig): diff --git a/sphinx/ext/imgconverter.py b/sphinx/ext/imgconverter.py index ae9f6e607..fe086b1fe 100644 --- a/sphinx/ext/imgconverter.py +++ b/sphinx/ext/imgconverter.py @@ -29,6 +29,7 @@ logger = logging.getLogger(__name__) class ImagemagickConverter(ImageConverter): conversion_rules = [ ('image/svg+xml', 'image/png'), + ('image/gif', 'image/png'), ('application/pdf', 'image/png'), ] @@ -65,6 +66,10 @@ class ImagemagickConverter(ImageConverter): # type: (unicode, unicode) -> bool """Converts the image to expected one.""" try: + if _from.lower().endswith('.gif'): + # when target is GIF format, pick the first frame + _from += '[0]' + args = ([self.config.image_converter] + self.config.image_converter_args + [_from, _to]) diff --git a/sphinx/ext/imgmath.py b/sphinx/ext/imgmath.py index 5f9d7a1e2..a1faf2c1b 100644 --- a/sphinx/ext/imgmath.py +++ b/sphinx/ext/imgmath.py @@ -25,7 +25,7 @@ import sphinx from sphinx.errors import SphinxError, ExtensionError from sphinx.ext.mathbase import get_node_equation_number from sphinx.ext.mathbase import setup_math as mathbase_setup, wrap_displaymath -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.osutil import ensuredir, ENOENT, cd from sphinx.util.png import read_png_depth, write_png_depth @@ -141,8 +141,8 @@ def compile_math(latex, builder): except OSError as err: if err.errno != ENOENT: # No such file or directory raise - logger.warning('LaTeX command %r cannot be run (needed for math ' - 'display), check the imgmath_latex setting', + logger.warning(__('LaTeX command %r cannot be run (needed for math ' + 'display), check the imgmath_latex setting'), builder.config.imgmath_latex) raise InvokeError @@ -161,8 +161,8 @@ def convert_dvi_to_image(command, name): except OSError as err: if err.errno != ENOENT: # No such file or directory raise - logger.warning('%s command %r cannot be run (needed for math ' - 'display), check the imgmath_%s setting', + logger.warning(__('%s command %r cannot be run (needed for math ' + 'display), check the imgmath_%s setting'), name, command[0], name) raise InvokeError @@ -300,7 +300,7 @@ def html_visit_math(self, node): sm = nodes.system_message(msg, type='WARNING', level=2, backrefs=[], source=node['latex']) sm.walkabout(self) - logger.warning('display latex %r: %s', node['latex'], msg) + logger.warning(__('display latex %r: %s'), node['latex'], msg) raise nodes.SkipNode if fname is None: # something failed -- use text-only as a bad substitute @@ -328,7 +328,7 @@ def html_visit_displaymath(self, node): sm = nodes.system_message(msg, type='WARNING', level=2, backrefs=[], source=node['latex']) sm.walkabout(self) - logger.warning('inline latex %r: %s', node['latex'], msg) + logger.warning(__('inline latex %r: %s'), node['latex'], msg) raise nodes.SkipNode self.body.append(self.starttag(node, 'div', CLASS='math')) self.body.append('<p>') diff --git a/sphinx/ext/inheritance_diagram.py b/sphinx/ext/inheritance_diagram.py index 14593ac99..d91848aa1 100644 --- a/sphinx/ext/inheritance_diagram.py +++ b/sphinx/ext/inheritance_diagram.py @@ -42,7 +42,7 @@ import sys from hashlib import md5 from docutils import nodes -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from six import text_type from six.moves import builtins @@ -51,6 +51,7 @@ from sphinx.ext.graphviz import render_dot_html, render_dot_latex, \ render_dot_texinfo, figure_wrapper from sphinx.pycode import ModuleAnalyzer from sphinx.util import force_decode +from sphinx.util.docutils import SphinxDirective if False: # For type annotation @@ -322,7 +323,7 @@ class inheritance_diagram(nodes.General, nodes.Element): pass -class InheritanceDiagram(Directive): +class InheritanceDiagram(SphinxDirective): """ Run when the inheritance_diagram directive is first encountered. """ @@ -341,9 +342,8 @@ class InheritanceDiagram(Directive): # type: () -> List[nodes.Node] node = inheritance_diagram() node.document = self.state.document - env = self.state.document.settings.env class_names = self.arguments[0].split() - class_role = env.get_domain('py').role('class') + class_role = self.env.get_domain('py').role('class') # Store the original content for use as a hash node['parts'] = self.options.get('parts', 0) node['content'] = ', '.join(class_names) @@ -356,10 +356,10 @@ class InheritanceDiagram(Directive): # Create a graph starting with the list of classes try: graph = InheritanceGraph( - class_names, env.ref_context.get('py:module'), + class_names, self.env.ref_context.get('py:module'), parts=node['parts'], private_bases='private-bases' in self.options, - aliases=env.config.inheritance_alias, + aliases=self.config.inheritance_alias, top_classes=node['top-classes']) except InheritanceException as err: return [node.document.reporter.warning(err.args[0], diff --git a/sphinx/ext/intersphinx.py b/sphinx/ext/intersphinx.py index 372f7baab..af219d16a 100644 --- a/sphinx/ext/intersphinx.py +++ b/sphinx/ext/intersphinx.py @@ -41,7 +41,7 @@ from six.moves.urllib.parse import urlsplit, urlunsplit import sphinx from sphinx.builders.html import INVENTORY_FILENAME from sphinx.deprecation import RemovedInSphinx20Warning -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import requests, logging from sphinx.util.inventory import InventoryFile @@ -64,31 +64,33 @@ class InventoryAdapter(object): """Inventory adapter for environment""" def __init__(self, env): + # type: (BuildEnvironment) -> None self.env = env if not hasattr(env, 'intersphinx_cache'): - self.env.intersphinx_cache = {} - self.env.intersphinx_inventory = {} - self.env.intersphinx_named_inventory = {} + self.env.intersphinx_cache = {} # type: ignore + self.env.intersphinx_inventory = {} # type: ignore + self.env.intersphinx_named_inventory = {} # type: ignore @property def cache(self): # type: () -> Dict[unicode, Tuple[unicode, int, Inventory]] - return self.env.intersphinx_cache + return self.env.intersphinx_cache # type: ignore @property def main_inventory(self): # type: () -> Inventory - return self.env.intersphinx_inventory + return self.env.intersphinx_inventory # type: ignore @property def named_inventory(self): # type: () -> Dict[unicode, Inventory] - return self.env.intersphinx_named_inventory + return self.env.intersphinx_named_inventory # type: ignore def clear(self): - self.env.intersphinx_inventory.clear() - self.env.intersphinx_named_inventory.clear() + # type: () -> None + self.env.intersphinx_inventory.clear() # type: ignore + self.env.intersphinx_named_inventory.clear() # type: ignore def _strip_basic_auth(url): @@ -221,7 +223,7 @@ def load_mappings(app): # new format name, (uri, inv) = key, value if not isinstance(name, string_types): - logger.warning('intersphinx identifier %r is not string. Ignored', name) + logger.warning(__('intersphinx identifier %r is not string. Ignored'), name) continue else: # old format, no name @@ -263,8 +265,8 @@ def load_mappings(app): for fail in failures: logger.info(*fail) else: - logger.warning("failed to reach any of the inventories " - "with the following issues:") + logger.warning(__("failed to reach any of the inventories " + "with the following issues:")) for fail in failures: logger.warning(*fail) @@ -371,7 +373,11 @@ def setup(app): app.add_config_value('intersphinx_timeout', None, False) app.connect('missing-reference', missing_reference) app.connect('builder-inited', load_mappings) - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} + return { + 'version': sphinx.__display_version__, + 'env_version': 1, + 'parallel_read_safe': True + } def debug(argv): @@ -401,6 +407,7 @@ def inspect_main(argv): config = MockConfig() def warn(self, msg): + # type: (unicode) -> None print(msg, file=sys.stderr) filename = argv[0] diff --git a/sphinx/ext/jsmath.py b/sphinx/ext/jsmath.py index 5d8ec876e..97ea400a3 100644 --- a/sphinx/ext/jsmath.py +++ b/sphinx/ext/jsmath.py @@ -13,19 +13,26 @@ from docutils import nodes import sphinx -from sphinx.application import ExtensionError +from sphinx.errors import ExtensionError from sphinx.ext.mathbase import get_node_equation_number from sphinx.ext.mathbase import setup_math as mathbase_setup from sphinx.locale import _ +if False: + # For type annotation + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + def html_visit_math(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None self.body.append(self.starttag(node, 'span', '', CLASS='math notranslate nohighlight')) self.body.append(self.encode(node['latex']) + '</span>') raise nodes.SkipNode def html_visit_displaymath(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None if node['nowrap']: self.body.append(self.starttag(node, 'div', CLASS='math notranslate nohighlight')) self.body.append(self.encode(node['latex'])) @@ -53,6 +60,7 @@ def html_visit_displaymath(self, node): def builder_inited(app): + # type: (Sphinx) -> None if not app.config.jsmath_path: raise ExtensionError('jsmath_path config value must be set for the ' 'jsmath extension to work') @@ -60,6 +68,7 @@ def builder_inited(app): def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] try: mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None)) except ExtensionError: diff --git a/sphinx/ext/mathbase.py b/sphinx/ext/mathbase.py index 3cc734537..e6a6929e6 100644 --- a/sphinx/ext/mathbase.py +++ b/sphinx/ext/mathbase.py @@ -11,19 +11,21 @@ from docutils import nodes, utils from docutils.nodes import make_id -from docutils.parsers.rst import Directive, directives +from docutils.parsers.rst import directives from sphinx.config import string_classes from sphinx.domains import Domain from sphinx.locale import __ from sphinx.roles import XRefRole from sphinx.util import logging +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_refnode, set_source_info if False: # For type annotation from typing import Any, Callable, Dict, Iterable, List, Tuple # NOQA from docutils.parsers.rst.states import Inliner # NOQA + from docutils.writers.html4css1 import Writer # NOQA from sphinx.application import Sphinx # NOQA from sphinx.builders import Builder # NOQA from sphinx.environment import BuildEnvironment # NOQA @@ -61,6 +63,9 @@ class MathDomain(Domain): dangling_warnings = { 'eq': 'equation not found: %(target)s', } + enumerable_nodes = { # node_class -> (figtype, title_getter) + displaymath: ('displaymath', None), + } # type: Dict[nodes.Node, Tuple[unicode, Callable]] def clear_doc(self, docname): # type: (unicode) -> None @@ -97,7 +102,7 @@ class MathDomain(Domain): eqref_format = env.config.math_eqref_format or "({number})" title = nodes.Text(eqref_format.format(number=number)) except KeyError as exc: - logger.warning('Invalid math_eqref_format: %r', exc, + logger.warning(__('Invalid math_eqref_format: %r'), exc, location=node) title = nodes.Text("(%d)" % number) title = nodes.Text("(%d)" % number) @@ -136,6 +141,7 @@ class MathDomain(Domain): def get_node_equation_number(writer, node): + # type: (Writer, nodes.Node) -> unicode if writer.builder.config.math_numfig and writer.builder.config.numfig: figtype = 'displaymath' if writer.builder.name == 'singlehtml': @@ -207,7 +213,7 @@ def is_in_section_title(node): return False -class MathDirective(Directive): +class MathDirective(SphinxDirective): has_content = True required_arguments = 0 @@ -233,7 +239,7 @@ class MathDirective(Directive): if 'label' in self.options: node['label'] = self.options['label'] node['nowrap'] = 'nowrap' in self.options - node['docname'] = self.state.document.settings.env.docname + node['docname'] = self.env.docname ret = [node] set_source_info(self, node) if hasattr(self, 'src'): @@ -244,21 +250,20 @@ class MathDirective(Directive): def add_target(self, ret): # type: (List[nodes.Node]) -> None node = ret[0] - env = self.state.document.settings.env # assign label automatically if math_number_all enabled - if node['label'] == '' or (env.config.math_number_all and not node['label']): - seq = env.new_serialno('sphinx.ext.math#equations') - node['label'] = "%s:%d" % (env.docname, seq) + if node['label'] == '' or (self.config.math_number_all and not node['label']): + seq = self.env.new_serialno('sphinx.ext.math#equations') + node['label'] = "%s:%d" % (self.env.docname, seq) # no targets and numbers are needed if not node['label']: return # register label to domain - domain = env.get_domain('math') + domain = self.env.get_domain('math') try: - eqno = domain.add_equation(env, env.docname, node['label']) + eqno = domain.add_equation(self.env, self.env.docname, node['label']) # type: ignore # NOQA node['number'] = eqno # add target node @@ -307,7 +312,7 @@ def latex_visit_eqref(self, node): ref = '\\ref{%s}' % label self.body.append(eqref_format.format(number=ref)) except KeyError as exc: - logger.warning('Invalid math_eqref_format: %r', exc, + logger.warning(__('Invalid math_eqref_format: %r'), exc, location=node) self.body.append('\\eqref{%s}' % label) else: @@ -376,12 +381,12 @@ def setup_math(app, htmlinlinevisitors, htmldisplayvisitors): man=(man_visit_math, None), texinfo=(texinfo_visit_math, None), html=htmlinlinevisitors) - app.add_enumerable_node(displaymath, 'displaymath', - latex=(latex_visit_displaymath, None), - text=(text_visit_displaymath, None), - man=(man_visit_displaymath, man_depart_displaymath), - texinfo=(texinfo_visit_displaymath, texinfo_depart_displaymath), - html=htmldisplayvisitors) + app.add_node(displaymath, + latex=(latex_visit_displaymath, None), + text=(text_visit_displaymath, None), + man=(man_visit_displaymath, man_depart_displaymath), + texinfo=(texinfo_visit_displaymath, texinfo_depart_displaymath), + html=htmldisplayvisitors) app.add_node(eqref, latex=(latex_visit_eqref, None)) app.add_role('math', math_role) app.add_role('eq', EqXRefRole(warn_dangling=True)) diff --git a/sphinx/ext/mathjax.py b/sphinx/ext/mathjax.py index bb9964fa8..2bb7eec09 100644 --- a/sphinx/ext/mathjax.py +++ b/sphinx/ext/mathjax.py @@ -3,7 +3,7 @@ sphinx.ext.mathjax ~~~~~~~~~~~~~~~~~~ - Allow `MathJax <http://mathjax.org/>`_ to be used to display math in + Allow `MathJax <https://www.mathjax.org/>`_ to be used to display math in Sphinx's HTML writer -- requires the MathJax JavaScript library on your webserver/computer. @@ -19,8 +19,14 @@ from sphinx.ext.mathbase import get_node_equation_number from sphinx.ext.mathbase import setup_math as mathbase_setup from sphinx.locale import _ +if False: + # For type annotation + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + def html_visit_math(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None self.body.append(self.starttag(node, 'span', '', CLASS='math notranslate nohighlight')) self.body.append(self.builder.config.mathjax_inline[0] + self.encode(node['latex']) + @@ -29,6 +35,7 @@ def html_visit_math(self, node): def html_visit_displaymath(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None self.body.append(self.starttag(node, 'div', CLASS='math notranslate nohighlight')) if node['nowrap']: self.body.append(self.encode(node['latex'])) @@ -61,6 +68,7 @@ def html_visit_displaymath(self, node): def builder_inited(app): + # type: (Sphinx) -> None if not app.config.mathjax_path: raise ExtensionError('mathjax_path config value must be set for the ' 'mathjax extension to work') @@ -68,13 +76,14 @@ def builder_inited(app): def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] try: mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None)) except ExtensionError: raise ExtensionError('sphinx.ext.mathjax: other math package is already loaded') # more information for mathjax secure url is here: - # http://docs.mathjax.org/en/latest/start.html#secure-access-to-the-cdn + # https://docs.mathjax.org/en/latest/start.html#secure-access-to-the-cdn app.add_config_value('mathjax_path', 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?' 'config=TeX-AMS-MML_HTMLorMML', False) diff --git a/sphinx/ext/napoleon/__init__.py b/sphinx/ext/napoleon/__init__.py index b65f7f2a1..b968f5948 100644 --- a/sphinx/ext/napoleon/__init__.py +++ b/sphinx/ext/napoleon/__init__.py @@ -47,9 +47,10 @@ class Config(object): napoleon_use_param = True napoleon_use_rtype = True napoleon_use_keyword = True + napoleon_custom_sections = None .. _Google style: - http://google.github.io/styleguide/pyguide.html + https://google.github.io/styleguide/pyguide.html .. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt @@ -241,6 +242,19 @@ class Config(object): :returns: *bool* -- True if successful, False otherwise + napoleon_custom_sections : :obj:`list` (Defaults to None) + Add a list of custom sections to include, expanding the list of parsed sections. + + The entries can either be strings or tuples, depending on the intention: + * To create a custom "generic" section, just pass a string. + * To create an alias for an existing section, pass a tuple containing the + alias name and the original, in that order. + + If an entry is just a string, it is interpreted as a header for a generic + section. If the entry is a tuple/list/indexed container, the first entry + is the name of the section, the second is the section key to emulate. + + """ _config_values = { 'napoleon_google_docstring': (True, 'env'), @@ -254,7 +268,8 @@ class Config(object): 'napoleon_use_ivar': (False, 'env'), 'napoleon_use_param': (True, 'env'), 'napoleon_use_rtype': (True, 'env'), - 'napoleon_use_keyword': (True, 'env') + 'napoleon_use_keyword': (True, 'env'), + 'napoleon_custom_sections': (None, 'env') } def __init__(self, **settings): @@ -310,14 +325,13 @@ def _patch_python_domain(): pass else: import sphinx.domains.python - import sphinx.locale - l_ = sphinx.locale.lazy_gettext + from sphinx.locale import _ for doc_field in sphinx.domains.python.PyObject.doc_field_types: if doc_field.name == 'parameter': doc_field.names = ('param', 'parameter', 'arg', 'argument') break sphinx.domains.python.PyObject.doc_field_types.append( - PyTypedField('keyword', label=l_('Keyword Arguments'), + PyTypedField('keyword', label=_('Keyword Arguments'), names=('keyword', 'kwarg', 'kwparam'), typerolename='obj', typenames=('paramtype', 'kwtype'), can_collapse=True)) diff --git a/sphinx/ext/napoleon/docstring.py b/sphinx/ext/napoleon/docstring.py index b349c761f..530cec203 100644 --- a/sphinx/ext/napoleon/docstring.py +++ b/sphinx/ext/napoleon/docstring.py @@ -14,6 +14,7 @@ import collections import inspect import re +from functools import partial from six import string_types, u from six.moves import range @@ -140,13 +141,19 @@ class GoogleDocstring(UnicodeMixin): self._sections = { 'args': self._parse_parameters_section, 'arguments': self._parse_parameters_section, + 'attention': partial(self._parse_admonition, 'attention'), 'attributes': self._parse_attributes_section, + 'caution': partial(self._parse_admonition, 'caution'), + 'danger': partial(self._parse_admonition, 'danger'), + 'error': partial(self._parse_admonition, 'error'), 'example': self._parse_examples_section, 'examples': self._parse_examples_section, + 'hint': partial(self._parse_admonition, 'hint'), + 'important': partial(self._parse_admonition, 'important'), 'keyword args': self._parse_keyword_arguments_section, 'keyword arguments': self._parse_keyword_arguments_section, 'methods': self._parse_methods_section, - 'note': self._parse_note_section, + 'note': partial(self._parse_admonition, 'note'), 'notes': self._parse_notes_section, 'other parameters': self._parse_other_parameters_section, 'parameters': self._parse_parameters_section, @@ -155,13 +162,17 @@ class GoogleDocstring(UnicodeMixin): 'raises': self._parse_raises_section, 'references': self._parse_references_section, 'see also': self._parse_see_also_section, - 'todo': self._parse_todo_section, - 'warning': self._parse_warning_section, - 'warnings': self._parse_warning_section, + 'tip': partial(self._parse_admonition, 'tip'), + 'todo': partial(self._parse_admonition, 'todo'), + 'warning': partial(self._parse_admonition, 'warning'), + 'warnings': partial(self._parse_admonition, 'warning'), 'warns': self._parse_warns_section, 'yield': self._parse_yields_section, 'yields': self._parse_yields_section, } # type: Dict[unicode, Callable] + + self._load_custom_sections() + self._parse() def __unicode__(self): @@ -522,6 +533,23 @@ class GoogleDocstring(UnicodeMixin): line and not self._is_indented(line, self._section_indent))) + def _load_custom_sections(self): + # type: () -> None + + if self._config.napoleon_custom_sections is not None: + for entry in self._config.napoleon_custom_sections: + if isinstance(entry, string_types): + # if entry is just a label, add to sections list, + # using generic section logic. + self._sections[entry.lower()] = self._parse_custom_generic_section + else: + # otherwise, assume entry is container; + # [0] is new section, [1] is the section to alias. + # in the case of key mismatch, just handle as generic section. + self._sections[entry[0].lower()] = \ + self._sections.get(entry[1].lower(), + self._parse_custom_generic_section) + def _parse(self): # type: () -> None self._parsed_lines = self._consume_empty() @@ -550,10 +578,18 @@ class GoogleDocstring(UnicodeMixin): lines = self._consume_to_next_section() self._parsed_lines.extend(lines) + def _parse_admonition(self, admonition, section): + # type (unicode, unicode) -> List[unicode] + lines = self._consume_to_next_section() + return self._format_admonition(admonition, lines) + def _parse_attribute_docstring(self): # type: () -> List[unicode] _type, _desc = self._consume_inline_attribute() - return self._format_field('', _type, _desc) + lines = self._format_field('', '', _desc) + if _type: + lines.extend(['', ':type: %s' % _type]) + return lines def _parse_attributes_section(self, section): # type: (unicode) -> List[unicode] @@ -566,8 +602,11 @@ class GoogleDocstring(UnicodeMixin): lines.append(':vartype %s: %s' % (_name, _type)) else: lines.extend(['.. attribute:: ' + _name, '']) - fields = self._format_field('', _type, _desc) + fields = self._format_field('', '', _desc) lines.extend(self._indent(fields, 3)) + if _type: + lines.append('') + lines.extend(self._indent([':type: %s' % _type], 3)) lines.append('') if self._config.napoleon_use_ivar: lines.append('') @@ -578,6 +617,10 @@ class GoogleDocstring(UnicodeMixin): use_admonition = self._config.napoleon_use_admonition_for_examples return self._parse_generic_section(section, use_admonition) + def _parse_custom_generic_section(self, section): + # for now, no admonition for simple custom sections + return self._parse_generic_section(section, False) + def _parse_usage_section(self, section): # type: (unicode) -> List[unicode] header = ['.. rubric:: Usage:', ''] # type: List[unicode] @@ -621,11 +664,6 @@ class GoogleDocstring(UnicodeMixin): lines.append('') return lines - def _parse_note_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('note', lines) - def _parse_notes_section(self, section): # type: (unicode) -> List[unicode] use_admonition = self._config.napoleon_use_admonition_for_notes @@ -717,19 +755,8 @@ class GoogleDocstring(UnicodeMixin): return lines def _parse_see_also_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('seealso', lines) - - def _parse_todo_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('todo', lines) - - def _parse_warning_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('warning', lines) + # type (unicode) -> List[unicode] + return self._parse_admonition('seealso', section) def _parse_warns_section(self, section): # type: (unicode) -> List[unicode] @@ -963,8 +990,9 @@ class NumpyDocstring(GoogleDocstring): items = [] def parse_item_name(text): + # type: (unicode) -> Tuple[unicode, unicode] """Match ':role:`name`' or 'name'""" - m = self._name_rgx.match(text) + m = self._name_rgx.match(text) # type: ignore if m: g = m.groups() if g[1] is None: @@ -974,6 +1002,7 @@ class NumpyDocstring(GoogleDocstring): raise ValueError("%s is not a item name" % text) def push_item(name, rest): + # type: (unicode, List[unicode]) -> None if not name: return name, role = parse_item_name(name) diff --git a/sphinx/ext/pngmath.py b/sphinx/ext/pngmath.py deleted file mode 100644 index ebb05e615..000000000 --- a/sphinx/ext/pngmath.py +++ /dev/null @@ -1,276 +0,0 @@ -# -*- coding: utf-8 -*- -""" - sphinx.ext.pngmath - ~~~~~~~~~~~~~~~~~~ - - Render math in HTML via dvipng. This extension has been deprecated; please - use sphinx.ext.imgmath instead. - - :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. - :license: BSD, see LICENSE for details. -""" - -import codecs -import posixpath -import re -import shutil -import tempfile -from hashlib import sha1 -from os import path -from subprocess import Popen, PIPE - -from docutils import nodes -from six import text_type - -import sphinx -from sphinx.errors import SphinxError, ExtensionError -from sphinx.ext.mathbase import get_node_equation_number -from sphinx.ext.mathbase import setup_math as mathbase_setup, wrap_displaymath -from sphinx.util import logging -from sphinx.util.osutil import ensuredir, ENOENT, cd -from sphinx.util.png import read_png_depth, write_png_depth -from sphinx.util.pycompat import sys_encoding - -if False: - # For type annotation - from typing import Any, Dict, Tuple # NOQA - from sphinx.application import Sphinx # NOQA - from sphinx.ext.mathbase import math as math_node, displaymath # NOQA - -logger = logging.getLogger(__name__) - - -class MathExtError(SphinxError): - category = 'Math extension error' - - def __init__(self, msg, stderr=None, stdout=None): - # type: (unicode, unicode, unicode) -> None - if stderr: - msg += '\n[stderr]\n' + stderr.decode(sys_encoding, 'replace') - if stdout: - msg += '\n[stdout]\n' + stdout.decode(sys_encoding, 'replace') - SphinxError.__init__(self, msg) - - -DOC_HEAD = r''' -\documentclass[12pt]{article} -\usepackage[utf8x]{inputenc} -\usepackage{amsmath} -\usepackage{amsthm} -\usepackage{amssymb} -\usepackage{amsfonts} -\usepackage{bm} -\pagestyle{empty} -''' - -DOC_BODY = r''' -\begin{document} -%s -\end{document} -''' - -DOC_BODY_PREVIEW = r''' -\usepackage[active]{preview} -\begin{document} -\begin{preview} -%s -\end{preview} -\end{document} -''' - -depth_re = re.compile(br'\[\d+ depth=(-?\d+)\]') - - -def render_math(self, math): - # type: (nodes.NodeVisitor, unicode) -> Tuple[unicode, int] - """Render the LaTeX math expression *math* using latex and dvipng. - - Return the filename relative to the built document and the "depth", - that is, the distance of image bottom and baseline in pixels, if the - option to use preview_latex is switched on. - - Error handling may seem strange, but follows a pattern: if LaTeX or - dvipng aren't available, only a warning is generated (since that enables - people on machines without these programs to at least build the rest - of the docs successfully). If the programs are there, however, they - may not fail since that indicates a problem in the math source. - """ - use_preview = self.builder.config.pngmath_use_preview - latex = DOC_HEAD + self.builder.config.pngmath_latex_preamble - latex += (use_preview and DOC_BODY_PREVIEW or DOC_BODY) % math - - shasum = "%s.png" % sha1(latex.encode('utf-8')).hexdigest() - relfn = posixpath.join(self.builder.imgpath, 'math', shasum) - outfn = path.join(self.builder.outdir, self.builder.imagedir, 'math', shasum) - if path.isfile(outfn): - depth = read_png_depth(outfn) - return relfn, depth - - # if latex or dvipng has failed once, don't bother to try again - if hasattr(self.builder, '_mathpng_warned_latex') or \ - hasattr(self.builder, '_mathpng_warned_dvipng'): - return None, None - - # use only one tempdir per build -- the use of a directory is cleaner - # than using temporary files, since we can clean up everything at once - # just removing the whole directory (see cleanup_tempdir) - if not hasattr(self.builder, '_mathpng_tempdir'): - tempdir = self.builder._mathpng_tempdir = tempfile.mkdtemp() - else: - tempdir = self.builder._mathpng_tempdir - - with codecs.open(path.join(tempdir, 'math.tex'), 'w', 'utf-8') as tf: - tf.write(latex) - - # build latex command; old versions of latex don't have the - # --output-directory option, so we have to manually chdir to the - # temp dir to run it. - ltx_args = [self.builder.config.pngmath_latex, '--interaction=nonstopmode'] - # add custom args from the config file - ltx_args.extend(self.builder.config.pngmath_latex_args) - ltx_args.append('math.tex') - - with cd(tempdir): - try: - p = Popen(ltx_args, stdout=PIPE, stderr=PIPE) - except OSError as err: - if err.errno != ENOENT: # No such file or directory - raise - logger.warning('LaTeX command %r cannot be run (needed for math ' - 'display), check the pngmath_latex setting', - self.builder.config.pngmath_latex) - self.builder._mathpng_warned_latex = True - return None, None - - stdout, stderr = p.communicate() - if p.returncode != 0: - raise MathExtError('latex exited with error', stderr, stdout) - - ensuredir(path.dirname(outfn)) - # use some standard dvipng arguments - dvipng_args = [self.builder.config.pngmath_dvipng] - dvipng_args += ['-o', outfn, '-T', 'tight', '-z9'] - # add custom ones from config value - dvipng_args.extend(self.builder.config.pngmath_dvipng_args) - if use_preview: - dvipng_args.append('--depth') - # last, the input file name - dvipng_args.append(path.join(tempdir, 'math.dvi')) - try: - p = Popen(dvipng_args, stdout=PIPE, stderr=PIPE) - except OSError as err: - if err.errno != ENOENT: # No such file or directory - raise - logger.warning('dvipng command %r cannot be run (needed for math ' - 'display), check the pngmath_dvipng setting', - self.builder.config.pngmath_dvipng) - self.builder._mathpng_warned_dvipng = True - return None, None - stdout, stderr = p.communicate() - if p.returncode != 0: - raise MathExtError('dvipng exited with error', stderr, stdout) - depth = None - if use_preview: - for line in stdout.splitlines(): - m = depth_re.match(line) - if m: - depth = int(m.group(1)) - write_png_depth(outfn, depth) - break - - return relfn, depth - - -def cleanup_tempdir(app, exc): - # type: (Sphinx, Exception) -> None - if exc: - return - if not hasattr(app.builder, '_mathpng_tempdir'): - return - try: - shutil.rmtree(app.builder._mathpng_tempdir) # type: ignore - except Exception: - pass - - -def get_tooltip(self, node): - # type: (nodes.NodeVisitor, math_node) -> unicode - if self.builder.config.pngmath_add_tooltips: - return ' alt="%s"' % self.encode(node['latex']).strip() - return '' - - -def html_visit_math(self, node): - # type: (nodes.NodeVisitor, math_node) -> None - try: - fname, depth = render_math(self, '$' + node['latex'] + '$') - except MathExtError as exc: - msg = text_type(exc) - sm = nodes.system_message(msg, type='WARNING', level=2, - backrefs=[], source=node['latex']) - sm.walkabout(self) - logger.warning('display latex %r: %s', node['latex'], msg) - raise nodes.SkipNode - if fname is None: - # something failed -- use text-only as a bad substitute - self.body.append('<span class="math">%s</span>' % - self.encode(node['latex']).strip()) - else: - c = ('<img class="math" src="%s"' % fname) + get_tooltip(self, node) - if depth is not None: - c += ' style="vertical-align: %dpx"' % (-depth) - self.body.append(c + '/>') - raise nodes.SkipNode - - -def html_visit_displaymath(self, node): - # type: (nodes.NodeVisitor, displaymath) -> None - if node['nowrap']: - latex = node['latex'] - else: - latex = wrap_displaymath(node['latex'], None, - self.builder.config.math_number_all) - try: - fname, depth = render_math(self, latex) - except MathExtError as exc: - msg = text_type(exc) - sm = nodes.system_message(msg, type='WARNING', level=2, - backrefs=[], source=node['latex']) - sm.walkabout(self) - logger.warning('inline latex %r: %s', node['latex'], msg) - raise nodes.SkipNode - self.body.append(self.starttag(node, 'div', CLASS='math')) - self.body.append('<p>') - if node['number']: - number = get_node_equation_number(self, node) - self.body.append('<span class="eqno">(%s)</span>' % number) - if fname is None: - # something failed -- use text-only as a bad substitute - self.body.append('<span class="math">%s</span></p>\n</div>' % - self.encode(node['latex']).strip()) - else: - self.body.append(('<img src="%s"' % fname) + get_tooltip(self, node) + - '/></p>\n</div>') - raise nodes.SkipNode - - -def setup(app): - # type: (Sphinx) -> Dict[unicode, Any] - logger.warning('sphinx.ext.pngmath has been deprecated. ' - 'Please use sphinx.ext.imgmath instead.') - try: - mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None)) - except ExtensionError: - raise ExtensionError('sphinx.ext.pngmath: other math package is already loaded') - - app.add_config_value('pngmath_dvipng', 'dvipng', 'html') - app.add_config_value('pngmath_latex', 'latex', 'html') - app.add_config_value('pngmath_use_preview', False, 'html') - app.add_config_value('pngmath_dvipng_args', - ['-gamma', '1.5', '-D', '110', '-bg', 'Transparent'], - 'html') - app.add_config_value('pngmath_latex_args', [], 'html') - app.add_config_value('pngmath_latex_preamble', '', 'html') - app.add_config_value('pngmath_add_tooltips', True, 'html') - app.connect('build-finished', cleanup_tempdir) - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/todo.py b/sphinx/ext/todo.py index fd3d5c062..d7413ac61 100644 --- a/sphinx/ext/todo.py +++ b/sphinx/ext/todo.py @@ -13,14 +13,14 @@ """ from docutils import nodes -from docutils.parsers.rst import Directive from docutils.parsers.rst import directives from docutils.parsers.rst.directives.admonitions import BaseAdmonition import sphinx from sphinx.environment import NoUri -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging +from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import set_source_info from sphinx.util.texescape import tex_escape_map @@ -41,7 +41,7 @@ class todolist(nodes.General, nodes.Element): pass -class Todo(BaseAdmonition): +class Todo(BaseAdmonition, SphinxDirective): """ A todo entry, displayed (if configured) in the form of an admonition. """ @@ -67,10 +67,9 @@ class Todo(BaseAdmonition): todo.insert(0, nodes.title(text=_('Todo'))) set_source_info(self, todo) - env = self.state.document.settings.env - targetid = 'index-%s' % env.new_serialno('index') + targetid = 'index-%s' % self.env.new_serialno('index') # Stash the target to be retrieved later in latex_visit_todo_node. - todo['targetref'] = '%s:%s' % (env.docname, targetid) + todo['targetref'] = '%s:%s' % (self.env.docname, targetid) targetnode = nodes.target('', '', ids=[targetid]) return [targetnode, todo] @@ -103,11 +102,11 @@ def process_todos(app, doctree): }) if env.config.todo_emit_warnings: - logger.warning("TODO entry found: %s", node[1].astext(), + logger.warning(__("TODO entry found: %s"), node[1].astext(), location=node) -class TodoList(Directive): +class TodoList(SphinxDirective): """ A list of all todo entries. """ @@ -258,4 +257,8 @@ def setup(app): app.connect('doctree-resolved', process_todo_nodes) app.connect('env-purge-doc', purge_todos) app.connect('env-merge-info', merge_info) - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} + return { + 'version': sphinx.__display_version__, + 'env_version': 1, + 'parallel_read_safe': True + } diff --git a/sphinx/ext/viewcode.py b/sphinx/ext/viewcode.py index 059aa5ef1..6d3c3c1c0 100644 --- a/sphinx/ext/viewcode.py +++ b/sphinx/ext/viewcode.py @@ -10,12 +10,14 @@ """ import traceback +import warnings from docutils import nodes from six import iteritems, text_type import sphinx from sphinx import addnodes +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.locale import _ from sphinx.pycode import ModuleAnalyzer from sphinx.util import get_full_modname, logging, status_iterator @@ -25,6 +27,7 @@ if False: # For type annotation from typing import Any, Dict, Iterable, Iterator, Set, Tuple # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA from sphinx.environment import BuildEnvironment # NOQA logger = logging.getLogger(__name__) @@ -61,20 +64,29 @@ def doctree_read(app, doctree): def has_tag(modname, fullname, docname, refname): entry = env._viewcode_modules.get(modname, None) # type: ignore - try: - analyzer = ModuleAnalyzer.for_module(modname) - except Exception: - env._viewcode_modules[modname] = False # type: ignore - return - if not isinstance(analyzer.code, text_type): - code = analyzer.code.decode(analyzer.encoding) - else: - code = analyzer.code if entry is False: return - elif entry is None or entry[0] != code: + + code_tags = app.emit_firstresult('viewcode-find-source', modname) + if code_tags is None: + try: + analyzer = ModuleAnalyzer.for_module(modname) + except Exception: + env._viewcode_modules[modname] = False # type: ignore + return + + if not isinstance(analyzer.code, text_type): + code = analyzer.code.decode(analyzer.encoding) + else: + code = analyzer.code + analyzer.find_tags() - entry = code, analyzer.tags, {}, refname + tags = analyzer.tags + else: + code, tags = code_tags + + if entry is None or entry[0] != code: + entry = code, tags, {}, refname env._viewcode_modules[modname] = entry # type: ignore _, tags, used, _ = entry if fullname in tags: @@ -91,7 +103,7 @@ def doctree_read(app, doctree): modname = signode.get('module') fullname = signode.get('fullname') refname = modname - if env.config.viewcode_import: + if env.config.viewcode_follow_imported_members: modname = _get_full_modname(app, modname, fullname) if not modname: continue @@ -230,14 +242,28 @@ def collect_pages(app): yield ('_modules/index', context, 'page.html') +def migrate_viewcode_import(app, config): + # type: (Sphinx, Config) -> None + if config.viewcode_import is not None: + warnings.warn('viewcode_import was renamed to viewcode_follow_imported_members. ' + 'Please update your configuration.', + RemovedInSphinx30Warning) + + def setup(app): # type: (Sphinx) -> Dict[unicode, Any] - app.add_config_value('viewcode_import', True, False) + app.add_config_value('viewcode_import', None, False) app.add_config_value('viewcode_enable_epub', False, False) + app.add_config_value('viewcode_follow_imported_members', True, False) app.connect('doctree-read', doctree_read) app.connect('env-merge-info', env_merge_info) app.connect('html-collect-pages', collect_pages) app.connect('missing-reference', missing_reference) # app.add_config_value('viewcode_include_modules', [], 'env') # app.add_config_value('viewcode_exclude_modules', [], 'env') - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} + app.add_event('viewcode-find-source') + return { + 'version': sphinx.__display_version__, + 'env_version': 1, + 'parallel_read_safe': True + } diff --git a/sphinx/extension.py b/sphinx/extension.py index aa1157ec8..732ea327c 100644 --- a/sphinx/extension.py +++ b/sphinx/extension.py @@ -17,14 +17,16 @@ from sphinx.util import logging if False: # For type annotation - from typing import Dict # NOQA + from typing import Any, Dict # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA logger = logging.getLogger(__name__) class Extension(object): def __init__(self, name, module, **kwargs): + # type: (unicode, Any, Any) -> None self.name = name self.module = module self.metadata = kwargs @@ -41,13 +43,13 @@ class Extension(object): self.parallel_write_safe = kwargs.pop('parallel_write_safe', True) -def verify_required_extensions(app, requirements): - # type: (Sphinx, Dict[unicode, unicode]) -> None +def verify_needs_extensions(app, config): + # type: (Sphinx, Config) -> None """Verify the required Sphinx extensions are loaded.""" - if requirements is None: + if config.needs_extensions is None: return - for extname, reqversion in iteritems(requirements): + for extname, reqversion in iteritems(config.needs_extensions): extension = app.extensions.get(extname) if extension is None: logger.warning(__('The %s extension is required by needs_extensions settings, ' @@ -59,3 +61,14 @@ def verify_required_extensions(app, requirements): 'version %s and therefore cannot be built with ' 'the loaded version (%s).') % (extname, reqversion, extension.version)) + + +def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] + app.connect('config-inited', verify_needs_extensions) + + return { + 'version': 'builtin', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/highlighting.py b/sphinx/highlighting.py index ac1d1118e..7d5956570 100644 --- a/sphinx/highlighting.py +++ b/sphinx/highlighting.py @@ -21,6 +21,7 @@ from pygments.util import ClassNotFound from six import text_type from sphinx.ext import doctest +from sphinx.locale import __ from sphinx.pygments_styles import SphinxStyle, NoneStyle from sphinx.util import logging from sphinx.util.pycompat import htmlescape @@ -131,7 +132,7 @@ class PygmentsBridge(object): try: lexer = lexers[lang] = get_lexer_by_name(lang, **(opts or {})) except ClassNotFound: - logger.warning('Pygments lexer name %r is not known', lang, + logger.warning(__('Pygments lexer name %r is not known'), lang, location=location) lexer = lexers['none'] else: @@ -152,8 +153,8 @@ class PygmentsBridge(object): if lang == 'default': pass # automatic highlighting failed. else: - logger.warning('Could not lex literal_block as "%s". ' - 'Highlighting skipped.', lang, + logger.warning(__('Could not lex literal_block as "%s". ' + 'Highlighting skipped.'), lang, type='misc', subtype='highlighting_failure', location=location) hlsource = highlight(source, lexers['none'], formatter) diff --git a/sphinx/io.py b/sphinx/io.py index 8f1da22bd..b9fa3f002 100644 --- a/sphinx/io.py +++ b/sphinx/io.py @@ -16,22 +16,25 @@ from docutils.io import FileInput, NullOutput from docutils.readers import standalone from docutils.statemachine import StringList, string2lines from docutils.writers import UnfilteredWriter -from six import text_type +from six import text_type, iteritems from typing import Any, Union # NOQA +from sphinx.locale import __ from sphinx.transforms import ( ApplySourceWorkaround, ExtraTranslatableNodes, CitationReferences, DefaultSubstitutions, MoveModuleTargets, HandleCodeBlocks, SortIds, AutoNumbering, AutoIndexUpgrader, FilterSystemMessages, - UnreferencedFootnotesDetector, SphinxSmartQuotes, ManpageLink + UnreferencedFootnotesDetector, SphinxSmartQuotes, DoctreeReadEvent, ManpageLink ) from sphinx.transforms import SphinxTransformer from sphinx.transforms.compact_bullet_list import RefOnlyBulletListTransform from sphinx.transforms.i18n import ( PreserveTranslatableMessages, Locale, RemoveTranslatableInline, ) +from sphinx.transforms.references import SphinxDomains, SubstitutionDefinitionsRemover from sphinx.util import logging from sphinx.util.docutils import LoggingReporter +from sphinx.versioning import UIDTransform if False: # For type annotation @@ -92,7 +95,9 @@ class SphinxStandaloneReader(SphinxBaseReader): Locale, CitationReferences, DefaultSubstitutions, MoveModuleTargets, HandleCodeBlocks, AutoNumbering, AutoIndexUpgrader, SortIds, RemoveTranslatableInline, FilterSystemMessages, RefOnlyBulletListTransform, - UnreferencedFootnotesDetector, SphinxSmartQuotes, ManpageLink + UnreferencedFootnotesDetector, SphinxSmartQuotes, ManpageLink, + SphinxDomains, SubstitutionDefinitionsRemover, DoctreeReadEvent, + UIDTransform, ] # type: List[Transform] def __init__(self, app, *args, **kwargs): @@ -115,7 +120,8 @@ class SphinxI18nReader(SphinxBaseReader): DefaultSubstitutions, MoveModuleTargets, HandleCodeBlocks, AutoNumbering, SortIds, RemoveTranslatableInline, FilterSystemMessages, RefOnlyBulletListTransform, - UnreferencedFootnotesDetector, SphinxSmartQuotes, ManpageLink] + UnreferencedFootnotesDetector, SphinxSmartQuotes, ManpageLink, + SubstitutionDefinitionsRemover] def set_lineno_for_reporter(self, lineno): # type: (int) -> None @@ -149,6 +155,7 @@ class SphinxDummyWriter(UnfilteredWriter): def SphinxDummySourceClass(source, *args, **kwargs): + # type: (Any, Any, Any) -> Any """Bypass source object as is to cheat Publisher.""" return source @@ -157,7 +164,7 @@ class SphinxBaseFileInput(FileInput): """A base class of SphinxFileInput. It supports to replace unknown Unicode characters to '?'. And it also emits - Sphinx events ``source-read`` on reading. + Sphinx events :event:`source-read` on reading. """ def __init__(self, app, env, *args, **kwds): @@ -198,7 +205,7 @@ class SphinxBaseFileInput(FileInput): if lineend == -1: lineend = len(error.object) lineno = error.object.count(b'\n', 0, error.start) + 1 - logger.warning('undecodable source characters, replacing with "?": %r', + logger.warning(__('undecodable source characters, replacing with "?": %r'), (error.object[linestart + 1:error.start] + b'>>>' + error.object[error.start:error.end] + b'<<<' + error.object[error.end:lineend]), @@ -274,14 +281,29 @@ class SphinxRSTFileInput(SphinxBaseFileInput): return lineno +class FiletypeNotFoundError(Exception): + pass + + +def get_filetype(source_suffix, filename): + # type: (Dict[unicode, unicode], unicode) -> unicode + for suffix, filetype in iteritems(source_suffix): + if filename.endswith(suffix): + # If default filetype (None), considered as restructuredtext. + return filetype or 'restructuredtext' + else: + raise FiletypeNotFoundError + + def read_doc(app, env, filename): # type: (Sphinx, BuildEnvironment, unicode) -> nodes.document """Parse a document and convert to doctree.""" - input_class = app.registry.get_source_input(filename) + filetype = get_filetype(app.config.source_suffix, filename) + input_class = app.registry.get_source_input(filetype) reader = SphinxStandaloneReader(app) source = input_class(app, env, source=None, source_path=filename, encoding=env.config.source_encoding) - parser = app.registry.create_source_parser(app, filename) + parser = app.registry.create_source_parser(app, filetype) pub = Publisher(reader=reader, parser=parser, diff --git a/sphinx/jinja2glue.py b/sphinx/jinja2glue.py index cc935d577..f19454c27 100644 --- a/sphinx/jinja2glue.py +++ b/sphinx/jinja2glue.py @@ -20,11 +20,12 @@ from jinja2.utils import open_if_exists from six import string_types from sphinx.application import TemplateBridge +from sphinx.util import logging from sphinx.util.osutil import mtimes_of_files if False: # For type annotation - from typing import Any, Callable, Dict, List, Iterator, Tuple # NOQA + from typing import Any, Callable, Dict, List, Iterator, Tuple, Union # NOQA from jinja2.environment import Environment # NOQA from sphinx.builders import Builder # NOQA from sphinx.theming import Theme # NOQA @@ -46,7 +47,7 @@ def _toint(val): def _todim(val): - # type (int or unicode) -> unicode + # type: (Union[int, unicode]) -> unicode """ Make val a css dimension. In particular the following transformations are performed: @@ -61,7 +62,7 @@ def _todim(val): return 'initial' elif str(val).isdigit(): return '0' if int(val) == 0 else '%spx' % val - return val + return val # type: ignore def _slice_index(values, slices): @@ -113,6 +114,17 @@ class idgen(object): next = __next__ # Python 2/Jinja compatibility +@contextfunction +def warning(context, message, *args, **kwargs): + # type: (Dict, unicode, Any, Any) -> unicode + if 'pagename' in context: + filename = context.get('pagename') + context.get('file_suffix', '') + message = 'in rendering %s: %s' % (filename, message) + logger = logging.getLogger('sphinx.themes') + logger.warning(message, *args, **kwargs) + return '' # return empty string not to output any values + + class SphinxFileSystemLoader(FileSystemLoader): """ FileSystemLoader subclass that is not so strict about '..' entries in @@ -186,6 +198,7 @@ class BuiltinTemplateLoader(TemplateBridge, BaseLoader): self.environment.filters['todim'] = _todim self.environment.filters['slice_index'] = _slice_index self.environment.globals['debug'] = contextfunction(pformat) + self.environment.globals['warning'] = warning self.environment.globals['accesskey'] = contextfunction(accesskey) self.environment.globals['idgen'] = idgen if use_i18n: diff --git a/sphinx/locale/__init__.py b/sphinx/locale/__init__.py index e148f2c12..b17da28e9 100644 --- a/sphinx/locale/__init__.py +++ b/sphinx/locale/__init__.py @@ -10,10 +10,16 @@ """ import gettext +import locale +import warnings +from collections import defaultdict +from gettext import NullTranslations -from six import PY3, text_type +from six import text_type from six.moves import UserString +from sphinx.deprecation import RemovedInSphinx30Warning + if False: # For type annotation from typing import Any, Callable, Dict, Iterator, List, Tuple # NOQA @@ -180,6 +186,8 @@ def mygettext(string): """Used instead of _ when creating TranslationProxies, because _ is not bound yet at that time. """ + warnings.warn('sphinx.locale.mygettext() is deprecated. Please use `_()` instead.', + RemovedInSphinx30Warning) return _(string) @@ -188,119 +196,162 @@ def lazy_gettext(string): """A lazy version of `gettext`.""" # if isinstance(string, _TranslationProxy): # return string + warnings.warn('sphinx.locale.laxy_gettext() is deprecated. Please use `_()` instead.', + RemovedInSphinx30Warning) return _TranslationProxy(mygettext, string) # type: ignore -l_ = lazy_gettext - - -admonitionlabels = { - 'attention': l_('Attention'), - 'caution': l_('Caution'), - 'danger': l_('Danger'), - 'error': l_('Error'), - 'hint': l_('Hint'), - 'important': l_('Important'), - 'note': l_('Note'), - 'seealso': l_('See also'), - 'tip': l_('Tip'), - 'warning': l_('Warning'), -} # type: Dict[unicode, unicode] - -versionlabels = { - 'versionadded': l_('New in version %s'), - 'versionchanged': l_('Changed in version %s'), - 'deprecated': l_('Deprecated since version %s'), -} # type: Dict[unicode, unicode] - -# XXX Python specific -pairindextypes = { - 'module': l_('module'), - 'keyword': l_('keyword'), - 'operator': l_('operator'), - 'object': l_('object'), - 'exception': l_('exception'), - 'statement': l_('statement'), - 'builtin': l_('built-in function'), -} # Dict[unicode, _TranslationProxy] - -translators = {} # type: Dict[unicode, Any] - -if PY3: - def _(message, *args): - # type: (unicode, *Any) -> unicode - try: - if len(args) <= 1: - return translators['sphinx'].gettext(message) - else: # support pluralization - return translators['sphinx'].ngettext(message, args[0], args[1]) - except KeyError: - return message -else: - def _(message, *args): - # type: (unicode, *Any) -> unicode - try: - if len(args) <= 1: - return translators['sphinx'].ugettext(message) - else: # support pluralization - return translators['sphinx'].ungettext(message, args[0], args[1]) - except KeyError: - return message - - -def __(message, *args): - # type: (unicode, *Any) -> unicode - """A dummy wrapper to i18n'ize exceptions and command line messages. - - In future, the messages are translated using LC_MESSAGES or any other - locale settings. - """ - return message if len(args) <= 1 else args[0] +translators = defaultdict(NullTranslations) # type: Dict[Tuple[unicode, unicode], NullTranslations] # NOQA -def init(locale_dirs, language, catalog='sphinx'): - # type: (List, unicode, unicode) -> Tuple[Any, bool] +def init(locale_dirs, language, catalog='sphinx', namespace='general'): + # type: (List[unicode], unicode, unicode, unicode) -> Tuple[NullTranslations, bool] """Look for message catalogs in `locale_dirs` and *ensure* that there is at least a NullTranslations catalog set in `translators`. If called multiple times or if several ``.mo`` files are found, their contents are merged together (thus making ``init`` reentrable). """ global translators - translator = translators.get(catalog) + translator = translators.get((namespace, catalog)) # ignore previously failed attempts to find message catalogs - if isinstance(translator, gettext.NullTranslations): + if translator.__class__ is NullTranslations: translator = None # the None entry is the system's default locale path has_translation = True + if language and '_' in language: + # for language having country code (like "de_AT") + languages = [language, language.split('_')[0]] + else: + languages = [language] + # loading for dir_ in locale_dirs: try: trans = gettext.translation(catalog, localedir=dir_, # type: ignore - languages=[language]) # type: ignore + languages=languages) if translator is None: translator = trans else: - translator._catalog.update(trans._catalog) # type: ignore + translator.add_fallback(trans) except Exception: # Language couldn't be found in the specified path pass - # guarantee translators[catalog] exists + # guarantee translators[(namespace, catalog)] exists if translator is None: - translator = gettext.NullTranslations() + translator = NullTranslations() has_translation = False - translators[catalog] = translator + translators[(namespace, catalog)] = translator if hasattr(translator, 'ugettext'): - translator.gettext = translator.ugettext + translator.gettext = translator.ugettext # type: ignore return translator, has_translation -def get_translator(catalog='sphinx'): - # type: (unicode) -> gettext.NullTranslations - global translators - translator = translators.get(catalog) - if translator is None: - translator = gettext.NullTranslations() - if hasattr(translator, 'ugettext'): - translator.gettext = translator.ugettext - return translator +def init_console(locale_dir, catalog): + # type: (unicode, unicode) -> Tuple[NullTranslations, bool] + """Initialize locale for console. + + .. versionadded:: 1.8 + """ + try: + # encoding is ignored + language, _ = locale.getlocale(locale.LC_MESSAGES) + except AttributeError: + # LC_MESSAGES is not always defined. Fallback to the default language + # in case it is not. + language = None + return init([locale_dir], language, catalog, 'console') + + +def get_translator(catalog='sphinx', namespace='general'): + # type: (unicode, unicode) -> NullTranslations + return translators[(namespace, catalog)] + + +def is_translator_registered(catalog='sphinx', namespace='general'): + # type: (unicode, unicode) -> bool + return (namespace, catalog) in translators + + +def _lazy_translate(catalog, namespace, message): + # type: (unicode, unicode, unicode) -> unicode + """Used instead of _ when creating TranslationProxy, because _ is + not bound yet at that time. + """ + translator = get_translator(catalog, namespace) + return translator.gettext(message) # type: ignore + + +def get_translation(catalog, namespace='general'): + """Get a translation function based on the *catalog* and *namespace*. + + The extension can use this API to translate the messages on the + extension:: + + import os + from sphinx.locale import get_translation + + _ = get_translation(__name__) + text = _('Hello Sphinx!') + + + def setup(app): + package_dir = path.abspath(path.dirname(__file__)) + locale_dir = os.path.join(package_dir, 'locales') + app.add_message_catalog(__name__, locale_dir) + + With this code, sphinx searches a message catalog from + ``${package_dir}/locales/${language}/LC_MESSAGES/${__name__}.mo`` + The :confval:`language` is used for the searching. + + .. versionadded:: 1.8 + """ + def gettext(message, *args): + # type: (unicode, *Any) -> unicode + if not is_translator_registered(catalog, namespace): + # not initialized yet + return _TranslationProxy(_lazy_translate, catalog, namespace, message) # type: ignore # NOQA + else: + translator = get_translator(catalog, namespace) + if len(args) <= 1: + return translator.gettext(message) # type: ignore + else: # support pluralization + return translator.ngettext(message, args[0], args[1]) # type: ignore + + return gettext + + +# A shortcut for sphinx-core +#: Translation function for messages on documentation (menu, labels, themes and so on). +#: This function follows :confval:`language` setting. +_ = get_translation('sphinx') +#: Translation function for console messages +#: This function follows locale setting (`LC_ALL`, `LC_MESSAGES` and so on). +__ = get_translation('sphinx', 'console') + + +def l_(*args): + warnings.warn('sphinx.locale.l_() is deprecated. Please use `_()` instead.', + RemovedInSphinx30Warning) + return _(*args) + + +# labels +admonitionlabels = { + 'attention': _('Attention'), + 'caution': _('Caution'), + 'danger': _('Danger'), + 'error': _('Error'), + 'hint': _('Hint'), + 'important': _('Important'), + 'note': _('Note'), + 'seealso': _('See also'), + 'tip': _('Tip'), + 'warning': _('Warning'), +} # type: Dict[unicode, unicode] + +# Moved to sphinx.directives.other (will be overrided later) +versionlabels = {} # type: Dict[unicode, unicode] + +# Moved to sphinx.domains.python (will be overrided later) +pairindextypes = {} # type: Dict[unicode, unicode] diff --git a/sphinx/make_mode.py b/sphinx/make_mode.py index aecad31b4..ae19e93ee 100644 --- a/sphinx/make_mode.py +++ b/sphinx/make_mode.py @@ -22,7 +22,7 @@ import sys from os import path import sphinx -from sphinx import cmdline +from sphinx.cmd.build import build_main from sphinx.util.console import color_terminal, nocolor, bold, blue # type: ignore from sphinx.util.osutil import cd, rmtree @@ -30,8 +30,6 @@ if False: # For type annotation from typing import List # NOQA -proj_name = os.getenv('SPHINXPROJ', '<project>') - BUILDERS = [ ("", "html", "to make standalone HTML files"), @@ -151,7 +149,7 @@ class Make(object): '-d', doctreedir, self.srcdir, self.builddir_join(builder)] - return cmdline.main(args + opts) + return build_main(args + opts) def run_make_mode(args): diff --git a/sphinx/parsers.py b/sphinx/parsers.py index 34822898f..db62e7154 100644 --- a/sphinx/parsers.py +++ b/sphinx/parsers.py @@ -91,7 +91,7 @@ class RSTParser(docutils.parsers.rst.Parser): def setup(app): # type: (Sphinx) -> Dict[unicode, Any] - app.add_source_parser('*', RSTParser) # register as a special parser + app.add_source_parser(RSTParser) return { 'version': 'builtin', diff --git a/sphinx/pycode/__init__.py b/sphinx/pycode/__init__.py index fca28817d..c4c055bf5 100644 --- a/sphinx/pycode/__init__.py +++ b/sphinx/pycode/__init__.py @@ -27,17 +27,19 @@ class ModuleAnalyzer(object): @classmethod def for_string(cls, string, modname, srcname='<string>'): + # type: (unicode, unicode, unicode) -> ModuleAnalyzer if isinstance(string, bytes): return cls(BytesIO(string), modname, srcname) - return cls(StringIO(string), modname, srcname, decoded=True) + return cls(StringIO(string), modname, srcname, decoded=True) # type: ignore @classmethod def for_file(cls, filename, modname): + # type: (unicode, unicode) -> ModuleAnalyzer if ('file', filename) in cls.cache: return cls.cache['file', filename] try: with open(filename, 'rb') as f: - obj = cls(f, modname, filename) + obj = cls(f, modname, filename) # type: ignore cls.cache['file', filename] = obj except Exception as err: raise PycodeError('error opening %r' % filename, err) @@ -45,6 +47,7 @@ class ModuleAnalyzer(object): @classmethod def for_module(cls, modname): + # type: (str) -> ModuleAnalyzer if ('module', modname) in cls.cache: entry = cls.cache['module', modname] if isinstance(entry, PycodeError): diff --git a/sphinx/pycode/parser.py b/sphinx/pycode/parser.py index deba48b1c..f943b7985 100644 --- a/sphinx/pycode/parser.py +++ b/sphinx/pycode/parser.py @@ -93,6 +93,7 @@ def dedent_docstring(s): # type: (unicode) -> unicode """Remove common leading indentation from docstring.""" def dummy(): + # type: () -> None # dummy function to mock `inspect.getdoc`. pass diff --git a/sphinx/quickstart.py b/sphinx/quickstart.py index ab4cf92da..8cad0640b 100644 --- a/sphinx/quickstart.py +++ b/sphinx/quickstart.py @@ -14,8 +14,13 @@ import warnings from sphinx.cmd.quickstart import main as _main from sphinx.deprecation import RemovedInSphinx20Warning +if False: + # For type annotation + from typing import Any # NOQA + def main(*args, **kwargs): + # type: (Any, Any) -> None warnings.warn( '`sphinx.quickstart.main()` has moved to `sphinx.cmd.quickstart.' 'main()`.', diff --git a/sphinx/registry.py b/sphinx/registry.py index cdae77224..e11f0654d 100644 --- a/sphinx/registry.py +++ b/sphinx/registry.py @@ -11,10 +11,15 @@ from __future__ import print_function import traceback +import warnings +from inspect import isclass +from types import MethodType +from docutils.parsers.rst import Directive from pkg_resources import iter_entry_points -from six import iteritems, itervalues, string_types +from six import iteritems, itervalues +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.domains import ObjType from sphinx.domains.std import GenericObject, Target from sphinx.errors import ExtensionError, SphinxError, VersionRequirementError @@ -22,14 +27,12 @@ from sphinx.extension import Extension from sphinx.locale import __ from sphinx.parsers import Parser as SphinxParser from sphinx.roles import XRefRole -from sphinx.util import import_object from sphinx.util import logging -from sphinx.util.console import bold # type: ignore from sphinx.util.docutils import directive_helper if False: # For type annotation - from typing import Any, Callable, Dict, Iterator, List, Type, Union # NOQA + from typing import Any, Callable, Dict, Iterator, List, Tuple, Type, Union # NOQA from docutils import nodes # NOQA from docutils.io import Input # NOQA from docutils.parsers import Parser # NOQA @@ -39,7 +42,7 @@ if False: from sphinx.domains import Domain, Index # NOQA from sphinx.environment import BuildEnvironment # NOQA from sphinx.ext.autodoc import Documenter # NOQA - from sphinx.util.typing import RoleFunction # NOQA + from sphinx.util.typing import RoleFunction, TitleGetter # NOQA logger = logging.getLogger(__name__) @@ -52,26 +55,73 @@ EXTENSION_BLACKLIST = { class SphinxComponentRegistry(object): def __init__(self): + # type: () -> None + #: special attrgetter for autodoc; class object -> attrgetter self.autodoc_attrgettrs = {} # type: Dict[Type, Callable[[Any, unicode, Any], Any]] + + #: builders; a dict of builder name -> bulider class self.builders = {} # type: Dict[unicode, Type[Builder]] + + #: autodoc documenters; a dict of documenter name -> documenter class self.documenters = {} # type: Dict[unicode, Type[Documenter]] + + #: css_files; a list of tuple of filename and attributes + self.css_files = [] # type: List[Tuple[unicode, Dict[unicode, unicode]]] + + #: domains; a dict of domain name -> domain class self.domains = {} # type: Dict[unicode, Type[Domain]] + + #: additional directives for domains + #: a dict of domain name -> dict of directive name -> directive self.domain_directives = {} # type: Dict[unicode, Dict[unicode, Any]] + + #: additional indices for domains + #: a dict of domain name -> list of index class self.domain_indices = {} # type: Dict[unicode, List[Type[Index]]] + + #: additional object types for domains + #: a dict of domain name -> dict of objtype name -> objtype self.domain_object_types = {} # type: Dict[unicode, Dict[unicode, ObjType]] + + #: additional roles for domains + #: a dict of domain name -> dict of role name -> role impl. self.domain_roles = {} # type: Dict[unicode, Dict[unicode, Union[RoleFunction, XRefRole]]] # NOQA + + #: additional enumerable nodes + #: a dict of node class -> tuple of figtype and title_getter function + self.enumerable_nodes = {} # type: Dict[nodes.Node, Tuple[unicode, TitleGetter]] + + #: LaTeX packages; list of package names and its options + self.latex_packages = [] # type: List[Tuple[unicode, unicode]] + + #: post transforms; list of transforms self.post_transforms = [] # type: List[Type[Transform]] - self.source_parsers = {} # type: Dict[unicode, Parser] + + #: source paresrs; file type -> parser class + self.source_parsers = {} # type: Dict[unicode, Type[Parser]] + + #: source inputs; file type -> input class self.source_inputs = {} # type: Dict[unicode, Input] + + #: source suffix: suffix -> file type + self.source_suffix = {} # type: Dict[unicode, unicode] + + #: custom translators; builder name -> translator class self.translators = {} # type: Dict[unicode, nodes.NodeVisitor] + + #: custom handlers for translators + #: a dict of builder name -> dict of node name -> visitor and departure functions + self.translation_handlers = {} # type: Dict[unicode, Dict[unicode, Tuple[Callable, Callable]]] # NOQA + + #: additional transforms; list of transforms self.transforms = [] # type: List[Type[Transform]] - def add_builder(self, builder): - # type: (Type[Builder]) -> None + def add_builder(self, builder, override=False): + # type: (Type[Builder], bool) -> None logger.debug('[app] adding builder: %r', builder) if not hasattr(builder, 'name'): raise ExtensionError(__('Builder class %s has no "name" attribute') % builder) - if builder.name in self.builders: + if builder.name in self.builders and not override: raise ExtensionError(__('Builder %r already exists (in module %s)') % (builder.name, self.builders[builder.name].__module__)) self.builders[builder.name] = builder @@ -98,10 +148,10 @@ class SphinxComponentRegistry(object): return self.builders[name](app) - def add_domain(self, domain): - # type: (Type[Domain]) -> None + def add_domain(self, domain, override=False): + # type: (Type[Domain], bool) -> None logger.debug('[app] adding domain: %r', domain) - if domain.name in self.domains: + if domain.name in self.domains and not override: raise ExtensionError(__('domain %s already registered') % domain.name) self.domains[domain.name] = domain @@ -125,44 +175,54 @@ class SphinxComponentRegistry(object): def override_domain(self, domain): # type: (Type[Domain]) -> None - logger.debug('[app] overriding domain: %r', domain) - if domain.name not in self.domains: - raise ExtensionError(__('domain %s not yet registered') % domain.name) - if not issubclass(domain, self.domains[domain.name]): - raise ExtensionError(__('new domain not a subclass of registered %s ' - 'domain') % domain.name) - self.domains[domain.name] = domain - - def add_directive_to_domain(self, domain, name, obj, - has_content=None, argument_spec=None, **option_spec): - # type: (unicode, unicode, Any, bool, Any, Any) -> None + warnings.warn('registry.override_domain() is deprecated. ' + 'Use app.add_domain(domain, override=True) instead.', + RemovedInSphinx30Warning) + self.add_domain(domain, override=True) + + def add_directive_to_domain(self, domain, name, obj, has_content=None, argument_spec=None, + override=False, **option_spec): + # type: (unicode, unicode, Any, bool, Any, bool, Any) -> None logger.debug('[app] adding directive to domain: %r', (domain, name, obj, has_content, argument_spec, option_spec)) if domain not in self.domains: raise ExtensionError(__('domain %s not yet registered') % domain) + directives = self.domain_directives.setdefault(domain, {}) - directives[name] = directive_helper(obj, has_content, argument_spec, **option_spec) + if name in directives and not override: + raise ExtensionError(__('The %r directive is already registered to %d domain') % + (name, domain)) + if not isclass(obj) or not issubclass(obj, Directive): + directives[name] = directive_helper(obj, has_content, argument_spec, **option_spec) + else: + directives[name] = obj - def add_role_to_domain(self, domain, name, role): - # type: (unicode, unicode, Union[RoleFunction, XRefRole]) -> None + def add_role_to_domain(self, domain, name, role, override=False): + # type: (unicode, unicode, Union[RoleFunction, XRefRole], bool) -> None logger.debug('[app] adding role to domain: %r', (domain, name, role)) if domain not in self.domains: raise ExtensionError(__('domain %s not yet registered') % domain) roles = self.domain_roles.setdefault(domain, {}) + if name in roles and not override: + raise ExtensionError(__('The %r role is already registered to %d domain') % + (name, domain)) roles[name] = role - def add_index_to_domain(self, domain, index): - # type: (unicode, Type[Index]) -> None + def add_index_to_domain(self, domain, index, override=False): + # type: (unicode, Type[Index], bool) -> None logger.debug('[app] adding index to domain: %r', (domain, index)) if domain not in self.domains: raise ExtensionError(__('domain %s not yet registered') % domain) indices = self.domain_indices.setdefault(domain, []) + if index in indices and not override: + raise ExtensionError(__('The %r index is already registered to %d domain') % + (index.name, domain)) indices.append(index) def add_object_type(self, directivename, rolename, indextemplate='', parse_node=None, ref_nodeclass=None, objname='', - doc_field_types=[]): - # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List) -> None + doc_field_types=[], override=False): + # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List, bool) -> None logger.debug('[app] adding object type: %r', (directivename, rolename, indextemplate, parse_node, ref_nodeclass, objname, doc_field_types)) @@ -178,11 +238,14 @@ class SphinxComponentRegistry(object): self.add_role_to_domain('std', rolename, XRefRole(innernodeclass=ref_nodeclass)) object_types = self.domain_object_types.setdefault('std', {}) + if directivename in object_types and not override: + raise ExtensionError(__('The %r object_type is already registered') % + directivename) object_types[directivename] = ObjType(objname or directivename, rolename) def add_crossref_type(self, directivename, rolename, indextemplate='', - ref_nodeclass=None, objname=''): - # type: (unicode, unicode, unicode, nodes.Node, unicode) -> None + ref_nodeclass=None, objname='', override=False): + # type: (unicode, unicode, unicode, nodes.Node, unicode, bool) -> None logger.debug('[app] adding crossref type: %r', (directivename, rolename, indextemplate, ref_nodeclass, objname)) @@ -195,30 +258,62 @@ class SphinxComponentRegistry(object): self.add_role_to_domain('std', rolename, XRefRole(innernodeclass=ref_nodeclass)) object_types = self.domain_object_types.setdefault('std', {}) + if directivename in object_types and not override: + raise ExtensionError(__('The %r crossref_type is already registered') % + directivename) object_types[directivename] = ObjType(objname or directivename, rolename) - def add_source_parser(self, suffix, parser): - # type: (unicode, Type[Parser]) -> None - logger.debug('[app] adding search source_parser: %r, %r', suffix, parser) - if suffix in self.source_parsers: + def add_source_suffix(self, suffix, filetype, override=False): + # type: (unicode, unicode, bool) -> None + logger.debug('[app] adding source_suffix: %r, %r', suffix, filetype) + if suffix in self.source_suffix and not override: raise ExtensionError(__('source_parser for %r is already registered') % suffix) - self.source_parsers[suffix] = parser - - def get_source_parser(self, filename): - # type: (unicode) -> Type[Parser] - for suffix, parser_class in iteritems(self.source_parsers): - if filename.endswith(suffix): - break else: - # use special parser for unknown file-extension '*' (if exists) - parser_class = self.source_parsers.get('*') - - if parser_class is None: - raise SphinxError(__('source_parser for %s not registered') % filename) + self.source_suffix[suffix] = filetype + + def add_source_parser(self, *args, **kwargs): + # type: (Any, bool) -> None + logger.debug('[app] adding search source_parser: %r', args) + if len(args) == 1: + # new sytle arguments: (source_parser) + suffix = None # type: unicode + parser = args[0] # type: Type[Parser] else: - if isinstance(parser_class, string_types): - parser_class = import_object(parser_class, 'source parser') # type: ignore - return parser_class + # old style arguments: (suffix, source_parser) + warnings.warn('app.add_source_parser() does not support suffix argument. ' + 'Use app.add_source_suffix() instead.', + RemovedInSphinx30Warning) + suffix = args[0] + parser = args[1] + + if suffix: + self.add_source_suffix(suffix, suffix) + + if len(parser.supported) == 0: + warnings.warn('Old source_parser has been detected. Please fill Parser.supported ' + 'attribute: %s' % parser.__name__, + RemovedInSphinx30Warning) + + # create a map from filetype to parser + for filetype in parser.supported: + if filetype in self.source_parsers and not kwargs.get('override'): + raise ExtensionError(__('source_parser for %r is already registered') % + filetype) + else: + self.source_parsers[filetype] = parser + + # also maps suffix to parser + # + # This rescues old styled parsers which does not have ``supported`` filetypes. + if suffix: + self.source_parsers[suffix] = parser + + def get_source_parser(self, filetype): + # type: (unicode) -> Type[Parser] + try: + return self.source_parsers[filetype] + except KeyError: + raise SphinxError(__('Source parser for %s not registered') % filetype) def get_source_parsers(self): # type: () -> Dict[unicode, Parser] @@ -232,44 +327,67 @@ class SphinxComponentRegistry(object): parser.set_application(app) return parser - def add_source_input(self, input_class): - # type: (Type[Input]) -> None + def add_source_input(self, input_class, override=False): + # type: (Type[Input], bool) -> None for filetype in input_class.supported: - if filetype in self.source_inputs: + if filetype in self.source_inputs and not override: raise ExtensionError(__('source_input for %r is already registered') % filetype) self.source_inputs[filetype] = input_class - def get_source_input(self, filename): + def get_source_input(self, filetype): # type: (unicode) -> Type[Input] - parser = self.get_source_parser(filename) - for filetype in parser.supported: - if filetype in self.source_inputs: - input_class = self.source_inputs[filetype] - break - else: - # use special source_input for unknown file-type '*' (if exists) - input_class = self.source_inputs.get('*') - - if input_class is None: - raise SphinxError(__('source_input for %s not registered') % filename) - else: - return input_class - - def add_translator(self, name, translator): - # type: (unicode, Type[nodes.NodeVisitor]) -> None - logger.info(bold(__('Change of translator for the %s builder.') % name)) + try: + return self.source_inputs[filetype] + except KeyError: + try: + # use special source_input for unknown filetype + return self.source_inputs['*'] + except KeyError: + raise SphinxError(__('source_input for %s not registered') % filetype) + + def add_translator(self, name, translator, override=False): + # type: (unicode, Type[nodes.NodeVisitor], bool) -> None + logger.debug('[app] Change of translator for the %s builder.' % name) + if name in self.translators and not override: + raise ExtensionError(__('Translatoro for %r already exists') % name) self.translators[name] = translator + def add_translation_handlers(self, node, **kwargs): + # type: (nodes.Node, Any) -> None + logger.debug('[app] adding translation_handlers: %r, %r', node, kwargs) + for builder_name, handlers in iteritems(kwargs): + translation_handlers = self.translation_handlers.setdefault(builder_name, {}) + try: + visit, depart = handlers # unpack once for assertion + translation_handlers[node.__name__] = (visit, depart) + except ValueError: + raise ExtensionError(__('kwargs for add_node() must be a (visit, depart) ' + 'function tuple: %r=%r') % builder_name, handlers) + def get_translator_class(self, builder): # type: (Builder) -> Type[nodes.NodeVisitor] return self.translators.get(builder.name, builder.default_translator_class) - def create_translator(self, builder, document): - # type: (Builder, nodes.Node) -> nodes.NodeVisitor + def create_translator(self, builder, *args): + # type: (Builder, Any) -> nodes.NodeVisitor translator_class = self.get_translator_class(builder) - return translator_class(builder, document) + assert translator_class, "translator not found for %s" % builder.name + translator = translator_class(*args) + + # transplant handlers for custom nodes to translator instance + handlers = self.translation_handlers.get(builder.name, None) + if handlers is None: + # retry with builder.format + handlers = self.translation_handlers.get(builder.format, {}) + + for name, (visit, depart) in iteritems(handlers): + setattr(translator, 'visit_' + name, MethodType(visit, translator)) + if depart: + setattr(translator, 'depart_' + name, MethodType(depart, translator)) + + return translator def add_transform(self, transform): # type: (Type[Transform]) -> None @@ -297,6 +415,21 @@ class SphinxComponentRegistry(object): # type: (Type, Callable[[Any, unicode, Any], Any]) -> None self.autodoc_attrgettrs[typ] = attrgetter + def add_css_files(self, filename, **attributes): + self.css_files.append((filename, attributes)) + + def add_latex_package(self, name, options): + # type: (unicode, unicode) -> None + logger.debug('[app] adding latex package: %r', name) + self.latex_packages.append((name, options)) + + def add_enumerable_node(self, node, figtype, title_getter=None, override=False): + # type: (nodes.Node, unicode, TitleGetter, bool) -> None + logger.debug('[app] adding enumerable node: (%r, %r, %r)', node, figtype, title_getter) + if node in self.enumerable_nodes and not override: + raise ExtensionError(__('enumerable_node %r already registered') % node) + self.enumerable_nodes[node] = (figtype, title_getter) + def load_extension(self, app, extname): # type: (Sphinx, unicode) -> None """Load a Sphinx extension.""" @@ -334,8 +467,6 @@ class SphinxComponentRegistry(object): if metadata is None: metadata = {} - if extname == 'rst2pdf.pdfbuilder': - metadata['parallel_read_safe'] = True elif not isinstance(metadata, dict): logger.warning(__('extension %r returned an unsupported object from ' 'its setup() function; it should return None or a ' @@ -344,3 +475,37 @@ class SphinxComponentRegistry(object): app.extensions[extname] = Extension(extname, mod, **metadata) app._setting_up_extension.pop() + + def get_envversion(self, app): + # type: (Sphinx) -> Dict[unicode, unicode] + from sphinx.environment import ENV_VERSION + envversion = {ext.name: ext.metadata['env_version'] for ext in app.extensions.values() + if ext.metadata.get('env_version')} + envversion['sphinx'] = ENV_VERSION + return envversion + + +def merge_source_suffix(app): + # type: (Sphinx) -> None + """Merge source_suffix which specified by user and added by extensions.""" + for suffix, filetype in iteritems(app.registry.source_suffix): + if suffix not in app.config.source_suffix: + app.config.source_suffix[suffix] = filetype + elif app.config.source_suffix[suffix] is None: + # filetype is not specified (default filetype). + # So it overrides default filetype by extensions setting. + app.config.source_suffix[suffix] = filetype + + # copy config.source_suffix to registry + app.registry.source_suffix = app.config.source_suffix + + +def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] + app.connect('builder-inited', merge_source_suffix) + + return { + 'version': 'builtin', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/roles.py b/sphinx/roles.py index 2971c8fc3..453eb5aa0 100644 --- a/sphinx/roles.py +++ b/sphinx/roles.py @@ -284,6 +284,7 @@ def menusel_role(typ, rawtext, text, lineno, inliner, options={}, content=[]): _litvar_re = re.compile('{([^}]+)}') +parens_re = re.compile(r'(\\*{|\\*})') def emph_literal_role(typ, rawtext, text, lineno, inliner, @@ -296,17 +297,43 @@ def emph_literal_role(typ, rawtext, text, lineno, inliner, else: typ = typ.lower() - text = utils.unescape(text) - pos = 0 retnode = nodes.literal(role=typ.lower(), classes=[typ]) - for m in _litvar_re.finditer(text): # type: ignore - if m.start() > pos: - txt = text[pos:m.start()] - retnode += nodes.Text(txt, txt) - retnode += nodes.emphasis(m.group(1), m.group(1)) - pos = m.end() - if pos < len(text): - retnode += nodes.Text(text[pos:], text[pos:]) + parts = list(parens_re.split(utils.unescape(text))) + stack = [''] + for part in parts: + matched = parens_re.match(part) + if matched: + backslashes = len(part) - 1 + if backslashes % 2 == 1: # escaped + stack[-1] += "\\" * int((backslashes - 1) / 2) + part[-1] + elif part[-1] == '{': # rparen + stack[-1] += "\\" * int(backslashes / 2) + if len(stack) >= 2 and stack[-2] == "{": + # nested + stack[-1] += "{" + else: + # start emphasis + stack.append('{') + stack.append('') + else: # lparen + stack[-1] += "\\" * int(backslashes / 2) + if len(stack) == 3 and stack[1] == "{" and len(stack[2]) > 0: + # emphasized word found + if stack[0]: + retnode += nodes.Text(stack[0], stack[0]) + retnode += nodes.emphasis(stack[2], stack[2]) + stack = [''] + else: + # emphasized word not found; the rparen is not a special symbol + stack.append('}') + stack = [''.join(stack)] + else: + stack[-1] += part + if ''.join(stack): + # remaining is treated as Text + text = ''.join(stack) + retnode += nodes.Text(text, text) + return [retnode], [] diff --git a/sphinx/search/da.py b/sphinx/search/da.py index 8b7ad2502..eb7394b5e 100644 --- a/sphinx/search/da.py +++ b/sphinx/search/da.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + danish_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/danish/stop.txt og | and @@ -125,7 +130,9 @@ class SearchDanish(SearchLanguage): stopwords = danish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('danish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/de.py b/sphinx/search/de.py index f511748f7..90fc4d0fe 100644 --- a/sphinx/search/de.py +++ b/sphinx/search/de.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + german_stopwords = parse_stop_word(u''' |source: http://snowball.tartarus.org/algorithms/german/stop.txt aber | but @@ -308,7 +313,9 @@ class SearchGerman(SearchLanguage): stopwords = german_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('german') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/es.py b/sphinx/search/es.py index c18b97da0..6fda9caba 100644 --- a/sphinx/search/es.py +++ b/sphinx/search/es.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + spanish_stopwords = parse_stop_word(u''' |source: http://snowball.tartarus.org/algorithms/spanish/stop.txt de | from, of @@ -368,7 +373,9 @@ class SearchSpanish(SearchLanguage): stopwords = spanish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('spanish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/fi.py b/sphinx/search/fi.py index 612d204be..bf757d54a 100644 --- a/sphinx/search/fi.py +++ b/sphinx/search/fi.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + finnish_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/finnish/stop.txt | forms of BE @@ -118,7 +123,9 @@ class SearchFinnish(SearchLanguage): stopwords = finnish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('finnish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/fr.py b/sphinx/search/fr.py index 615a47383..9976c1ca7 100644 --- a/sphinx/search/fr.py +++ b/sphinx/search/fr.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + french_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/french/stop.txt au | a + le @@ -204,7 +209,9 @@ class SearchFrench(SearchLanguage): stopwords = french_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('french') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/hu.py b/sphinx/search/hu.py index c8deab97d..03cbf8c29 100644 --- a/sphinx/search/hu.py +++ b/sphinx/search/hu.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + hungarian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/hungarian/stop.txt | prepared by Anna Tordai @@ -232,7 +237,9 @@ class SearchHungarian(SearchLanguage): stopwords = hungarian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('hungarian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/it.py b/sphinx/search/it.py index 7351f3373..860ab9325 100644 --- a/sphinx/search/it.py +++ b/sphinx/search/it.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + italian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/italian/stop.txt ad | a (to) before vowel @@ -321,7 +326,9 @@ class SearchItalian(SearchLanguage): stopwords = italian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('italian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/ja.py b/sphinx/search/ja.py index 0cdc14a11..a48a0df69 100644 --- a/sphinx/search/ja.py +++ b/sphinx/search/ja.py @@ -20,6 +20,7 @@ import os import re import sys +import warnings from six import iteritems, PY3 @@ -35,6 +36,7 @@ try: except ImportError: janome_module = False +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.errors import SphinxError, ExtensionError from sphinx.search import SearchLanguage from sphinx.util import import_object @@ -556,9 +558,12 @@ class SearchJapanese(SearchLanguage): def init(self, options): # type: (Dict) -> None - type = options.get('type', 'default') + type = options.get('type', 'sphinx.search.ja.DefaultSplitter') if type in self.splitters: dotted_path = self.splitters[type] + warnings.warn('html_search_options["type"]: %s is deprecated. ' + 'Please give "%s" instead.' % (type, dotted_path), + RemovedInSphinx30Warning) else: dotted_path = type try: diff --git a/sphinx/search/nl.py b/sphinx/search/nl.py index 909527449..de4fd13ec 100644 --- a/sphinx/search/nl.py +++ b/sphinx/search/nl.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + dutch_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/dutch/stop.txt de | the @@ -132,7 +137,9 @@ class SearchDutch(SearchLanguage): stopwords = dutch_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('dutch') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/no.py b/sphinx/search/no.py index 3b68f4e7c..81876bcdd 100644 --- a/sphinx/search/no.py +++ b/sphinx/search/no.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + norwegian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/norwegian/stop.txt og | and @@ -207,7 +212,9 @@ class SearchNorwegian(SearchLanguage): stopwords = norwegian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('norwegian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/pt.py b/sphinx/search/pt.py index 170ee01f9..9afe80870 100644 --- a/sphinx/search/pt.py +++ b/sphinx/search/pt.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + portuguese_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/portuguese/stop.txt de | of, from @@ -267,7 +272,9 @@ class SearchPortuguese(SearchLanguage): stopwords = portuguese_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('portuguese') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/ru.py b/sphinx/search/ru.py index 0cdb4bb61..cc189a953 100644 --- a/sphinx/search/ru.py +++ b/sphinx/search/ru.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + russian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/russian/stop.txt и | and @@ -256,7 +261,9 @@ class SearchRussian(SearchLanguage): stopwords = russian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('russian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/sv.py b/sphinx/search/sv.py index 186b56a3e..fdb64685e 100644 --- a/sphinx/search/sv.py +++ b/sphinx/search/sv.py @@ -13,6 +13,10 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any + swedish_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/swedish/stop.txt och | and @@ -145,7 +149,9 @@ class SearchSwedish(SearchLanguage): stopwords = swedish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('swedish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/setup_command.py b/sphinx/setup_command.py index ddb2f1105..c54179227 100644 --- a/sphinx/setup_command.py +++ b/sphinx/setup_command.py @@ -116,11 +116,11 @@ class BuildDoc(Command): for root, dirnames, filenames in os.walk(guess): if 'conf.py' in filenames: return root - return None + return os.curdir # Overriding distutils' Command._ensure_stringlike which doesn't support # unicode, causing finalize_options to fail if invoked again. Workaround - # for http://bugs.python.org/issue19570 + # for https://bugs.python.org/issue19570 def _ensure_stringlike(self, option, what, default=None): # type: (unicode, unicode, Any) -> Any val = getattr(self, option) @@ -134,30 +134,26 @@ class BuildDoc(Command): def finalize_options(self): # type: () -> None + self.ensure_string_list('builder') + if self.source_dir is None: self.source_dir = self._guess_source_dir() self.announce('Using source directory %s' % self.source_dir) + self.ensure_dirname('source_dir') - if self.source_dir is None: - self.source_dir = os.curdir - self.source_dir = abspath(self.source_dir) + if self.config_dir is None: self.config_dir = self.source_dir - self.config_dir = abspath(self.config_dir) - self.ensure_string_list('builder') if self.build_dir is None: build = self.get_finalized_command('build') self.build_dir = os.path.join(abspath(build.build_base), 'sphinx') # type: ignore - self.mkpath(self.build_dir) # type: ignore - self.build_dir = abspath(self.build_dir) + self.doctree_dir = os.path.join(self.build_dir, 'doctrees') - self.mkpath(self.doctree_dir) # type: ignore + self.builder_target_dirs = [ (builder, os.path.join(self.build_dir, builder)) for builder in self.builder] # type: List[Tuple[str, unicode]] - for _, builder_target_dir in self.builder_target_dirs: - self.mkpath(builder_target_dir) # type: ignore def run(self): # type: () -> None @@ -183,7 +179,8 @@ class BuildDoc(Command): app = None try: - with patch_docutils(), docutils_namespace(): + confdir = self.config_dir or self.source_dir + with patch_docutils(confdir), docutils_namespace(): app = Sphinx(self.source_dir, self.config_dir, builder_target_dir, self.doctree_dir, builder, confoverrides, status_stream, diff --git a/sphinx/templates/latex/latex.tex_t b/sphinx/templates/latex/latex.tex_t index 34086b1cb..0ea75557f 100644 --- a/sphinx/templates/latex/latex.tex_t +++ b/sphinx/templates/latex/latex.tex_t @@ -25,6 +25,7 @@ <%= fncychap %> \usepackage<%= sphinxpkgoptions %>{sphinx} <%= sphinxsetup %> +<%= fvset %> <%= geometry %> <%= usepackages %> <%= hyperref %> diff --git a/sphinx/templates/latex/longtable.tex_t b/sphinx/templates/latex/longtable.tex_t index b7310a780..5a151808c 100644 --- a/sphinx/templates/latex/longtable.tex_t +++ b/sphinx/templates/latex/longtable.tex_t @@ -25,8 +25,5 @@ \endfoot \endlastfoot -<% if table.caption_footnotetexts -%> -<%= ''.join(table.caption_footnotetexts) %> -<% endif -%> <%= ''.join(table.body) %> \end{longtable}\sphinxatlongtableend\end{savenotes} diff --git a/sphinx/templates/latex/tabular.tex_t b/sphinx/templates/latex/tabular.tex_t index 3fd347e53..a35d2b2cb 100644 --- a/sphinx/templates/latex/tabular.tex_t +++ b/sphinx/templates/latex/tabular.tex_t @@ -18,9 +18,6 @@ \begin{tabular}[t]<%= table.get_colspec() -%> \hline <%= ''.join(table.header) %> -<%- if table.caption_footnotetexts -%> -<%= ''.join(table.caption_footnotetexts) -%> -<%- endif -%> <%=- ''.join(table.body) %> \end{tabular} \par diff --git a/sphinx/templates/latex/tabulary.tex_t b/sphinx/templates/latex/tabulary.tex_t index 16d15192b..331ef845d 100644 --- a/sphinx/templates/latex/tabulary.tex_t +++ b/sphinx/templates/latex/tabulary.tex_t @@ -18,9 +18,6 @@ \begin{tabulary}{\linewidth}[t]<%= table.get_colspec() -%> \hline <%= ''.join(table.header) %> -<%- if table.caption_footnotetexts -%> -<%= ''.join(table.caption_footnotetexts) -%> -<%- endif -%> <%=- ''.join(table.body) %> \end{tabulary} \par diff --git a/sphinx/templates/quickstart/Makefile.new_t b/sphinx/templates/quickstart/Makefile.new_t index c7cd62dda..16a9d482f 100644 --- a/sphinx/templates/quickstart/Makefile.new_t +++ b/sphinx/templates/quickstart/Makefile.new_t @@ -4,7 +4,6 @@ # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build -SPHINXPROJ = {{ project_fn }} SOURCEDIR = {{ rsrcdir }} BUILDDIR = {{ rbuilddir }} diff --git a/sphinx/templates/quickstart/conf.py_t b/sphinx/templates/quickstart/conf.py_t index 6a648f39c..cc8073a82 100644 --- a/sphinx/templates/quickstart/conf.py_t +++ b/sphinx/templates/quickstart/conf.py_t @@ -74,7 +74,7 @@ language = {{ language | repr }} # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. -# This pattern also affects html_static_path and html_extra_path . +# This pattern also affects html_static_path and html_extra_path. exclude_patterns = [{{ exclude_patterns }}] # The name of the Pygments (syntax highlighting) style to use. diff --git a/sphinx/templates/quickstart/make.bat.new_t b/sphinx/templates/quickstart/make.bat.new_t index e49ffbe78..50a12e7af 100644 --- a/sphinx/templates/quickstart/make.bat.new_t +++ b/sphinx/templates/quickstart/make.bat.new_t @@ -9,7 +9,6 @@ if "%SPHINXBUILD%" == "" ( ) set SOURCEDIR={{ rsrcdir }} set BUILDDIR={{ rbuilddir }} -set SPHINXPROJ={{ project_fn }} if "%1" == "" goto help diff --git a/sphinx/testing/fixtures.py b/sphinx/testing/fixtures.py index be0037b70..76ed154fd 100644 --- a/sphinx/testing/fixtures.py +++ b/sphinx/testing/fixtures.py @@ -22,16 +22,19 @@ from six import StringIO, string_types from . import util if False: - from typing import Dict, Union # NOQA + # For type annotation + from typing import Any, Dict, Union # NOQA @pytest.fixture(scope='session') def rootdir(): + # type: () -> None return None @pytest.fixture def app_params(request, test_params, shared_result, sphinx_test_tempdir, rootdir): + # type: (Any, Any, Any, Any, Any) -> None """ parameters that is specified by 'pytest.mark.sphinx' for sphinx.application.Sphinx initialization @@ -137,12 +140,14 @@ def warning(app): @pytest.fixture() -def make_app(test_params): +def make_app(test_params, monkeypatch): """ provides make_app function to initialize SphinxTestApp instance. if you want to initialize 'app' in your test function. please use this instead of using SphinxTestApp class directory. """ + monkeypatch.setattr('sphinx.application.abspath', lambda x: x) + apps = [] syspath = sys.path[:] @@ -153,7 +158,7 @@ def make_app(test_params): app_ = util.SphinxTestApp(*args, **kwargs) # type: Union[util.SphinxTestApp, util.SphinxTestAppWrapperForSkipBuilding] # NOQA apps.append(app_) if test_params['shared_result']: - app_ = util.SphinxTestAppWrapperForSkipBuilding(app_) + app_ = util.SphinxTestAppWrapperForSkipBuilding(app_) # type: ignore return app_ yield make diff --git a/sphinx/testing/path.py b/sphinx/testing/path.py index 5836d2bc0..209046246 100644 --- a/sphinx/testing/path.py +++ b/sphinx/testing/path.py @@ -13,6 +13,10 @@ from io import open from six import PY2, text_type +if False: + # For type annotation + from typing import Any, Callable, IO, List # NOQA + FILESYSTEMENCODING = sys.getfilesystemencoding() or sys.getdefaultencoding() @@ -23,58 +27,68 @@ class path(text_type): """ if PY2: def __new__(cls, s, encoding=FILESYSTEMENCODING, errors='strict'): + # type: (unicode, unicode, unicode) -> path if isinstance(s, str): s = s.decode(encoding, errors) - return text_type.__new__(cls, s) # type: ignore + return text_type.__new__(cls, s) return text_type.__new__(cls, s) # type: ignore @property def parent(self): + # type: () -> path """ The name of the directory the file or directory is in. """ return self.__class__(os.path.dirname(self)) def basename(self): + # type: () -> unicode return os.path.basename(self) def abspath(self): + # type: () -> path """ Returns the absolute path. """ return self.__class__(os.path.abspath(self)) def isabs(self): + # type: () -> bool """ Returns ``True`` if the path is absolute. """ return os.path.isabs(self) def isdir(self): + # type: () -> bool """ Returns ``True`` if the path is a directory. """ return os.path.isdir(self) def isfile(self): + # type: () -> bool """ Returns ``True`` if the path is a file. """ return os.path.isfile(self) def islink(self): + # type: () -> bool """ Returns ``True`` if the path is a symbolic link. """ return os.path.islink(self) def ismount(self): + # type: () -> bool """ Returns ``True`` if the path is a mount point. """ return os.path.ismount(self) def rmtree(self, ignore_errors=False, onerror=None): + # type: (bool, Callable) -> None """ Removes the file or directory and any files or directories it may contain. @@ -93,6 +107,7 @@ class path(text_type): shutil.rmtree(self, ignore_errors=ignore_errors, onerror=onerror) def copytree(self, destination, symlinks=False): + # type: (unicode, bool) -> None """ Recursively copy a directory to the given `destination`. If the given `destination` does not exist it will be created. @@ -105,6 +120,7 @@ class path(text_type): shutil.copytree(self, destination, symlinks=symlinks) def movetree(self, destination): + # type: (unicode) -> None """ Recursively move the file or directory to the given `destination` similar to the Unix "mv" command. @@ -117,24 +133,29 @@ class path(text_type): move = movetree def unlink(self): + # type: () -> None """ Removes a file. """ os.unlink(self) def stat(self): + # type: () -> Any """ Returns a stat of the file. """ return os.stat(self) def utime(self, arg): + # type: (Any) -> None os.utime(self, arg) def open(self, mode='r', **kwargs): + # type: (unicode, Any) -> IO return open(self, mode, **kwargs) def write_text(self, text, encoding='utf-8', **kwargs): + # type: (unicode, unicode, Any) -> None """ Writes the given `text` to the file. """ @@ -144,6 +165,7 @@ class path(text_type): f.write(text) def text(self, encoding='utf-8', **kwargs): + # type: (unicode, Any) -> unicode """ Returns the text in the file. """ @@ -152,6 +174,7 @@ class path(text_type): return f.read() def bytes(self): + # type: () -> str """ Returns the bytes in the file. """ @@ -159,6 +182,7 @@ class path(text_type): return f.read() def write_bytes(self, bytes, append=False): + # type: (str, bool) -> None """ Writes the given `bytes` to the file. @@ -173,12 +197,14 @@ class path(text_type): f.write(bytes) def exists(self): + # type: () -> bool """ Returns ``True`` if the path exist. """ return os.path.exists(self) def lexists(self): + # type: () -> bool """ Returns ``True`` if the path exists unless it is a broken symbolic link. @@ -186,21 +212,25 @@ class path(text_type): return os.path.lexists(self) def makedirs(self, mode=0o777): + # type: (int) -> None """ Recursively create directories. """ os.makedirs(self, mode) def joinpath(self, *args): + # type: (Any) -> path """ Joins the path with the argument given and returns the result. """ - return self.__class__(os.path.join(self, *map(self.__class__, args))) + return self.__class__(os.path.join(self, *map(self.__class__, args))) # type: ignore # NOQA def listdir(self): + # type: () -> List[unicode] return os.listdir(self) __div__ = __truediv__ = joinpath def __repr__(self): + # type: () -> str return '%s(%s)' % (self.__class__.__name__, text_type.__repr__(self)) diff --git a/sphinx/testing/util.py b/sphinx/testing/util.py index f836a97f4..24f5267b8 100644 --- a/sphinx/testing/util.py +++ b/sphinx/testing/util.py @@ -18,7 +18,7 @@ from docutils import nodes from docutils.parsers.rst import directives, roles from six import string_types -from sphinx import application +from sphinx import application, locale from sphinx.builders.latex import LaTeXBuilder from sphinx.ext.autodoc import AutoDirective from sphinx.pycode import ModuleAnalyzer @@ -26,7 +26,9 @@ from sphinx.testing.path import path from sphinx.util.osutil import relpath if False: + # For type annotation from typing import List # NOQA + from typing import Any, Dict, Generator, IO, List, Pattern # NOQA __all__ = [ @@ -37,21 +39,25 @@ __all__ = [ def assert_re_search(regex, text, flags=0): + # type: (Pattern, unicode, int) -> None if not re.search(regex, text, flags): assert False, '%r did not match %r' % (regex, text) def assert_not_re_search(regex, text, flags=0): + # type: (Pattern, unicode, int) -> None if re.search(regex, text, flags): assert False, '%r did match %r' % (regex, text) def assert_startswith(thing, prefix): + # type: (unicode, unicode) -> None if not thing.startswith(prefix): assert False, '%r does not start with %r' % (thing, prefix) def assert_node(node, cls=None, xpath="", **kwargs): + # type: (nodes.Node, Any, unicode, Any) -> None if cls: if isinstance(cls, list): assert_node(node, cls[0], xpath=xpath, **kwargs) @@ -81,13 +87,15 @@ def assert_node(node, cls=None, xpath="", **kwargs): def etree_parse(path): + # type: (unicode) -> Any with warnings.catch_warnings(record=False): warnings.filterwarnings("ignore", category=DeprecationWarning) - return ElementTree.parse(path) + return ElementTree.parse(path) # type: ignore class Struct(object): def __init__(self, **kwds): + # type: (Any) -> None self.__dict__.update(kwds) @@ -100,6 +108,7 @@ class SphinxTestApp(application.Sphinx): def __init__(self, buildername='html', srcdir=None, freshenv=False, confoverrides=None, status=None, warning=None, tags=None, docutilsconf=None): + # type: (unicode, path, bool, Dict, IO, IO, unicode, unicode) -> None if docutilsconf is not None: (srcdir / 'docutils.conf').write_text(docutilsconf) @@ -128,7 +137,7 @@ class SphinxTestApp(application.Sphinx): if v.startswith('visit_')) try: - application.Sphinx.__init__(self, srcdir, confdir, outdir, doctreedir, + application.Sphinx.__init__(self, srcdir, confdir, outdir, doctreedir, # type: ignore # NOQA buildername, confoverrides, status, warning, freshenv, warningiserror, tags) except Exception: @@ -136,9 +145,11 @@ class SphinxTestApp(application.Sphinx): raise def cleanup(self, doctrees=False): + # type: (bool) -> None AutoDirective._registry.clear() ModuleAnalyzer.cache.clear() LaTeXBuilder.usepackages = [] + locale.translators.clear() sys.path[:] = self._saved_path sys.modules.pop('autodoc_fodder', None) directives._directives = self._saved_directives @@ -150,6 +161,7 @@ class SphinxTestApp(application.Sphinx): delattr(nodes.GenericNodeVisitor, 'depart_' + method[6:]) def __repr__(self): + # type: () -> str return '<%s buildername=%r>' % (self.__class__.__name__, self.builder.name) @@ -161,13 +173,16 @@ class SphinxTestAppWrapperForSkipBuilding(object): """ def __init__(self, app_): + # type: (SphinxTestApp) -> None self.app = app_ def __getattr__(self, name): + # type: (str) -> Any return getattr(self.app, name) def build(self, *args, **kw): - if not self.app.outdir.listdir(): + # type: (Any, Any) -> None + if not self.app.outdir.listdir(): # type: ignore # if listdir is empty, do build. self.app.build(*args, **kw) # otherwise, we can use built cache @@ -177,16 +192,19 @@ _unicode_literals_re = re.compile(r'u(".*?")|u(\'.*?\')') def remove_unicode_literals(s): - return _unicode_literals_re.sub(lambda x: x.group(1) or x.group(2), s) + # type: (unicode) -> unicode + return _unicode_literals_re.sub(lambda x: x.group(1) or x.group(2), s) # type: ignore def find_files(root, suffix=None): + # type: (unicode, bool) -> Generator for dirpath, dirs, files in os.walk(root, followlinks=True): dirpath = path(dirpath) - for f in [f for f in files if not suffix or f.endswith(suffix)]: + for f in [f for f in files if not suffix or f.endswith(suffix)]: # type: ignore fpath = dirpath / f yield relpath(fpath, root) def strip_escseq(text): + # type: (unicode) -> unicode return re.sub('\x1b.*?m', '', text) diff --git a/sphinx/texinputs/footnotehyper-sphinx.sty b/sphinx/texinputs/footnotehyper-sphinx.sty index 5995f012d..b6692cfb8 100644 --- a/sphinx/texinputs/footnotehyper-sphinx.sty +++ b/sphinx/texinputs/footnotehyper-sphinx.sty @@ -4,10 +4,10 @@ %% %% Package: footnotehyper-sphinx %% Version: based on footnotehyper.sty 2017/03/07 v1.0 -%% as available at http://www.ctan.org/pkg/footnotehyper +%% as available at https://www.ctan.org/pkg/footnotehyper %% License: the one applying to Sphinx %% -%% Refer to the PDF documentation at http://www.ctan.org/pkg/footnotehyper for +%% Refer to the PDF documentation at https://www.ctan.org/pkg/footnotehyper for %% the code comments. %% %% Differences: diff --git a/sphinx/texinputs/sphinx.sty b/sphinx/texinputs/sphinx.sty index e323b2a5d..ef21691e3 100644 --- a/sphinx/texinputs/sphinx.sty +++ b/sphinx/texinputs/sphinx.sty @@ -6,7 +6,7 @@ % \NeedsTeXFormat{LaTeX2e}[1995/12/01] -\ProvidesPackage{sphinx}[2018/03/11 v1.7.2 LaTeX package (Sphinx markup)] +\ProvidesPackage{sphinx}[2018/03/28 v1.8 LaTeX package (Sphinx markup)] % provides \ltx@ifundefined % (many packages load ltxcmds: graphicx does for pdftex and lualatex but @@ -164,7 +164,6 @@ } % For highlighted code. \RequirePackage{fancyvrb} -\fvset{fontsize=\small} \define@key{FV}{hllines}{\def\sphinx@verbatim@checkifhl##1{\in@{, ##1,}{#1}}} % For hyperlinked footnotes in tables; also for gathering footnotes from % topic and warning blocks. Also to allow code-blocks in footnotes. @@ -447,7 +446,7 @@ \renewenvironment{sphinxthebibliography}[1] {\cleardoublepage% \phantomsection % not needed here since TeXLive 2010's hyperref - \begin{thebibliography}{1}} + \begin{thebibliography}{#1}} {\end{thebibliography}} \fi @@ -664,6 +663,10 @@ \belowcaptionskip\smallskipamount} +%% CITATIONS +% +\protected\def\sphinxcite{\cite} + %% FOOTNOTES % % Support large numbered footnotes in minipage @@ -1579,6 +1582,7 @@ \protected\def\sphinxtablecontinued#1{\textsf{#1}} \protected\def\sphinxtitleref#1{\emph{#1}} \protected\def\sphinxmenuselection#1{\emph{#1}} +\protected\def\sphinxguilabel#1{\emph{#1}} \protected\def\sphinxaccelerator#1{\underline{#1}} \protected\def\sphinxcrossref#1{\emph{#1}} \protected\def\sphinxtermref#1{\emph{#1}} diff --git a/sphinx/texinputs/sphinxhowto.cls b/sphinx/texinputs/sphinxhowto.cls index 11a49a205..8b0530af7 100644 --- a/sphinx/texinputs/sphinxhowto.cls +++ b/sphinx/texinputs/sphinxhowto.cls @@ -80,7 +80,7 @@ % \newenvironment{sphinxthebibliography}[1]{% % \phantomsection % not needed here since TeXLive 2010's hyperref - \begin{thebibliography}{1}% + \begin{thebibliography}{#1}% \addcontentsline{toc}{section}{\ifdefined\refname\refname\else\ifdefined\bibname\bibname\fi\fi}}{\end{thebibliography}} diff --git a/sphinx/texinputs/sphinxmanual.cls b/sphinx/texinputs/sphinxmanual.cls index 5b3d183cb..94c71d742 100644 --- a/sphinx/texinputs/sphinxmanual.cls +++ b/sphinx/texinputs/sphinxmanual.cls @@ -99,7 +99,7 @@ \newenvironment{sphinxthebibliography}[1]{% \if@openright\cleardoublepage\else\clearpage\fi % \phantomsection % not needed here since TeXLive 2010's hyperref - \begin{thebibliography}{1}% + \begin{thebibliography}{#1}% \addcontentsline{toc}{chapter}{\bibname}}{\end{thebibliography}} % Same for the indices. diff --git a/sphinx/themes/basic/layout.html b/sphinx/themes/basic/layout.html index 68c7d9e51..a25a18b90 100644 --- a/sphinx/themes/basic/layout.html +++ b/sphinx/themes/basic/layout.html @@ -97,8 +97,8 @@ <link rel="stylesheet" href="{{ pathto('_static/' + style, 1) }}" type="text/css" /> <link rel="stylesheet" href="{{ pathto('_static/pygments.css', 1) }}" type="text/css" /> {%- for css in css_files %} - {%- if css|attr("rel") %} - <link rel="{{ css.rel }}" href="{{ pathto(css.filename, 1) }}" type="text/css"{% if css.title is not none %} title="{{ css.title }}"{% endif %} /> + {%- if css|attr("filename") %} + {{ css_tag(css) }} {%- else %} <link rel="stylesheet" href="{{ pathto(css, 1) }}" type="text/css" /> {%- endif %} diff --git a/sphinx/theming.py b/sphinx/theming.py index 7a4720e0c..944c446c3 100644 --- a/sphinx/theming.py +++ b/sphinx/theming.py @@ -133,7 +133,7 @@ class Theme(object): for option, value in iteritems(overrides): if option not in options: - logger.warning('unsupported theme option %r given' % option) + logger.warning(__('unsupported theme option %r given') % option) else: options[option] = value diff --git a/sphinx/transforms/__init__.py b/sphinx/transforms/__init__.py index a2e92223d..ab69f981a 100644 --- a/sphinx/transforms/__init__.py +++ b/sphinx/transforms/__init__.py @@ -19,7 +19,7 @@ from docutils.utils import normalize_language_tag from docutils.utils.smartquotes import smartchars from sphinx import addnodes -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.docutils import new_document from sphinx.util.i18n import format_date @@ -27,6 +27,7 @@ from sphinx.util.nodes import apply_source_workaround, is_smartquotable if False: # For type annotation + from typing import Generator, List # NOQA from sphinx.application import Sphinx # NOQA from sphinx.config import Config # NOQA from sphinx.domain.std import StandardDomain # NOQA @@ -43,35 +44,28 @@ default_substitutions = set([ class SphinxTransform(Transform): - """ - A base class of Transforms. + """A base class of Transforms. Compared with ``docutils.transforms.Transform``, this class improves accessibility to Sphinx APIs. - - The subclasses can access following objects and functions: - - self.app - The application object (:class:`sphinx.application.Sphinx`) - self.config - The config object (:class:`sphinx.config.Config`) - self.env - The environment object (:class:`sphinx.environment.BuildEnvironment`) """ @property def app(self): # type: () -> Sphinx + """Reference to the :class:`.Sphinx` object.""" return self.document.settings.env.app @property def env(self): # type: () -> BuildEnvironment + """Reference to the :class:`.BuildEnvironment` object.""" return self.document.settings.env @property def config(self): # type: () -> Config + """Reference to the :class:`.Config` object.""" return self.document.settings.env.config @@ -124,7 +118,7 @@ class DefaultSubstitutions(SphinxTransform): text = self.config[refname] if refname == 'today' and not text: # special handling: can also specify a strftime format - text = format_date(self.config.today_fmt or _('%b %d, %Y'), # type: ignore + text = format_date(self.config.today_fmt or _('%b %d, %Y'), language=self.config.language) ref.replace_self(nodes.Text(text, text)) @@ -261,8 +255,8 @@ class AutoIndexUpgrader(SphinxTransform): # type: () -> None for node in self.document.traverse(addnodes.index): if 'entries' in node and any(len(entry) == 4 for entry in node['entries']): - msg = ('4 column based index found. ' - 'It might be a bug of extensions you use: %r' % node['entries']) + msg = __('4 column based index found. ' + 'It might be a bug of extensions you use: %r') % node['entries'] logger.warning(msg, location=node) for i, entry in enumerate(node['entries']): if len(entry) == 4: @@ -297,18 +291,19 @@ class UnreferencedFootnotesDetector(SphinxTransform): default_priority = 200 def apply(self): + # type: () -> None for node in self.document.footnotes: if node['names'] == []: # footnote having duplicated number. It is already warned at parser. pass elif node['names'][0] not in self.document.footnote_refs: - logger.warning('Footnote [%s] is not referenced.', node['names'][0], + logger.warning(__('Footnote [%s] is not referenced.'), node['names'][0], type='ref', subtype='footnote', location=node) for node in self.document.autofootnotes: if not any(ref['auto'] == node['auto'] for ref in self.document.autofootnote_refs): - logger.warning('Footnote [#] is not referenced.', + logger.warning(__('Footnote [#] is not referenced.'), type='ref', subtype='footnote', location=node) @@ -348,6 +343,8 @@ class SphinxSmartQuotes(SmartQuotes, SphinxTransform): refs: sphinx.parsers.RSTParser """ + default_priority = 750 + def apply(self): # type: () -> None if not self.is_available(): @@ -391,6 +388,7 @@ class SphinxSmartQuotes(SmartQuotes, SphinxTransform): return self.config.smartquotes_action def get_tokens(self, txtnodes): + # type: (List[nodes.Node]) -> Generator # A generator that yields ``(texttype, nodetext)`` tuples for a list # of "Text" nodes (interface to ``smartquotes.educate_tokens()``). @@ -401,11 +399,21 @@ class SphinxSmartQuotes(SmartQuotes, SphinxTransform): yield (texttype[notsmartquotable], txtnode.astext()) +class DoctreeReadEvent(SphinxTransform): + """Emit :event:`doctree-read` event.""" + default_priority = 880 + + def apply(self): + # type: () -> None + self.app.emit('doctree-read', self.document) + + class ManpageLink(SphinxTransform): """Find manpage section numbers and names""" default_priority = 999 def apply(self): + # type: () -> None for node in self.document.traverse(addnodes.manpage): manpage = ' '.join([str(x) for x in node.children if isinstance(x, nodes.Text)]) diff --git a/sphinx/transforms/i18n.py b/sphinx/transforms/i18n.py index 073bcb34f..397470317 100644 --- a/sphinx/transforms/i18n.py +++ b/sphinx/transforms/i18n.py @@ -17,7 +17,7 @@ from docutils.utils import relative_path from sphinx import addnodes from sphinx.domains.std import make_glossary_term, split_term_classifiers -from sphinx.locale import init as init_locale +from sphinx.locale import __, init as init_locale from sphinx.transforms import SphinxTransform from sphinx.util import split_index_msg, logging from sphinx.util.i18n import find_catalog @@ -52,7 +52,7 @@ def publish_msgstr(app, source, source_path, source_line, config, settings): from sphinx.io import SphinxI18nReader reader = SphinxI18nReader(app) reader.set_lineno_for_reporter(source_line) - parser = app.registry.create_source_parser(app, '') + parser = app.registry.create_source_parser(app, 'restructuredtext') doc = reader.read( source=StringInput(source=source, source_path=source_path), parser=parser, @@ -86,12 +86,13 @@ class Locale(SphinxTransform): def apply(self): # type: () -> None settings, source = self.document.settings, self.document['source'] + msgstr = u'' + # XXX check if this is reliable assert source.startswith(self.env.srcdir) docname = path.splitext(relative_path(path.join(self.env.srcdir, 'dummy'), source))[0] - textdomain = find_catalog(docname, - self.document.settings.gettext_compact) + textdomain = find_catalog(docname, self.config.gettext_compact) # fetch translations dirs = [path.join(self.env.srcdir, directory) @@ -102,7 +103,7 @@ class Locale(SphinxTransform): # phase1: replace reference ids with translated names for node, msg in extract_messages(self.document): - msgstr = catalog.gettext(msg) + msgstr = catalog.gettext(msg) # type: ignore # XXX add marker to untranslated parts if not msgstr or msgstr == msg or not msgstr.strip(): # as-of-yet untranslated @@ -219,7 +220,7 @@ class Locale(SphinxTransform): if node.get('translated', False): # to avoid double translation continue # skip if the node is already translated by phase1 - msgstr = catalog.gettext(msg) + msgstr = catalog.gettext(msg) # type: ignore # XXX add marker to untranslated parts if not msgstr or msgstr == msg: # as-of-yet untranslated continue @@ -291,8 +292,8 @@ class Locale(SphinxTransform): if len(old_foot_refs) != len(new_foot_refs): old_foot_ref_rawsources = [ref.rawsource for ref in old_foot_refs] new_foot_ref_rawsources = [ref.rawsource for ref in new_foot_refs] - logger.warning('inconsistent footnote references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent footnote references in translated message.' + + ' original: {0}, translated: {1}') .format(old_foot_ref_rawsources, new_foot_ref_rawsources), location=node) old_foot_namerefs = {} # type: Dict[unicode, List[nodes.footnote_reference]] @@ -331,8 +332,8 @@ class Locale(SphinxTransform): if len(old_refs) != len(new_refs): old_ref_rawsources = [ref.rawsource for ref in old_refs] new_ref_rawsources = [ref.rawsource for ref in new_refs] - logger.warning('inconsistent references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent references in translated message.' + + ' original: {0}, translated: {1}') .format(old_ref_rawsources, new_ref_rawsources), location=node) old_ref_names = [r['refname'] for r in old_refs] @@ -362,8 +363,8 @@ class Locale(SphinxTransform): if len(old_foot_refs) != len(new_foot_refs): old_foot_ref_rawsources = [ref.rawsource for ref in old_foot_refs] new_foot_ref_rawsources = [ref.rawsource for ref in new_foot_refs] - logger.warning('inconsistent footnote references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent footnote references in translated message.' + + ' original: {0}, translated: {1}') .format(old_foot_ref_rawsources, new_foot_ref_rawsources), location=node) for old in old_foot_refs: @@ -384,8 +385,8 @@ class Locale(SphinxTransform): if len(old_cite_refs) != len(new_cite_refs): old_cite_ref_rawsources = [ref.rawsource for ref in old_cite_refs] new_cite_ref_rawsources = [ref.rawsource for ref in new_cite_refs] - logger.warning('inconsistent citation references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent citation references in translated message.' + + ' original: {0}, translated: {1}') .format(old_cite_ref_rawsources, new_cite_ref_rawsources), location=node) for old in old_cite_refs: @@ -404,8 +405,8 @@ class Locale(SphinxTransform): if len(old_refs) != len(new_refs): old_ref_rawsources = [ref.rawsource for ref in old_refs] new_ref_rawsources = [ref.rawsource for ref in new_refs] - logger.warning('inconsistent term references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent term references in translated message.' + + ' original: {0}, translated: {1}') .format(old_ref_rawsources, new_ref_rawsources), location=node) @@ -457,7 +458,7 @@ class Locale(SphinxTransform): msg_parts = split_index_msg(type, msg) msgstr_parts = [] for part in msg_parts: - msgstr = catalog.gettext(part) + msgstr = catalog.gettext(part) # type: ignore if not msgstr: msgstr = part msgstr_parts.append(msgstr) @@ -469,6 +470,7 @@ class Locale(SphinxTransform): # remove translated attribute that is used for avoiding double translation. def has_translatable(node): + # type: (nodes.Node) -> bool return isinstance(node, nodes.Element) and 'translated' in node for node in self.document.traverse(has_translatable): node.delattr('translated') diff --git a/sphinx/transforms/post_transforms/__init__.py b/sphinx/transforms/post_transforms/__init__.py index 7c71e8585..6e53fec1d 100644 --- a/sphinx/transforms/post_transforms/__init__.py +++ b/sphinx/transforms/post_transforms/__init__.py @@ -156,13 +156,13 @@ class ReferencesResolver(SphinxTransform): warn = node.get('refwarn') if self.config.nitpicky: warn = True - if self.env._nitpick_ignore: + if self.config.nitpick_ignore: dtype = domain and '%s:%s' % (domain.name, typ) or typ - if (dtype, target) in self.env._nitpick_ignore: + if (dtype, target) in self.config.nitpick_ignore: warn = False # for "std" types also try without domain name if (not domain or domain.name == 'std') and \ - (typ, target) in self.env._nitpick_ignore: + (typ, target) in self.config.nitpick_ignore: warn = False if not warn: return diff --git a/sphinx/transforms/post_transforms/images.py b/sphinx/transforms/post_transforms/images.py index 6dd135e1e..a6b82f262 100644 --- a/sphinx/transforms/post_transforms/images.py +++ b/sphinx/transforms/post_transforms/images.py @@ -16,6 +16,7 @@ from math import ceil from docutils import nodes from six import text_type +from sphinx.locale import __ from sphinx.transforms import SphinxTransform from sphinx.util import epoch_to_rfc1123, rfc1123_to_epoch from sphinx.util import logging, requests @@ -85,12 +86,12 @@ class ImageDownloader(BaseImageConverter): headers = {} if os.path.exists(path): - timestamp = ceil(os.stat(path).st_mtime) + timestamp = ceil(os.stat(path).st_mtime) # type: float headers['If-Modified-Since'] = epoch_to_rfc1123(timestamp) r = requests.get(node['uri'], headers=headers) if r.status_code >= 400: - logger.warning('Could not fetch remote image: %s [%d]' % + logger.warning(__('Could not fetch remote image: %s [%d]') % (node['uri'], r.status_code)) else: self.app.env.original_image_uri[path] = node['uri'] @@ -118,7 +119,7 @@ class ImageDownloader(BaseImageConverter): node['uri'] = path self.app.env.images.add_file(self.env.docname, path) except Exception as exc: - logger.warning('Could not fetch remote image: %s [%s]' % + logger.warning(__('Could not fetch remote image: %s [%s]') % (node['uri'], text_type(exc))) @@ -139,7 +140,7 @@ class DataURIExtractor(BaseImageConverter): image = parse_data_uri(node['uri']) ext = get_image_extension(image.mimetype) if ext is None: - logger.warning('Unknown image format: %s...', node['uri'][:32], + logger.warning(__('Unknown image format: %s...'), node['uri'][:32], location=node) return @@ -164,19 +165,37 @@ def get_filename_for(filename, mimetype): class ImageConverter(BaseImageConverter): - """A base class images converter. + """A base class for image converters. - The concrete image converters should derive this class and - overrides the following methods and attributes: + An image converter is kind of Docutils transform module. It is used to + convert image files which does not supported by builder to appropriate + format for that builder. - * default_priority (if needed) - * conversion_rules - * is_available() - * convert() + For example, :py:class:`LaTeX builder <.LaTeXBuilder>` supports PDF, + PNG and JPEG as image formats. However it does not support SVG images. + For such case, to use image converters allows to embed these + unsupported images into the document. One of image converters; + :ref:`sphinx.ext.imgconverter <sphinx.ext.imgconverter>` can convert + a SVG image to PNG format using Imagemagick internally. + + There are three steps to make your custom image converter: + + 1. Make a subclass of ``ImageConverter`` class + 2. Override ``conversion_rules``, ``is_available()`` and ``convert()`` + 3. Register your image converter to Sphinx using + :py:meth:`.Sphinx.add_post_transform` """ default_priority = 200 - #: A conversion rules between two mimetypes which this converters supports + #: A conversion rules the image converter supports. + #: It is represented as a list of pair of source image format (mimetype) and + #: destination one:: + #: + #: conversion_rules = [ + #: ('image/svg+xml', 'image/png'), + #: ('image/gif', 'image/png'), + #: ('application/pdf', 'image/png'), + #: ] conversion_rules = [] # type: List[Tuple[unicode, unicode]] def __init__(self, *args, **kwargs): @@ -215,7 +234,7 @@ class ImageConverter(BaseImageConverter): def is_available(self): # type: () -> bool - """Confirms the converter is available or not.""" + """Return the image converter is available or not.""" raise NotImplementedError() def guess_mimetypes(self, node): @@ -254,7 +273,11 @@ class ImageConverter(BaseImageConverter): def convert(self, _from, _to): # type: (unicode, unicode) -> bool - """Converts the image to expected one.""" + """Convert a image file to expected format. + + *_from* is a path for source image file, and *_to* is a path for + destination file. + """ raise NotImplementedError() diff --git a/sphinx/transforms/references.py b/sphinx/transforms/references.py index affe4012b..40efbf615 100644 --- a/sphinx/transforms/references.py +++ b/sphinx/transforms/references.py @@ -11,15 +11,13 @@ from docutils import nodes from docutils.transforms.references import Substitutions +from six import itervalues from sphinx.transforms import SphinxTransform class SubstitutionDefinitionsRemover(SphinxTransform): - """Remove ``substitution_definition node from doctrees. - - .. note:: In Sphinx-1.7, this transform is only used in LaTeX builder. - """ + """Remove ``substitution_definition node from doctrees.""" # should be invoked after Substitutions process default_priority = Substitutions.default_priority + 1 @@ -28,3 +26,13 @@ class SubstitutionDefinitionsRemover(SphinxTransform): # type: () -> None for node in self.document.traverse(nodes.substitution_definition): node.parent.remove(node) + + +class SphinxDomains(SphinxTransform): + """Collect objects to Sphinx domains for cross references.""" + default_priority = 850 + + def apply(self): + # type: () -> None + for domain in itervalues(self.env.domains): + domain.process_doc(self.env, self.env.docname, self.document) diff --git a/sphinx/util/__init__.py b/sphinx/util/__init__.py index dda3fb04c..6a28432e3 100644 --- a/sphinx/util/__init__.py +++ b/sphinx/util/__init__.py @@ -18,6 +18,7 @@ import sys import tempfile import traceback import unicodedata +import warnings from codecs import BOM_UTF8 from collections import deque from datetime import datetime @@ -29,6 +30,7 @@ from six import text_type, binary_type, itervalues from six.moves import range from six.moves.urllib.parse import urlsplit, urlunsplit, quote_plus, parse_qsl, urlencode +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.errors import PycodeError, SphinxParallelError, ExtensionError from sphinx.util import logging from sphinx.util.console import strip_colors, colorize, bold, term_width_line # type: ignore @@ -172,6 +174,9 @@ def copy_static_entry(source, targetdir, builder, context={}, Handles all possible cases of files, directories and subdirectories. """ + warnings.warn('sphinx.util.copy_static_entry is deprecated for removal', + RemovedInSphinx30Warning) + if exclude_matchers: relpath = relative_path(path.join(builder.srcdir, 'dummy'), source) for matcher in exclude_matchers: @@ -609,6 +614,7 @@ def status_iterator(iterable, summary, color="darkgreen", length=0, verbosity=0, def epoch_to_rfc1123(epoch): + # type: (float) -> unicode """Convert datetime format epoch to RFC1123.""" from babel.dates import format_datetime @@ -618,10 +624,12 @@ def epoch_to_rfc1123(epoch): def rfc1123_to_epoch(rfc1123): + # type: (str) -> float return mktime(strptime(rfc1123, '%a, %d %b %Y %H:%M:%S %Z')) def xmlname_checker(): + # type: () -> Pattern # https://www.w3.org/TR/REC-xml/#NT-Name # Only Python 3.3 or newer support character code in regular expression name_start_chars = [ @@ -640,6 +648,7 @@ def xmlname_checker(): ] def convert(entries, splitter=u'|'): + # type: (Any, unicode) -> unicode results = [] for entry in entries: if isinstance(entry, list): diff --git a/sphinx/util/build_phase.py b/sphinx/util/build_phase.py new file mode 100644 index 000000000..e5a53551c --- /dev/null +++ b/sphinx/util/build_phase.py @@ -0,0 +1,24 @@ +# -*- coding: utf-8 -*- +""" + sphinx.util.build_phase + ~~~~~~~~~~~~~~~~~~~~~~~ + + Build phase of Sphinx application. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +try: + from enum import IntEnum +except ImportError: # py27 + IntEnum = object # type: ignore + + +class BuildPhase(IntEnum): + """Build phase of Sphinx application.""" + INITIALIZATION = 1 + READING = 2 + CONSISTENCY_CHECK = 3 + RESOLVING = 3 + WRITING = 4 diff --git a/sphinx/util/compat.py b/sphinx/util/compat.py index e01558077..43ced1f5e 100644 --- a/sphinx/util/compat.py +++ b/sphinx/util/compat.py @@ -8,15 +8,34 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ + from __future__ import absolute_import import sys +import warnings + +from six import string_types, iteritems +from sphinx.deprecation import RemovedInSphinx30Warning +from sphinx.util import import_object if False: # For type annotation from typing import Any, Dict # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA + + +def deprecate_source_parsers(app, config): + # type: (Sphinx, Config) -> None + if config.source_parsers: + warnings.warn('The config variable "source_parsers" is deprecated. ' + 'Please use app.add_source_parser() API instead.', + RemovedInSphinx30Warning) + for suffix, parser in iteritems(config.source_parsers): + if isinstance(parser, string_types): + parser = import_object(parser, 'source parser') # type: ignore + app.add_source_parser(suffix, parser) def register_application_for_autosummary(app): @@ -35,6 +54,7 @@ def register_application_for_autosummary(app): def setup(app): # type: (Sphinx) -> Dict[unicode, Any] + app.connect('config-inited', deprecate_source_parsers) app.connect('builder-inited', register_application_for_autosummary) return { diff --git a/sphinx/util/docfields.py b/sphinx/util/docfields.py index 2f952d7cc..202616337 100644 --- a/sphinx/util/docfields.py +++ b/sphinx/util/docfields.py @@ -288,6 +288,12 @@ class DocFieldTransformer(object): fieldtype, fieldarg = fieldname.astext(), '' typedesc, is_typefield = typemap.get(fieldtype, (None, None)) + # collect the content, trying not to keep unnecessary paragraphs + if _is_single_paragraph(fieldbody): + content = fieldbody.children[0].children + else: + content = fieldbody.children + # sort out unknown fields if typedesc is None or typedesc.has_arg != bool(fieldarg): # either the field name is unknown, or the argument doesn't @@ -297,16 +303,27 @@ class DocFieldTransformer(object): new_fieldname += ' ' + fieldarg fieldname[0] = nodes.Text(new_fieldname) entries.append(field) + + # but if this has a type then we can at least link it + if typedesc and is_typefield and content: + target = content[0].astext() + xrefs = typedesc.make_xrefs( + typedesc.typerolename, + self.directive.domain, + target, + contnode=content[0], + ) + if _is_single_paragraph(fieldbody): + fieldbody.children[0].clear() + fieldbody.children[0].extend(xrefs) + else: + fieldbody.clear() + fieldbody.extend(xrefs) + continue typename = typedesc.name - # collect the content, trying not to keep unnecessary paragraphs - if _is_single_paragraph(fieldbody): - content = fieldbody.children[0].children - else: - content = fieldbody.children - # if the field specifies a type, put it in the types collection if is_typefield: # filter out only inline nodes; others will result in invalid diff --git a/sphinx/util/docutils.py b/sphinx/util/docutils.py index 38f275824..b6262d793 100644 --- a/sphinx/util/docutils.py +++ b/sphinx/util/docutils.py @@ -10,19 +10,22 @@ """ from __future__ import absolute_import +import os import re import types +import warnings from contextlib import contextmanager from copy import copy from distutils.version import LooseVersion +from os import path import docutils from docutils import nodes -from docutils.languages import get_language -from docutils.parsers.rst import directives, roles, convert_directive_function +from docutils.parsers.rst import Directive, directives, roles, convert_directive_function from docutils.statemachine import StateMachine from docutils.utils import Reporter +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.errors import ExtensionError from sphinx.locale import __ from sphinx.util import logging @@ -32,18 +35,20 @@ report_re = re.compile('^(.+?:(?:\\d+)?): \\((DEBUG|INFO|WARNING|ERROR|SEVERE)/( if False: # For type annotation - from typing import Any, Callable, Generator, Iterator, List, Tuple # NOQA + from typing import Any, Callable, Generator, List, Set, Tuple # NOQA from docutils.statemachine import State, ViewList # NOQA + from sphinx.config import Config # NOQA from sphinx.environment import BuildEnvironment # NOQA from sphinx.io import SphinxFileInput # NOQA __version_info__ = tuple(LooseVersion(docutils.__version__).version) +additional_nodes = set() # type: Set[nodes.Node] @contextmanager def docutils_namespace(): - # type: () -> Iterator[None] + # type: () -> Generator[None, None, None] """Create namespace for reST parsers.""" try: _directives = copy(directives._directives) @@ -54,30 +59,89 @@ def docutils_namespace(): directives._directives = _directives roles._roles = _roles + for node in list(additional_nodes): + unregister_node(node) + additional_nodes.discard(node) -def patched_get_language(language_code, reporter=None): - # type: (unicode, Reporter) -> Any - """A wrapper for docutils.languages.get_language(). + +def is_node_registered(node): + # type: (nodes.Node) -> bool + """Check the *node* is already registered.""" + return hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__) + + +def register_node(node): + # type: (nodes.Node) -> None + """Register a node to docutils. + + This modifies global state of some visitors. So it is better to use this + inside ``docutils_namespace()`` to prevent side-effects. + """ + if not hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__): + nodes._add_node_class_names([node.__name__]) + additional_nodes.add(node) + + +def unregister_node(node): + # type: (nodes.Node) -> None + """Unregister a node from docutils. + + This is inverse of ``nodes._add_nodes_class_names()``. + """ + if hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__): + delattr(nodes.GenericNodeVisitor, "visit_" + node.__name__) + delattr(nodes.GenericNodeVisitor, "depart_" + node.__name__) + delattr(nodes.SparseNodeVisitor, 'visit_' + node.__name__) + delattr(nodes.SparseNodeVisitor, 'depart_' + node.__name__) + + +@contextmanager +def patched_get_language(): + # type: () -> Generator[None, None, None] + """Patch docutils.languages.get_language() temporarily. This ignores the second argument ``reporter`` to suppress warnings. refs: https://github.com/sphinx-doc/sphinx/issues/3788 """ - return get_language(language_code) + from docutils.languages import get_language + def patched_get_language(language_code, reporter=None): + # type: (unicode, Reporter) -> Any + return get_language(language_code) -@contextmanager -def patch_docutils(): - # type: () -> Iterator[None] - """Patch to docutils temporarily.""" try: docutils.languages.get_language = patched_get_language - yield finally: # restore original implementations docutils.languages.get_language = get_language +@contextmanager +def using_user_docutils_conf(confdir): + # type: (unicode) -> Generator[None, None, None] + """Let docutils know the location of ``docutils.conf`` for Sphinx.""" + try: + docutilsconfig = os.environ.get('DOCUTILSCONFIG', None) + if confdir: + os.environ['DOCUTILSCONFIG'] = path.join(path.abspath(confdir), 'docutils.conf') # type: ignore # NOQA + + yield + finally: + if docutilsconfig is None: + os.environ.pop('DOCUTILSCONFIG') + else: + os.environ['DOCUTILSCONFIG'] = docutilsconfig + + +@contextmanager +def patch_docutils(confdir=None): + # type: (unicode) -> Generator[None, None, None] + """Patch to docutils temporarily.""" + with patched_get_language(), using_user_docutils_conf(confdir): + yield + + class ElementLookupError(Exception): pass @@ -177,8 +241,9 @@ class LoggingReporter(Reporter): return cls(reporter.source, reporter.report_level, reporter.halt_level, reporter.debug_flag, reporter.error_handler) - def __init__(self, source, report_level, halt_level, - debug=False, error_handler='backslashreplace'): + def __init__(self, source, report_level=Reporter.WARNING_LEVEL, + halt_level=Reporter.SEVERE_LEVEL, debug=False, + error_handler='backslashreplace'): # type: (unicode, int, int, bool, unicode) -> None stream = WarningStream() Reporter.__init__(self, source, report_level, halt_level, @@ -200,6 +265,10 @@ def is_html5_writer_available(): def directive_helper(obj, has_content=None, argument_spec=None, **option_spec): # type: (Any, bool, Tuple[int, int, bool], Any) -> Any + warnings.warn('function based directive support is now deprecated. ' + 'Use class based directive instead.', + RemovedInSphinx30Warning) + if isinstance(obj, (types.FunctionType, types.MethodType)): obj.content = has_content # type: ignore obj.arguments = argument_spec or (0, 0, False) # type: ignore @@ -214,7 +283,7 @@ def directive_helper(obj, has_content=None, argument_spec=None, **option_spec): @contextmanager def switch_source_input(state, content): - # type: (State, ViewList) -> Generator + # type: (State, ViewList) -> Generator[None, None, None] """Switch current source input of state temporarily.""" try: # remember the original ``get_source_and_line()`` method @@ -231,6 +300,26 @@ def switch_source_input(state, content): state.memo.reporter.get_source_and_line = get_source_and_line +class SphinxDirective(Directive): + """A base class for Directives. + + Compared with ``docutils.parsers.rst.Directive``, this class improves + accessibility to Sphinx APIs. + """ + + @property + def env(self): + # type: () -> BuildEnvironment + """Reference to the :class:`.BuildEnvironment` object.""" + return self.state.document.settings.env + + @property + def config(self): + # type: () -> Config + """Reference to the :class:`.Config` object.""" + return self.env.config + + # cache a vanilla instance of nodes.document # Used in new_document() function __document_cache__ = None # type: nodes.document diff --git a/sphinx/util/fileutil.py b/sphinx/util/fileutil.py index 7caf40275..fcbc8abe6 100644 --- a/sphinx/util/fileutil.py +++ b/sphinx/util/fileutil.py @@ -74,6 +74,10 @@ def copy_asset(source, destination, excluded=lambda path: False, context=None, r if not os.path.exists(source): return + if renderer is None: + from sphinx.util.template import SphinxRenderer + renderer = SphinxRenderer() + ensuredir(destination) if os.path.isfile(source): copy_asset_file(source, destination, context, renderer) diff --git a/sphinx/util/i18n.py b/sphinx/util/i18n.py index ff7f8bd75..d18889756 100644 --- a/sphinx/util/i18n.py +++ b/sphinx/util/i18n.py @@ -21,6 +21,7 @@ from babel.messages.mofile import write_mo from babel.messages.pofile import read_po from sphinx.errors import SphinxError +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import SEP, relpath, walk @@ -68,14 +69,14 @@ class CatalogInfo(LocaleFileInfoBase): try: po = read_po(file_po, locale) except Exception as exc: - logger.warning('reading error: %s, %s', self.po_path, exc) + logger.warning(__('reading error: %s, %s'), self.po_path, exc) return with io.open(self.mo_path, 'wb') as file_mo: try: write_mo(file_mo, po) except Exception as exc: - logger.warning('writing error: %s, %s', self.mo_path, exc) + logger.warning(__('writing error: %s, %s'), self.mo_path, exc) def find_catalog(docname, compaction): @@ -208,8 +209,8 @@ def babel_format_date(date, format, locale, formatter=babel.dates.format_date): # fallback to English return formatter(date, format, locale='en') except AttributeError: - logger.warning('Invalid date format. Quote the string by single quote ' - 'if you want to output it directly: %s', format) + logger.warning(__('Invalid date format. Quote the string by single quote ' + 'if you want to output it directly: %s'), format) return format diff --git a/sphinx/util/images.py b/sphinx/util/images.py index dd2f2a9e2..9abe748e4 100644 --- a/sphinx/util/images.py +++ b/sphinx/util/images.py @@ -123,6 +123,7 @@ def parse_data_uri(uri): def test_svg(h, f): + # type: (unicode, IO) -> unicode """An additional imghdr library helper; test the header is SVG's or not.""" try: if '<svg' in h.decode('utf-8').lower(): @@ -130,6 +131,8 @@ def test_svg(h, f): except UnicodeDecodeError: pass + return None + # install test_svg() to imghdr # refs: https://docs.python.org/3.6/library/imghdr.html#imghdr.tests diff --git a/sphinx/util/inspect.py b/sphinx/util/inspect.py index 492db8700..5dab5a710 100644 --- a/sphinx/util/inspect.py +++ b/sphinx/util/inspect.py @@ -244,8 +244,19 @@ def object_description(object): except TypeError: pass # Cannot sort dict keys, fall back to generic repr else: - items = ("%r: %r" % (key, object[key]) for key in sorted_keys) + items = ("%s: %s" % + (object_description(key), object_description(object[key])) + for key in sorted_keys) return "{%s}" % ", ".join(items) + if isinstance(object, set): + try: + sorted_values = sorted(object) + except TypeError: + pass # Cannot sort set values, fall back to generic repr + else: + template = "{%s}" if PY3 else "set([%s])" + return template % ", ".join(object_description(x) + for x in sorted_values) try: s = repr(object) except Exception: @@ -583,6 +594,7 @@ else: # of Python 3.5 def _findclass(func): + # type: (Any) -> Any cls = sys.modules.get(func.__module__) if cls is None: return None @@ -596,6 +608,7 @@ else: return cls def _finddoc(obj): + # type: (Any) -> unicode if inspect.isclass(obj): for base in obj.__mro__: if base is not object: @@ -656,6 +669,7 @@ else: return None def getdoc(object): + # type: (Any) -> unicode """Get the documentation string for an object. All tabs are expanded to spaces. To clean up docstrings that are diff --git a/sphinx/util/logging.py b/sphinx/util/logging.py index 09db0028f..5034c007d 100644 --- a/sphinx/util/logging.py +++ b/sphinx/util/logging.py @@ -60,11 +60,18 @@ COLOR_MAP.update({ def getLogger(name): # type: (str) -> SphinxLoggerAdapter - """Get logger wrapped by SphinxLoggerAdapter. + """Get logger wrapped by :class:`sphinx.util.logging.SphinxLoggerAdapter`. - Sphinx logger always uses ``sphinx.*`` namesapce to be independent from - settings of root logger. It enables to log stably even if 3rd party - extension or imported application resets logger settings. + Sphinx logger always uses ``sphinx.*`` namespace to be independent from + settings of root logger. It ensures logging is consistent even if a + third-party extension or imported application resets logger settings. + + Example usage:: + + >>> from sphinx.util import logging + >>> logger = logging.getLogger(__name__) + >>> logger.info('Hello, this is an extension!') + Hello, this is an extension! """ # add sphinx prefix to name forcely logger = logging.getLogger(NAMESPACE + '.' + name) @@ -217,7 +224,10 @@ class MemoryHandler(logging.handlers.BufferingHandler): @contextmanager def pending_warnings(): # type: () -> Generator - """contextmanager to pend logging warnings temporary.""" + """Contextmanager to pend logging warnings temporary. + + Similar to :func:`pending_logging`. + """ logger = logging.getLogger(NAMESPACE) memhandler = MemoryHandler() memhandler.setLevel(logging.WARNING) @@ -243,7 +253,16 @@ def pending_warnings(): @contextmanager def pending_logging(): # type: () -> Generator - """contextmanager to pend logging all logs temporary.""" + """Contextmanager to pend logging all logs temporary. + + For example:: + + >>> with pending_logging(): + >>> logger.warning('Warning message!') # not flushed yet + >>> some_long_process() + >>> + Warning message! # the warning is flushed here + """ logger = logging.getLogger(NAMESPACE) memhandler = MemoryHandler() diff --git a/sphinx/util/nodes.py b/sphinx/util/nodes.py index 0d86c1ec0..d3441565b 100644 --- a/sphinx/util/nodes.py +++ b/sphinx/util/nodes.py @@ -16,12 +16,12 @@ from docutils import nodes from six import text_type from sphinx import addnodes -from sphinx.locale import pairindextypes +from sphinx.locale import __ from sphinx.util import logging if False: # For type annotation - from typing import Any, Callable, Iterable, List, Set, Tuple, Union # NOQA + from typing import Any, Callable, Iterable, List, Set, Tuple, Optional # NOQA from sphinx.builders import Builder # NOQA from sphinx.utils.tags import Tags # NOQA @@ -33,6 +33,36 @@ explicit_title_re = re.compile(r'^(.+?)\s*(?<!\x00)<(.*?)>$', re.DOTALL) caption_ref_re = explicit_title_re # b/w compat alias +def get_full_module_name(node): + # type: (nodes.Node) -> str + """ + return full module dotted path like: 'docutils.nodes.paragraph' + + :param nodes.Node node: target node + :return: full module dotted path + """ + return '{}.{}'.format(node.__module__, node.__class__.__name__) + + +def repr_domxml(node, length=80): + # type: (nodes.Node, Optional[int]) -> unicode + """ + return DOM XML representation of the specified node like: + '<paragraph translatable="False"><inline classes="versionmodified">New in version...' + + :param nodes.Node node: target node + :param int length: + length of return value to be striped. if false-value is specified, repr_domxml + returns full of DOM XML representation. + :return: DOM XML representation + """ + # text = node.asdom().toxml() # #4919 crush if node has secnumber with tuple value + text = text_type(node) # workaround for #4919 + if length and len(text) > length: + text = text[:length] + '...' + return text + + def apply_source_workaround(node): # type: (nodes.Node) -> None # workaround: nodes.term have wrong rawsource if classifier is specified. @@ -41,11 +71,15 @@ def apply_source_workaround(node): # * rawsource of term node will have: ``term text : classifier1 : classifier2`` # * rawsource of classifier node will be None if isinstance(node, nodes.classifier) and not node.rawsource: + logger.debug('[i18n] PATCH: %r to have source, line and rawsource: %s', + get_full_module_name(node), repr_domxml(node)) definition_list_item = node.parent node.source = definition_list_item.source node.line = definition_list_item.line - 1 node.rawsource = node.astext() # set 'classifier1' (or 'classifier2') if isinstance(node, nodes.image) and node.source is None: + logger.debug('[i18n] PATCH: %r to have source, line: %s', + get_full_module_name(node), repr_domxml(node)) node.source, node.line = node.parent.source, node.parent.line if isinstance(node, nodes.title) and node.source is None: # Uncomment these lines after merging into master(1.8) @@ -53,6 +87,8 @@ def apply_source_workaround(node): # get_full_module_name(node), repr_domxml(node)) node.source, node.line = node.parent.source, node.parent.line if isinstance(node, nodes.term): + logger.debug('[i18n] PATCH: %r to have rawsource: %s', + get_full_module_name(node), repr_domxml(node)) # strip classifier from rawsource of term for classifier in reversed(node.parent.traverse(nodes.classifier)): node.rawsource = re.sub(r'\s*:\s*%s' % re.escape(classifier.astext()), @@ -76,6 +112,8 @@ def apply_source_workaround(node): nodes.image, # #3093 image directive in substitution nodes.field_name, # #3335 field list syntax ))): + logger.debug('[i18n] PATCH: %r to have source and line: %s', + get_full_module_name(node), repr_domxml(node)) node.source = find_source_node(node) node.line = 0 # need fix docutils to get `node.line` return @@ -83,7 +121,6 @@ def apply_source_workaround(node): IGNORED_NODES = ( nodes.Invisible, - nodes.Inline, nodes.literal_block, nodes.doctest_block, addnodes.versionmodified, @@ -105,17 +142,30 @@ def is_translatable(node): if isinstance(node, addnodes.translatable): return True + if isinstance(node, nodes.Inline) and 'translatable' not in node: + # inline node must not be translated if 'translatable' is not set + return False + if isinstance(node, nodes.TextElement): if not node.source: + logger.debug('[i18n] SKIP %r because no node.source: %s', + get_full_module_name(node), repr_domxml(node)) return False # built-in message if isinstance(node, IGNORED_NODES) and 'translatable' not in node: + logger.debug("[i18n] SKIP %r because node is in IGNORED_NODES " + "and no node['translatable']: %s", + get_full_module_name(node), repr_domxml(node)) return False if not node.get('translatable', True): # not(node['translatable'] == True or node['translatable'] is None) + logger.debug("[i18n] SKIP %r because not node['translatable']: %s", + get_full_module_name(node), repr_domxml(node)) return False # <field_name>orphan</field_name> # XXX ignore all metadata (== docinfo) if isinstance(node, nodes.field_name) and node.children[0] == 'orphan': + logger.debug('[i18n] SKIP %r because orphan node: %s', + get_full_module_name(node), repr_domxml(node)) return False return True @@ -249,6 +299,8 @@ indextypes = [ def process_index_entry(entry, targetid): # type: (unicode, unicode) -> List[Tuple[unicode, unicode, unicode, unicode, unicode]] + from sphinx.domains.python import pairindextypes + indexentries = [] # type: List[Tuple[unicode, unicode, unicode, unicode, unicode]] entry = entry.strip() oentry = entry @@ -304,7 +356,7 @@ def inline_all_toctrees(builder, docnameset, docname, tree, colorfunc, traversed colorfunc, traversed) docnameset.add(includefile) except Exception: - logger.warning('toctree contains ref to nonexisting file %r', + logger.warning(__('toctree contains ref to nonexisting file %r'), includefile, location=docname) else: sof = addnodes.start_of_file(docname=includefile) @@ -377,7 +429,7 @@ def process_only_nodes(document, tags): try: ret = tags.eval_condition(node['expr']) except Exception as err: - logger.warning('exception while evaluating only directive expression: %s', err, + logger.warning(__('exception while evaluating only directive expression: %s'), err, location=node) node.replace_self(node.children or nodes.comment()) else: diff --git a/sphinx/util/osutil.py b/sphinx/util/osutil.py index fdcbda9a6..986171293 100644 --- a/sphinx/util/osutil.py +++ b/sphinx/util/osutil.py @@ -19,11 +19,14 @@ import re import shutil import sys import time +import warnings from io import BytesIO, StringIO from os import path from six import PY2, PY3, text_type +from sphinx.deprecation import RemovedInSphinx30Warning + if False: # For type annotation from typing import Any, Iterator, List, Tuple, Union # NOQA @@ -181,8 +184,10 @@ def make_filename(string): def ustrftime(format, *args): # type: (unicode, Any) -> unicode - # [DEPRECATED] strftime for unicode strings - # It will be removed at Sphinx-1.5 + """[DEPRECATED] strftime for unicode strings.""" + warnings.warn('sphinx.util.osutil.ustrtime is deprecated for removal', + RemovedInSphinx30Warning) + if not args: # If time is not specified, try to use $SOURCE_DATE_EPOCH variable # See https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal @@ -197,7 +202,7 @@ def ustrftime(format, *args): return time.strftime(text_type(format).encode(enc), *args).decode(enc) else: # Py3 # On Windows, time.strftime() and Unicode characters will raise UnicodeEncodeError. - # http://bugs.python.org/issue8304 + # https://bugs.python.org/issue8304 try: return time.strftime(format, *args) except UnicodeEncodeError: @@ -227,7 +232,12 @@ def abspath(pathdir): # type: (unicode) -> unicode pathdir = path.abspath(pathdir) if isinstance(pathdir, bytes): - pathdir = pathdir.decode(fs_encoding) + try: + pathdir = pathdir.decode(fs_encoding) + except UnicodeDecodeError: + raise UnicodeDecodeError('multibyte filename not supported on ' + 'this filesystem encoding ' + '(%r)' % fs_encoding) return pathdir diff --git a/sphinx/util/rst.py b/sphinx/util/rst.py index f39549276..0e49f991a 100644 --- a/sphinx/util/rst.py +++ b/sphinx/util/rst.py @@ -17,6 +17,7 @@ from docutils.parsers.rst import roles from docutils.parsers.rst.languages import en as english from docutils.utils import Reporter +from sphinx.locale import __ from sphinx.util import logging if False: @@ -43,7 +44,7 @@ def default_role(docname, name): if role_fn: roles._roles[''] = role_fn else: - logger.warning('default role %s not found', name, location=docname) + logger.warning(__('default role %s not found'), name, location=docname) yield diff --git a/sphinx/util/smartypants.py b/sphinx/util/smartypants.py index bca901b18..03fc1816c 100644 --- a/sphinx/util/smartypants.py +++ b/sphinx/util/smartypants.py @@ -19,8 +19,8 @@ notices and this notice are preserved. This file is offered as-is, without any warranty. - .. _SmartyPants: http://daringfireball.net/projects/smartypants/ - .. _2-Clause BSD license: http://www.spdx.org/licenses/BSD-2-Clause + .. _SmartyPants: https://daringfireball.net/projects/smartypants/ + .. _2-Clause BSD license: https://spdx.org/licenses/BSD-2-Clause See the LICENSE file and the original docutils code for details. @@ -70,7 +70,7 @@ langquotes = {'af': u'“”‘’', 'he': u'”“»«', # Hebrew is RTL, test position: 'he-x-altquot': u'„”‚’', # low quotation marks are opening. # 'he-x-altquot': u'“„‘‚', # RTL: low quotation marks opening - 'hr': u'„”‘’', # http://hrvatska-tipografija.com/polunavodnici/ + 'hr': u'„”‘’', # https://hrvatska-tipografija.com/polunavodnici/ 'hr-x-altquot': u'»«›‹', 'hsb': u'„“‚‘', 'hsb-x-altquot': u'»«›‹', diff --git a/sphinx/util/typing.py b/sphinx/util/typing.py index 3cc4a46b5..a26dac473 100644 --- a/sphinx/util/typing.py +++ b/sphinx/util/typing.py @@ -22,3 +22,6 @@ if PY3: # common role functions RoleFunction = Callable[[unicode, unicode, unicode, int, Inliner, Dict, List[unicode]], Tuple[List[nodes.Node], List[nodes.Node]]] + +# title getter functions for enumerable nodes (see sphinx.domains.std) +TitleGetter = Callable[[nodes.Node], unicode] diff --git a/sphinx/util/websupport.py b/sphinx/util/websupport.py index 59133b9e1..59496ec02 100644 --- a/sphinx/util/websupport.py +++ b/sphinx/util/websupport.py @@ -10,5 +10,8 @@ try: from sphinxcontrib.websupport.utils import is_commentable # NOQA except ImportError: + from docutils import nodes # NOQA + def is_commentable(node): + # type: (nodes.Node) -> bool raise RuntimeError diff --git a/sphinx/versioning.py b/sphinx/versioning.py index bd0928775..58b648069 100644 --- a/sphinx/versioning.py +++ b/sphinx/versioning.py @@ -9,6 +9,7 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ +import warnings from itertools import product from operator import itemgetter from uuid import uuid4 @@ -17,6 +18,7 @@ from six import iteritems from six.moves import cPickle as pickle from six.moves import range, zip_longest +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.transforms import SphinxTransform if False: @@ -155,11 +157,15 @@ def levenshtein_distance(a, b): class UIDTransform(SphinxTransform): """Add UIDs to doctree for versioning.""" - default_priority = 100 + default_priority = 880 def apply(self): + # type: () -> None env = self.env old_doctree = None + if not env.versioning_condition: + return + if env.versioning_compare: # get old doctree try: @@ -177,6 +183,9 @@ class UIDTransform(SphinxTransform): def prepare(document): + # type: (nodes.Node) -> None """Simple wrapper for UIDTransform.""" + warnings.warn('versioning.prepare() is deprecated. Use UIDTransform instead.', + RemovedInSphinx30Warning) transform = UIDTransform(document) transform.apply() diff --git a/sphinx/writers/html.py b/sphinx/writers/html.py index 2aef7833e..0586636df 100644 --- a/sphinx/writers/html.py +++ b/sphinx/writers/html.py @@ -19,7 +19,7 @@ from docutils.writers.html4css1 import Writer, HTMLTranslator as BaseTranslator from six import string_types from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.images import get_image_size @@ -335,17 +335,17 @@ class HTMLTranslator(BaseTranslator): self.body.append('<span class="caption-number">') prefix = self.builder.config.numfig_format.get(figtype) if prefix is None: - msg = 'numfig_format is not defined for %s' % figtype + msg = __('numfig_format is not defined for %s') % figtype logger.warning(msg) else: numbers = self.builder.fignumbers[key][figure_id] self.body.append(prefix % '.'.join(map(str, numbers)) + ' ') self.body.append('</span>') - figtype = self.builder.env.domains['std'].get_figtype(node) # type: ignore + figtype = self.builder.env.domains['std'].get_enumerable_node_type(node) if figtype: if len(node['ids']) == 0: - msg = 'Any IDs not assigned for %s node' % node.tagname + msg = __('Any IDs not assigned for %s node') % node.tagname logger.warning(msg, location=node) else: append_fignumber(figtype, node['ids'][0]) @@ -625,7 +625,7 @@ class HTMLTranslator(BaseTranslator): if not ('width' in node and 'height' in node): size = get_image_size(os.path.join(self.builder.srcdir, olduri)) if size is None: - logger.warning('Could not obtain image size. :scale: option is ignored.', + logger.warning(__('Could not obtain image size. :scale: option is ignored.'), # NOQA location=node) else: if 'width' not in node: @@ -689,6 +689,7 @@ class HTMLTranslator(BaseTranslator): self.body.append('</td>') def visit_option_group(self, node): + # type: (nodes.Node) -> None BaseTranslator.visit_option_group(self, node) self.context[-2] = self.context[-2].replace(' ', ' ') @@ -845,6 +846,7 @@ class HTMLTranslator(BaseTranslator): node.column = 0 def visit_entry(self, node): + # type: (nodes.Node) -> None BaseTranslator.visit_entry(self, node) if self.body[-1] == ' ': self.body[-1] = ' ' @@ -864,6 +866,7 @@ class HTMLTranslator(BaseTranslator): self.body.append(self.starttag(node, 'tr', '', CLASS='field')) def visit_field_name(self, node): + # type: (nodes.Node) -> None context_count = len(self.context) BaseTranslator.visit_field_name(self, node) if context_count != len(self.context): @@ -871,9 +874,9 @@ class HTMLTranslator(BaseTranslator): def visit_math(self, node, math_env=''): # type: (nodes.Node, unicode) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/en/master/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/en/master/ext/math.html'), location=(self.builder.current_docname, node.line)) raise nodes.SkipNode diff --git a/sphinx/writers/html5.py b/sphinx/writers/html5.py index fcf1e8875..85a7ef7c2 100644 --- a/sphinx/writers/html5.py +++ b/sphinx/writers/html5.py @@ -18,7 +18,7 @@ from docutils.writers.html5_polyglot import HTMLTranslator as BaseTranslator from six import string_types from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.images import get_image_size @@ -303,17 +303,17 @@ class HTML5Translator(BaseTranslator): self.body.append('<span class="caption-number">') prefix = self.builder.config.numfig_format.get(figtype) if prefix is None: - msg = 'numfig_format is not defined for %s' % figtype + msg = __('numfig_format is not defined for %s') % figtype logger.warning(msg) else: numbers = self.builder.fignumbers[key][figure_id] self.body.append(prefix % '.'.join(map(str, numbers)) + ' ') self.body.append('</span>') - figtype = self.builder.env.domains['std'].get_figtype(node) # type: ignore + figtype = self.builder.env.domains['std'].get_enumerable_node_type(node) if figtype: if len(node['ids']) == 0: - msg = 'Any IDs not assigned for %s node' % node.tagname + msg = __('Any IDs not assigned for %s node') % node.tagname logger.warning(msg, location=node) else: append_fignumber(figtype, node['ids'][0]) @@ -571,7 +571,7 @@ class HTML5Translator(BaseTranslator): if not ('width' in node and 'height' in node): size = get_image_size(os.path.join(self.builder.srcdir, olduri)) if size is None: - logger.warning('Could not obtain image size. :scale: option is ignored.', + logger.warning(__('Could not obtain image size. :scale: option is ignored.'), # NOQA location=node) else: if 'width' not in node: @@ -825,9 +825,9 @@ class HTML5Translator(BaseTranslator): def visit_math(self, node, math_env=''): # type: (nodes.Node, unicode) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/en/master/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/en/master/ext/math.html'), location=(self.builder.current_docname, node.line)) raise nodes.SkipNode diff --git a/sphinx/writers/latex.py b/sphinx/writers/latex.py index 17eabc462..6c4b66881 100644 --- a/sphinx/writers/latex.py +++ b/sphinx/writers/latex.py @@ -14,6 +14,7 @@ import re import sys +import warnings from collections import defaultdict from os import path @@ -23,8 +24,10 @@ from six import itervalues, text_type from sphinx import addnodes from sphinx import highlighting +from sphinx.builders.latex.nodes import footnotetext +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.errors import SphinxError -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import split_into, logging from sphinx.util.i18n import format_date from sphinx.util.nodes import clean_astext @@ -46,6 +49,7 @@ BEGIN_DOC = r''' ''' +MAX_CITATION_LABEL_LENGTH = 8 LATEXSECTIONNAMES = ["part", "chapter", "section", "subsection", "subsubsection", "paragraph", "subparagraph"] @@ -59,6 +63,7 @@ DEFAULT_SETTINGS = { 'maxlistdepth': '', 'sphinxpkgoptions': '', 'sphinxsetup': '', + 'fvset': '\\fvset{fontsize=\\small}', 'passoptionstopackages': '', 'geometry': '\\usepackage{geometry}', 'inputenc': '', @@ -130,6 +135,7 @@ ADDITIONAL_SETTINGS = { 'fontpkg': '', 'utf8extra': ('\\catcode`^^^^00a0\\active\\protected\\def^^^^00a0' '{\\leavevmode\\nobreak\\ }'), + 'fvset': '\\fvset{fontsize=auto}', }, 'lualatex': { 'latex_engine': 'lualatex', @@ -139,6 +145,7 @@ ADDITIONAL_SETTINGS = { 'fontpkg': '', 'utf8extra': ('\\catcode`^^^^00a0\\active\\protected\\def^^^^00a0' '{\\leavevmode\\nobreak\\ }'), + 'fvset': '\\fvset{fontsize=auto}', }, 'platex': { 'latex_engine': 'platex', @@ -175,8 +182,6 @@ class LaTeXWriter(writers.Writer): def translate(self): # type: () -> None - transform = ShowUrlsTransform(self.document) - transform.apply() visitor = self.builder.create_translator(self.document, self.builder) self.document.walkabout(visitor) self.output = visitor.astext() @@ -185,43 +190,51 @@ class LaTeXWriter(writers.Writer): # Helper classes class ExtBabel(Babel): + cyrillic_languages = ('bulgarian', 'kazakh', 'mongolian', 'russian', 'ukrainian') + shorthands = { + 'ngerman': '"', + 'slovene': '"', + 'portuges': '"', + 'spanish': '"', + 'dutch': '"', + 'polish': '"', + 'italian': '"', + } + def __init__(self, language_code, use_polyglossia=False): # type: (unicode, bool) -> None - super(ExtBabel, self).__init__(language_code or '') self.language_code = language_code self.use_polyglossia = use_polyglossia + self.supported = True + super(ExtBabel, self).__init__(language_code or '') def get_shorthandoff(self): # type: () -> unicode - shortlang = self.language.split('_')[0] - if shortlang in ('de', 'ngerman', 'sl', 'slovene', 'pt', 'portuges', - 'es', 'spanish', 'nl', 'dutch', 'pl', 'polish', 'it', - 'italian'): - return '\\ifnum\\catcode`\\"=\\active\\shorthandoff{"}\\fi' - elif shortlang in ('tr', 'turkish'): + shorthand = self.shorthands.get(self.language) + if shorthand: + return r'\ifnum\catcode`\%s=\active\shorthandoff{%s}\fi' % (shorthand, shorthand) + elif self.language == 'turkish': # memo: if ever Sphinx starts supporting 'Latin', do as for Turkish - return '\\ifnum\\catcode`\\=\\string=\\active\\shorthandoff{=}\\fi' + return r'\ifnum\catcode`\=\string=\active\shorthandoff{=}\fi' return '' def uses_cyrillic(self): # type: () -> bool - shortlang = self.language.split('_')[0] - return shortlang in ('bg', 'bulgarian', 'kk', 'kazakh', - 'mn', 'mongolian', 'ru', 'russian', - 'uk', 'ukrainian') + return self.language in self.cyrillic_languages def is_supported_language(self): # type: () -> bool - return bool(super(ExtBabel, self).get_language()) + return self.supported - def get_language(self): - # type: () -> unicode - language = super(ExtBabel, self).get_language() + def language_name(self, language_code): + # type: (unicode) -> unicode + language = super(ExtBabel, self).language_name(language_code) if language == 'ngerman' and self.use_polyglossia: # polyglossia calls new orthography (Neue Rechtschreibung) as # german (with new spelling option). return 'german' elif not language: + self.supported = False return 'english' # fallback to english else: return language @@ -229,13 +242,14 @@ class ExtBabel(Babel): def get_mainlanguage_options(self): # type: () -> unicode """Return options for polyglossia's ``\setmainlanguage``.""" - language = super(ExtBabel, self).get_language() if self.use_polyglossia is False: return None - elif language == 'ngerman': - return 'spelling=new' - elif language == 'german': - return 'spelling=old' + elif self.language == 'german': + language = super(ExtBabel, self).language_name(self.language_code) + if language == 'ngerman': + return 'spelling=new' + else: + return 'spelling=old' else: return None @@ -255,8 +269,6 @@ class Table(object): self.has_oldproblematic = False self.has_verbatim = False self.caption = None # type: List[unicode] - self.caption_footnotetexts = [] # type: List[unicode] - self.header_footnotetexts = [] # type: List[unicode] self.stubs = [] # type: List[int] # current position @@ -270,6 +282,20 @@ class Table(object): # (cell = rectangular area) self.cell_id = 0 # last assigned cell_id + @property + def caption_footnotetexts(self): + # type: () -> List[unicode] + warnings.warn('table.caption_footnotetexts is deprecated.', + RemovedInSphinx30Warning) + return [] + + @property + def header_footnotetexts(self): + # type: () -> List[unicode] + warnings.warn('table.header_footnotetexts is deprecated.', + RemovedInSphinx30Warning) + return [] + def is_longtable(self): # type: () -> bool """True if and only if table uses longtable environment.""" @@ -484,13 +510,13 @@ class LaTeXTranslator(nodes.NodeVisitor): self.top_sectionlevel = \ self.sectionnames.index(builder.config.latex_toplevel_sectioning) except ValueError: - logger.warning('unknown %r toplevel_sectioning for class %r' % + logger.warning(__('unknown %r toplevel_sectioning for class %r') % (builder.config.latex_toplevel_sectioning, docclass)) if builder.config.today: self.elements['date'] = builder.config.today else: - self.elements['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), # type: ignore # NOQA + self.elements['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), language=builder.config.language) if builder.config.numfig: @@ -535,7 +561,7 @@ class LaTeXTranslator(nodes.NodeVisitor): if builder.config.language and not self.babel.is_supported_language(): # emit warning if specified language is invalid # (only emitting, nothing changed to processing) - logger.warning('no Babel option known for language %r', + logger.warning(__('no Babel option known for language %r'), builder.config.language) # simply use babel.get_language() always, as get_language() returns @@ -598,7 +624,7 @@ class LaTeXTranslator(nodes.NodeVisitor): self.top_sectionlevel > 0: tocdepth += 1 # because top_sectionlevel is shifted by -1 if tocdepth > len(LATEXSECTIONNAMES) - 2: # default is 5 <-> subparagraph - logger.warning('too large :maxdepth:, ignored.') + logger.warning(__('too large :maxdepth:, ignored.')) tocdepth = len(LATEXSECTIONNAMES) - 2 self.elements['tocdepth'] = '\\setcounter{tocdepth}{%d}' % tocdepth @@ -644,7 +670,6 @@ class LaTeXTranslator(nodes.NodeVisitor): builder.config.pygments_style, builder.config.trim_doctest_flags) self.context = [] # type: List[Any] self.descstack = [] # type: List[unicode] - self.bibitems = [] # type: List[List[unicode]] self.table = None # type: Table self.next_table_colspec = None # type: unicode # stack of [language, linenothreshold] settings per file @@ -654,7 +679,6 @@ class LaTeXTranslator(nodes.NodeVisitor): self.hlsettingstack = 2 * [[builder.config.highlight_language, sys.maxsize]] self.bodystack = [] # type: List[List[unicode]] - self.footnotestack = [] # type: List[Dict[unicode, List[Union[collected_footnote, bool]]]] # NOQA self.footnote_restricted = False self.pending_footnotes = [] # type: List[nodes.footnote_reference] self.curfilestack = [] # type: List[unicode] @@ -686,17 +710,23 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: () -> None for key in self.builder.config.latex_elements: if key not in self.elements: - msg = _("Unknown configure key: latex_elements[%r] is ignored.") + msg = __("Unknown configure key: latex_elements[%r] is ignored.") logger.warning(msg % key) def restrict_footnote(self, node): # type: (nodes.Node) -> None + warnings.warn('LaTeXWriter.restrict_footnote() is deprecated.', + RemovedInSphinx30Warning) + if self.footnote_restricted is False: self.footnote_restricted = node self.pending_footnotes = [] def unrestrict_footnote(self, node): # type: (nodes.Node) -> None + warnings.warn('LaTeXWriter.unrestrict_footnote() is deprecated.', + RemovedInSphinx30Warning) + if self.footnote_restricted == node: self.footnote_restricted = False for footnode in self.pending_footnotes: @@ -704,6 +734,13 @@ class LaTeXTranslator(nodes.NodeVisitor): footnode.walkabout(self) self.pending_footnotes = [] + @property + def footnotestack(self): + # type: () -> List[Dict[unicode, List[Union[collected_footnote, bool]]]] + warnings.warn('LaTeXWriter.footnotestack is deprecated.', + RemovedInSphinx30Warning) + return [] + def format_docclass(self, docclass): # type: (unicode) -> unicode """ prepends prefix to sphinx document classes @@ -858,7 +895,6 @@ class LaTeXTranslator(nodes.NodeVisitor): def visit_document(self, node): # type: (nodes.Node) -> None - self.footnotestack.append(self.collect_footnotes(node)) self.curfilestack.append(node.get('docname', '')) if self.first_document == 1: # the first document is all the regular content ... @@ -875,25 +911,10 @@ class LaTeXTranslator(nodes.NodeVisitor): def depart_document(self, node): # type: (nodes.Node) -> None - if self.bibitems: - widest_label = "" # type: unicode - for bi in self.bibitems: - if len(widest_label) < len(bi[0]): - widest_label = bi[0] - self.body.append(u'\n\\begin{sphinxthebibliography}{%s}\n' % widest_label) - for bi in self.bibitems: - target = self.hypertarget(bi[2] + ':' + bi[3], - withdoc=False) - self.body.append(u'\\bibitem[%s]{%s}{%s %s}\n' % - (self.encode(bi[0]), self.idescape(bi[0]), - target, bi[1])) - self.body.append(u'\\end{sphinxthebibliography}\n') - self.bibitems = [] + pass def visit_start_of_file(self, node): # type: (nodes.Node) -> None - # collect new footnotes - self.footnotestack.append(self.collect_footnotes(node)) # also add a document target self.next_section_ids.add(':doc') self.curfilestack.append(node['docname']) @@ -922,7 +943,6 @@ class LaTeXTranslator(nodes.NodeVisitor): def depart_start_of_file(self, node): # type: (nodes.Node) -> None - self.footnotestack.pop() self.curfilestack.pop() self.hlsettingstack.pop() @@ -1013,7 +1033,7 @@ class LaTeXTranslator(nodes.NodeVisitor): if self.this_is_the_title: if len(node.children) != 1 and not isinstance(node.children[0], nodes.Text): - logger.warning('document title is not a single Text node', + logger.warning(__('document title is not a single Text node'), location=(self.curfilestack[-1], node.line)) if not self.elements['title']: # text needs to be escaped since it is inserted into @@ -1034,7 +1054,6 @@ class LaTeXTranslator(nodes.NodeVisitor): self.body.append(r'\%s%s{' % (self.sectionnames[-1], short)) self.context.append('}\n') - self.restrict_footnote(node) if self.next_section_ids: for id in self.next_section_ids: self.context[-1] += self.hypertarget(id, anchor=False) @@ -1051,10 +1070,9 @@ class LaTeXTranslator(nodes.NodeVisitor): elif isinstance(parent, nodes.table): # Redirect body output until title is finished. self.pushbody([]) - self.restrict_footnote(node) else: - logger.warning('encountered title node not in section, topic, table, ' - 'admonition or sidebar', + logger.warning(__('encountered title node not in section, topic, table, ' + 'admonition or sidebar'), location=(self.curfilestack[-1], node.line or '')) self.body.append('\\sphinxstyleothertitle{') self.context.append('}\n') @@ -1065,14 +1083,8 @@ class LaTeXTranslator(nodes.NodeVisitor): self.in_title = 0 if isinstance(node.parent, nodes.table): self.table.caption = self.popbody() - # temporary buffer for footnotes from caption - self.pushbody([]) - self.unrestrict_footnote(node) - # the footnote texts from caption - self.table.caption_footnotetexts = self.popbody() else: self.body.append(self.context.pop()) - self.unrestrict_footnote(node) def visit_subtitle(self, node): # type: (nodes.Node) -> None @@ -1249,39 +1261,23 @@ class LaTeXTranslator(nodes.NodeVisitor): def visit_footnote(self, node): # type: (nodes.Node) -> None - raise nodes.SkipNode - - def visit_collected_footnote(self, node): - # type: (nodes.Node) -> None self.in_footnote += 1 - if 'footnotetext' in node: - self.body.append('%%\n\\begin{footnotetext}[%s]' - '\\sphinxAtStartFootnote\n' % node['number']) + if self.in_parsed_literal: + self.body.append('\\begin{footnote}[%s]' % node[0].astext()) else: - if self.in_parsed_literal: - self.body.append('\\begin{footnote}[%s]' % node['number']) - else: - self.body.append('%%\n\\begin{footnote}[%s]' % node['number']) - self.body.append('\\sphinxAtStartFootnote\n') + self.body.append('%%\n\\begin{footnote}[%s]' % node[0].astext()) + self.body.append('\\sphinxAtStartFootnote\n') - def depart_collected_footnote(self, node): + def depart_footnote(self, node): # type: (nodes.Node) -> None - if 'footnotetext' in node: - # the \ignorespaces in particular for after table header use - self.body.append('%\n\\end{footnotetext}\\ignorespaces ') + if self.in_parsed_literal: + self.body.append('\\end{footnote}') else: - if self.in_parsed_literal: - self.body.append('\\end{footnote}') - else: - self.body.append('%\n\\end{footnote}') + self.body.append('%\n\\end{footnote}') self.in_footnote -= 1 def visit_label(self, node): # type: (nodes.Node) -> None - if isinstance(node.parent, nodes.citation): - self.bibitems[-1][0] = node.astext() - self.bibitems[-1][2] = self.curfilestack[-1] - self.bibitems[-1][3] = node.parent['ids'][0] raise nodes.SkipNode def visit_tabular_col_spec(self, node): @@ -1344,25 +1340,15 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None # Redirect head output until header is finished. self.pushbody(self.table.header) - # footnotes in longtable header must be restricted - self.restrict_footnote(node) def depart_thead(self, node): # type: (nodes.Node) -> None self.popbody() - # temporary buffer for footnotes from table header - self.pushbody([]) - self.unrestrict_footnote(node) - # the footnote texts from header - self.table.header_footnotetexts = self.popbody() def visit_tbody(self, node): # type: (nodes.Node) -> None # Redirect body output until table is finished. self.pushbody(self.table.body) - # insert footnotetexts from header at start of body (due to longtable) - # those from caption are handled by templates (to allow caption at foot) - self.body.extend(self.table.header_footnotetexts) def depart_tbody(self, node): # type: (nodes.Node) -> None @@ -1558,13 +1544,11 @@ class LaTeXTranslator(nodes.NodeVisitor): if node.get('ids'): ctx += self.hypertarget(node['ids'][0]) self.body.append('\\item[{') - self.restrict_footnote(node) self.context.append(ctx) def depart_term(self, node): # type: (nodes.Node) -> None self.body.append(self.context.pop()) - self.unrestrict_footnote(node) self.in_term -= 1 def visit_classifier(self, node): @@ -1615,8 +1599,9 @@ class LaTeXTranslator(nodes.NodeVisitor): not isinstance(node.parent[index - 1], nodes.compound)): # insert blank line, if the paragraph follows a non-paragraph node in a compound self.body.append('\\noindent\n') - elif index == 0 and isinstance(node.parent, nodes.footnote): - # don't insert blank line, if the paragraph is first child of a footnote + elif index == 1 and isinstance(node.parent, (nodes.footnote, footnotetext)): + # don't insert blank line, if the paragraph is second child of a footnote + # (first one is label node) pass else: self.body.append('\n') @@ -1663,7 +1648,7 @@ class LaTeXTranslator(nodes.NodeVisitor): try: return rstdim_to_latexdim(width_str) except ValueError: - logger.warning('dimension unit %s is invalid. Ignored.', width_str) + logger.warning(__('dimension unit %s is invalid. Ignored.'), width_str) return None def is_inline(self, node): @@ -1707,7 +1692,7 @@ class LaTeXTranslator(nodes.NodeVisitor): (0, 'center'): ('{\\hspace*{\\fill}', '\\hspace*{\\fill}}'), # These 2 don't exactly do the right thing. The image should # be floated alongside the paragraph. See - # http://www.w3.org/TR/html4/struct/objects.html#adef-align-IMG + # https://www.w3.org/TR/html4/struct/objects.html#adef-align-IMG (0, 'left'): ('{', '\\hspace*{\\fill}}'), (0, 'right'): ('{\\hspace*{\\fill}', '}'), } @@ -1759,7 +1744,6 @@ class LaTeXTranslator(nodes.NodeVisitor): ids += self.hypertarget(id, anchor=False) if node['ids']: ids += self.hypertarget(node['ids'][0], anchor=False) - self.restrict_footnote(node) if (len(node.children) and isinstance(node.children[0], nodes.image) and node.children[0]['ids']): @@ -1798,12 +1782,10 @@ class LaTeXTranslator(nodes.NodeVisitor): def depart_figure(self, node): # type: (nodes.Node) -> None self.body.append(self.context.pop()) - self.unrestrict_footnote(node) def visit_caption(self, node): # type: (nodes.Node) -> None self.in_caption += 1 - self.restrict_footnote(node) if self.in_container_literal_block: self.body.append('\\sphinxSetupCaptionForVerbatim{') elif self.in_minipage and isinstance(node.parent, nodes.figure): @@ -1817,12 +1799,13 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None self.body.append('}') self.in_caption -= 1 - self.unrestrict_footnote(node) def visit_legend(self, node): + # type: (nodes.Node) -> None self.body.append('\n\\begin{sphinxlegend}') def depart_legend(self, node): + # type: (nodes.Node) -> None self.body.append('\\end{sphinxlegend}\n') def visit_admonition(self, node): @@ -1915,7 +1898,7 @@ class LaTeXTranslator(nodes.NodeVisitor): return else: domain = self.builder.env.get_domain('std') - figtype = domain.get_figtype(next) + figtype = domain.get_enumerable_node_type(next) if figtype and domain.get_numfig_title(next): ids = set() # labels for figures go in the figure body, not before @@ -1977,7 +1960,7 @@ class LaTeXTranslator(nodes.NodeVisitor): p1, p2 = [self.encode(x) for x in split_into(2, 'seealso', string)] self.body.append(r'\index{%s|see{%s}}' % (p1, p2)) else: - logger.warning('unknown index entry type %s found', type) + logger.warning(__('unknown index entry type %s found'), type) except ValueError as err: logger.warning(str(err)) if not node.get('inline', True): @@ -2158,26 +2141,41 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None self.body.append('}') + def visit_thebibliography(self, node): + # type: (nodes.Node) -> None + longest_label = max((subnode[0].astext() for subnode in node), key=len) + if len(longest_label) > MAX_CITATION_LABEL_LENGTH: + # adjust max width of citation labels not to break the layout + longest_label = longest_label[:MAX_CITATION_LABEL_LENGTH] + + self.body.append(u'\n\\begin{sphinxthebibliography}{%s}\n' % + self.encode(longest_label)) + + def depart_thebibliography(self, node): + # type: (nodes.Node) -> None + self.body.append(u'\\end{sphinxthebibliography}\n') + def visit_citation(self, node): # type: (nodes.Node) -> None - # TODO maybe use cite bibitems - # bibitem: [citelabel, citetext, docname, citeid] - self.bibitems.append(['', '', '', '']) - self.context.append(len(self.body)) + label = node[0].astext() + self.body.append(u'\\bibitem[%s]{%s:%s}' % + (self.encode(label), node['docname'], node['ids'][0])) def depart_citation(self, node): # type: (nodes.Node) -> None - size = self.context.pop() - text = ''.join(self.body[size:]) - del self.body[size:] - self.bibitems[-1][1] = text + pass def visit_citation_reference(self, node): # type: (nodes.Node) -> None - # This is currently never encountered, since citation_reference nodes - # are already replaced by pending_xref nodes in the environment. - self.body.append('\\cite{%s}' % self.idescape(node.astext())) - raise nodes.SkipNode + if self.in_title: + pass + else: + self.body.append('\\sphinxcite{%s:%s}' % (node['docname'], node['refname'])) + raise nodes.SkipNode + + def depart_citation_reference(self, node): + # type: (nodes.Node) -> None + pass def visit_literal(self, node): # type: (nodes.Node) -> None @@ -2194,27 +2192,26 @@ class LaTeXTranslator(nodes.NodeVisitor): def visit_footnote_reference(self, node): # type: (nodes.Node) -> None - num = node.astext().strip() - try: - footnode, used = self.footnotestack[-1][num] - except (KeyError, IndexError): - raise nodes.SkipNode - # if a footnote has been inserted once, it shouldn't be repeated - # by the next reference - if used: - self.body.append('\\sphinxfootnotemark[%s]' % num) - elif self.footnote_restricted: - self.footnotestack[-1][num][1] = True - self.body.append('\\sphinxfootnotemark[%s]' % num) - self.pending_footnotes.append(footnode) - else: - self.footnotestack[-1][num][1] = True - footnode.walkabout(self) # type: ignore - raise nodes.SkipChildren + raise nodes.SkipNode - def depart_footnote_reference(self, node): + def visit_footnotemark(self, node): # type: (nodes.Node) -> None - pass + self.body.append('\\sphinxfootnotemark[') + + def depart_footnotemark(self, node): + # type: (nodes.Node) -> None + self.body.append(']') + + def visit_footnotetext(self, node): + # type: (nodes.Node) -> None + number = node[0].astext() + self.body.append('%%\n\\begin{footnotetext}[%s]' + '\\sphinxAtStartFootnote\n' % number) + + def depart_footnotetext(self, node): + # type: (nodes.Node) -> None + # the \ignorespaces in particular for after table header use + self.body.append('%\n\\end{footnotetext}\\ignorespaces ') def visit_literal_block(self, node): # type: (nodes.Node) -> None @@ -2418,20 +2415,15 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None self.body.append('}}$') - def visit_substitution_definition(self, node): - # type: (nodes.Node) -> None - raise nodes.SkipNode - - def visit_substitution_reference(self, node): - # type: (nodes.Node) -> None - raise nodes.SkipNode - def visit_inline(self, node): # type: (nodes.Node) -> None classes = node.get('classes', []) - if classes in [['menuselection'], ['guilabel']]: + if classes in [['menuselection']]: self.body.append(r'\sphinxmenuselection{') self.context.append('}') + elif classes in [['guilabel']]: + self.body.append(r'\sphinxguilabel{') + self.context.append('}') elif classes in [['accelerator']]: self.body.append(r'\sphinxaccelerator{') self.context.append('}') @@ -2549,9 +2541,9 @@ class LaTeXTranslator(nodes.NodeVisitor): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/en/master/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/en/master/ext/math.html'), location=(self.curfilestack[-1], node.line)) raise nodes.SkipNode @@ -2561,6 +2553,15 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None raise NotImplementedError('Unknown node: ' + node.__class__.__name__) + # --------- METHODS FOR COMPATIBILITY -------------------------------------- + + @property + def bibitems(self): + # type: () -> List[List[unicode]] + warnings.warn('LaTeXTranslator.bibitems() is deprecated.', + RemovedInSphinx30Warning) + return [] + # Import old modules here for compatibility # They should be imported after `LaTeXTranslator` to avoid recursive import. diff --git a/sphinx/writers/manpage.py b/sphinx/writers/manpage.py index 3f58fc31d..c6c8723dd 100644 --- a/sphinx/writers/manpage.py +++ b/sphinx/writers/manpage.py @@ -18,7 +18,7 @@ from docutils.writers.manpage import ( import sphinx.util.docutils from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.i18n import format_date @@ -107,7 +107,7 @@ class ManualPageTranslator(BaseTranslator): if builder.config.today: self._docinfo['date'] = builder.config.today else: - self._docinfo['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), # type: ignore # NOQA + self._docinfo['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), language=builder.config.language) self._docinfo['copyright'] = builder.config.copyright self._docinfo['version'] = builder.config.version @@ -513,9 +513,9 @@ class ManualPageTranslator(BaseTranslator): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/en/master/ext/math.html') + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/en/master/ext/math.html')) raise nodes.SkipNode visit_math_block = visit_math diff --git a/sphinx/writers/texinfo.py b/sphinx/writers/texinfo.py index 051ce9de6..e58d659ab 100644 --- a/sphinx/writers/texinfo.py +++ b/sphinx/writers/texinfo.py @@ -19,7 +19,7 @@ from six.moves import range from sphinx import addnodes, __display_version__ from sphinx.errors import ExtensionError -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.i18n import format_date from sphinx.writers.latex import collected_footnote @@ -237,7 +237,7 @@ class TexinfoTranslator(nodes.NodeVisitor): 'project': self.escape(self.builder.config.project), 'copyright': self.escape(self.builder.config.copyright), 'date': self.escape(self.builder.config.today or - format_date(self.builder.config.today_fmt or _('%b %d, %Y'), # type: ignore # NOQA + format_date(self.builder.config.today_fmt or _('%b %d, %Y'), language=self.builder.config.language)) }) # title @@ -648,8 +648,8 @@ class TexinfoTranslator(nodes.NodeVisitor): if isinstance(parent, (nodes.Admonition, nodes.sidebar, nodes.topic)): raise nodes.SkipNode elif not isinstance(parent, nodes.section): - logger.warning('encountered title node not in section, topic, table, ' - 'admonition or sidebar', + logger.warning(__('encountered title node not in section, topic, table, ' + 'admonition or sidebar'), location=(self.curfilestack[-1], node.line)) self.visit_rubric(node) else: @@ -1318,7 +1318,7 @@ class TexinfoTranslator(nodes.NodeVisitor): node.parent.get('literal_block'))): self.body.append('\n@caption{') else: - logger.warning('caption not inside a figure.', + logger.warning(__('caption not inside a figure.'), location=(self.curfilestack[-1], node.line)) def depart_caption(self, node): @@ -1385,18 +1385,6 @@ class TexinfoTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None pass - def visit_substitution_reference(self, node): - # type: (nodes.Node) -> None - pass - - def depart_substitution_reference(self, node): - # type: (nodes.Node) -> None - pass - - def visit_substitution_definition(self, node): - # type: (nodes.Node) -> None - raise nodes.SkipNode - def visit_system_message(self, node): # type: (nodes.Node) -> None self.body.append('\n@verbatim\n' @@ -1421,12 +1409,12 @@ class TexinfoTranslator(nodes.NodeVisitor): def unimplemented_visit(self, node): # type: (nodes.Node) -> None - logger.warning("unimplemented node type: %r", node, + logger.warning(__("unimplemented node type: %r"), node, location=(self.curfilestack[-1], node.line)) def unknown_visit(self, node): # type: (nodes.Node) -> None - logger.warning("unknown node type: %r", node, + logger.warning(__("unknown node type: %r"), node, location=(self.curfilestack[-1], node.line)) def unknown_departure(self, node): @@ -1743,9 +1731,9 @@ class TexinfoTranslator(nodes.NodeVisitor): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/en/master/ext/math.html') + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/en/master/ext/math.html')) raise nodes.SkipNode visit_math_block = visit_math diff --git a/sphinx/writers/text.py b/sphinx/writers/text.py index d6a72ad58..379f06b46 100644 --- a/sphinx/writers/text.py +++ b/sphinx/writers/text.py @@ -18,7 +18,7 @@ from docutils.utils import column_width from six.moves import zip_longest from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging if False: @@ -987,10 +987,6 @@ class TextTranslator(nodes.NodeVisitor): # type: (nodes.Node) -> None raise nodes.SkipNode - def visit_substitution_definition(self, node): - # type: (nodes.Node) -> None - raise nodes.SkipNode - def visit_pending_xref(self, node): # type: (nodes.Node) -> None pass @@ -1183,9 +1179,9 @@ class TextTranslator(nodes.NodeVisitor): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/en/master/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/en/master/ext/math.html'), location=(self.builder.current_docname, node.line)) raise nodes.SkipNode diff --git a/tests/etree13/ElementPath.py b/tests/etree13/ElementPath.py deleted file mode 100644 index 8cf1ab578..000000000 --- a/tests/etree13/ElementPath.py +++ /dev/null @@ -1,226 +0,0 @@ -# -# ElementTree -# $Id$ -# -# limited xpath support for element trees -# -# history: -# 2003-05-23 fl created -# 2003-05-28 fl added support for // etc -# 2003-08-27 fl fixed parsing of periods in element names -# 2007-09-10 fl new selection engine -# -# Copyright (c) 2003-2007 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2007 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -## -# Implementation module for XPath support. There's usually no reason -# to import this module directly; the <b>ElementTree</b> does this for -# you, if needed. -## - -import re - -xpath_tokenizer = re.compile( - r"(" - r"'[^']*'|\"[^\"]*\"|" - r"::|" - r"//?|" - r"\.\.|" - r"\(\)|" - r"[/.*:\[\]\(\)@=])|" - r"((?:\{[^}]+\})?[^/:\[\]\(\)@=\s]+)|" - r"\s+" - ).findall - -def prepare_tag(next, token): - tag = token[1] - def select(context, result): - for elem in result: - for e in elem: - if e.tag == tag: - yield e - return select - -def prepare_star(next, token): - def select(context, result): - for elem in result: - for e in elem: - yield e - return select - -def prepare_dot(next, token): - def select(context, result): - for elem in result: - yield elem - return select - -def prepare_iter(next, token): - token = next() - if token[0] == "*": - tag = "*" - elif not token[0]: - tag = token[1] - else: - raise SyntaxError - def select(context, result): - for elem in result: - for e in elem.iter(tag): - if e is not elem: - yield e - return select - -def prepare_dot_dot(next, token): - def select(context, result): - parent_map = context.parent_map - if parent_map is None: - context.parent_map = parent_map = {} - for p in context.root.iter(): - for e in p: - parent_map[e] = p - for elem in result: - if elem in parent_map: - yield parent_map[elem] - return select - -def prepare_predicate(next, token): - # this one should probably be refactored... - token = next() - if token[0] == "@": - # attribute - token = next() - if token[0]: - raise SyntaxError("invalid attribute predicate") - key = token[1] - token = next() - if token[0] == "]": - def select(context, result): - for elem in result: - if elem.get(key) is not None: - yield elem - elif token[0] == "=": - value = next()[0] - if value[:1] == "'" or value[:1] == '"': - value = value[1:-1] - else: - raise SyntaxError("invalid comparision target") - token = next() - def select(context, result): - for elem in result: - if elem.get(key) == value: - yield elem - if token[0] != "]": - raise SyntaxError("invalid attribute predicate") - elif not token[0]: - tag = token[1] - token = next() - if token[0] != "]": - raise SyntaxError("invalid node predicate") - def select(context, result): - for elem in result: - if elem.find(tag) is not None: - yield elem - else: - raise SyntaxError("invalid predicate") - return select - -ops = { - "": prepare_tag, - "*": prepare_star, - ".": prepare_dot, - "..": prepare_dot_dot, - "//": prepare_iter, - "[": prepare_predicate, - } - -_cache = {} - -class _SelectorContext: - parent_map = None - def __init__(self, root): - self.root = root - -# -------------------------------------------------------------------- - -## -# Find first matching object. - -def find(elem, path): - try: - return next(findall(elem, path)) - except StopIteration: - return None - -## -# Find all matching objects. - -def findall(elem, path): - # compile selector pattern - try: - selector = _cache[path] - except KeyError: - if len(_cache) > 100: - _cache.clear() - if path[:1] == "/": - raise SyntaxError("cannot use absolute path on element") - stream = iter(xpath_tokenizer(path)) - next_ = lambda: next(stream); token = next_() - selector = [] - while 1: - try: - selector.append(ops[token[0]](next_, token)) - except StopIteration: - raise SyntaxError("invalid path") - try: - token = next_() - if token[0] == "/": - token = next_() - except StopIteration: - break - _cache[path] = selector - # execute selector pattern - result = [elem] - context = _SelectorContext(elem) - for select in selector: - result = select(context, result) - return result - -## -# Find text for first matching object. - -def findtext(elem, path, default=None): - try: - elem = next(findall(elem, path)) - return elem.text - except StopIteration: - return default diff --git a/tests/etree13/ElementTree.py b/tests/etree13/ElementTree.py deleted file mode 100644 index 134abf313..000000000 --- a/tests/etree13/ElementTree.py +++ /dev/null @@ -1,1553 +0,0 @@ -# -# ElementTree -# $Id$ -# -# light-weight XML support for Python 2.2 and later. -# -# history: -# 2001-10-20 fl created (from various sources) -# 2001-11-01 fl return root from parse method -# 2002-02-16 fl sort attributes in lexical order -# 2002-04-06 fl TreeBuilder refactoring, added PythonDoc markup -# 2002-05-01 fl finished TreeBuilder refactoring -# 2002-07-14 fl added basic namespace support to ElementTree.write -# 2002-07-25 fl added QName attribute support -# 2002-10-20 fl fixed encoding in write -# 2002-11-24 fl changed default encoding to ascii; fixed attribute encoding -# 2002-11-27 fl accept file objects or file names for parse/write -# 2002-12-04 fl moved XMLTreeBuilder back to this module -# 2003-01-11 fl fixed entity encoding glitch for us-ascii -# 2003-02-13 fl added XML literal factory -# 2003-02-21 fl added ProcessingInstruction/PI factory -# 2003-05-11 fl added tostring/fromstring helpers -# 2003-05-26 fl added ElementPath support -# 2003-07-05 fl added makeelement factory method -# 2003-07-28 fl added more well-known namespace prefixes -# 2003-08-15 fl fixed typo in ElementTree.findtext (Thomas Dartsch) -# 2003-09-04 fl fall back on emulator if ElementPath is not installed -# 2003-10-31 fl markup updates -# 2003-11-15 fl fixed nested namespace bug -# 2004-03-28 fl added XMLID helper -# 2004-06-02 fl added default support to findtext -# 2004-06-08 fl fixed encoding of non-ascii element/attribute names -# 2004-08-23 fl take advantage of post-2.1 expat features -# 2004-09-03 fl made Element class visible; removed factory -# 2005-02-01 fl added iterparse implementation -# 2005-03-02 fl fixed iterparse support for pre-2.2 versions -# 2005-11-12 fl added tostringlist/fromstringlist helpers -# 2006-07-05 fl merged in selected changes from the 1.3 sandbox -# 2006-07-05 fl removed support for 2.1 and earlier -# 2007-06-21 fl added deprecation/future warnings -# 2007-08-25 fl added doctype hook, added parser version attribute etc -# 2007-08-26 fl added new serializer code (better namespace handling, etc) -# 2007-08-27 fl warn for broken /tag searches on tree level -# 2007-09-02 fl added html/text methods to serializer (experimental) -# 2007-09-05 fl added method argument to tostring/tostringlist -# 2007-09-06 fl improved error handling -# -# Copyright (c) 1999-2007 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2007 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -from __future__ import generators -from __future__ import absolute_import - -from six import string_types - - -__all__ = [ - # public symbols - "Comment", - "dump", - "Element", "ElementTree", - "fromstring", "fromstringlist", - "iselement", "iterparse", - "parse", "ParseError", - "PI", "ProcessingInstruction", - "QName", - "SubElement", - "tostring", "tostringlist", - "TreeBuilder", - "VERSION", - "XML", - "XMLParser", "XMLTreeBuilder", - ] - -## -# The <b>Element</b> type is a flexible container object, designed to -# store hierarchical data structures in memory. The type can be -# described as a cross between a list and a dictionary. -# <p> -# Each element has a number of properties associated with it: -# <ul> -# <li>a <i>tag</i>. This is a string identifying what kind of data -# this element represents (the element type, in other words).</li> -# <li>a number of <i>attributes</i>, stored in a Python dictionary.</li> -# <li>a <i>text</i> string.</li> -# <li>an optional <i>tail</i> string.</li> -# <li>a number of <i>child elements</i>, stored in a Python sequence</li> -# </ul> -# -# To create an element instance, use the {@link #Element} constructor -# or the {@link #SubElement} factory function. -# <p> -# The {@link #ElementTree} class can be used to wrap an element -# structure, and convert it from and to XML. -## - -import sys, re - -class _SimpleElementPath(object): - # emulate pre-1.2 find/findtext/findall behaviour - def find(self, element, tag): - for elem in element: - if elem.tag == tag: - return elem - return None - def findtext(self, element, tag, default=None): - for elem in element: - if elem.tag == tag: - return elem.text or "" - return default - def findall(self, element, tag): - if tag[:3] == ".//": - return element.getiterator(tag[3:]) - result = [] - for elem in element: - if elem.tag == tag: - result.append(elem) - return result - -try: - from . import ElementPath -except ImportError: - # FIXME: issue warning in this case? - ElementPath = _SimpleElementPath() - -VERSION = "1.3a2" - -class ParseError(SyntaxError): - pass - -# -------------------------------------------------------------------- - -## -# Checks if an object appears to be a valid element object. -# -# @param An element instance. -# @return A true value if this is an element object. -# @defreturn flag - -def iselement(element): - # FIXME: not sure about this; might be a better idea to look - # for tag/attrib/text attributes - return isinstance(element, Element) or hasattr(element, "tag") - -## -# Element class. This class defines the Element interface, and -# provides a reference implementation of this interface. -# <p> -# The element name, attribute names, and attribute values can be -# either 8-bit ASCII strings or Unicode strings. -# -# @param tag The element name. -# @param attrib An optional dictionary, containing element attributes. -# @param **extra Additional attributes, given as keyword arguments. -# @see Element -# @see SubElement -# @see Comment -# @see ProcessingInstruction - -class Element(object): - # <tag attrib>text<child/>...</tag>tail - - ## - # (Attribute) Element tag. - - tag = None - - ## - # (Attribute) Element attribute dictionary. Where possible, use - # {@link #Element.get}, - # {@link #Element.set}, - # {@link #Element.keys}, and - # {@link #Element.items} to access - # element attributes. - - attrib = None - - ## - # (Attribute) Text before first subelement. This is either a - # string or the value None, if there was no text. - - text = None - - ## - # (Attribute) Text after this element's end tag, but before the - # next sibling element's start tag. This is either a string or - # the value None, if there was no text. - - tail = None # text after end tag, if any - - def __init__(self, tag, attrib={}, **extra): - attrib = attrib.copy() - attrib.update(extra) - self.tag = tag - self.attrib = attrib - self._children = [] - - def __repr__(self): - return "<Element %s at %x>" % (repr(self.tag), id(self)) - - ## - # Creates a new element object of the same type as this element. - # - # @param tag Element tag. - # @param attrib Element attributes, given as a dictionary. - # @return A new element instance. - - def makeelement(self, tag, attrib): - return Element(tag, attrib) - - ## - # Returns the number of subelements. - # - # @return The number of subelements. - - def __len__(self): - return len(self._children) - - def __bool__(self): - import warnings - warnings.warn( - "The behavior of this method will change in future versions. " - "Use specific 'len(elem)' or 'elem is not None' test instead.", - FutureWarning - ) - return len(self._children) != 0 # emulate old behaviour - __nonzero__ = __bool__ # for python2 compatibility - - ## - # Returns the given subelement. - # - # @param index What subelement to return. - # @return The given subelement. - # @exception IndexError If the given element does not exist. - - def __getitem__(self, index): - return self._children[index] - - ## - # Replaces the given subelement. - # - # @param index What subelement to replace. - # @param element The new element value. - # @exception IndexError If the given element does not exist. - # @exception AssertionError If element is not a valid object. - - def __setitem__(self, index, element): - assert iselement(element) - self._children[index] = element - - ## - # Deletes the given subelement. - # - # @param index What subelement to delete. - # @exception IndexError If the given element does not exist. - - def __delitem__(self, index): - del self._children[index] - - ## - # Returns a list containing subelements in the given range. - # - # @param start The first subelement to return. - # @param stop The first subelement that shouldn't be returned. - # @return A sequence object containing subelements. - - def __getslice__(self, start, stop): - return self._children[start:stop] - - ## - # Replaces a number of subelements with elements from a sequence. - # - # @param start The first subelement to replace. - # @param stop The first subelement that shouldn't be replaced. - # @param elements A sequence object with zero or more elements. - # @exception AssertionError If a sequence member is not a valid object. - - def __setslice__(self, start, stop, elements): - for element in elements: - assert iselement(element) - self._children[start:stop] = list(elements) - - ## - # Deletes a number of subelements. - # - # @param start The first subelement to delete. - # @param stop The first subelement to leave in there. - - def __delslice__(self, start, stop): - del self._children[start:stop] - - ## - # Adds a subelement to the end of this element. - # - # @param element The element to add. - # @exception AssertionError If a sequence member is not a valid object. - - def append(self, element): - assert iselement(element) - self._children.append(element) - - ## - # Appends subelements from a sequence. - # - # @param elements A sequence object with zero or more elements. - # @exception AssertionError If a subelement is not a valid object. - # @since 1.3 - - def extend(self, elements): - for element in elements: - assert iselement(element) - self._children.extend(elements) - - ## - # Inserts a subelement at the given position in this element. - # - # @param index Where to insert the new subelement. - # @exception AssertionError If the element is not a valid object. - - def insert(self, index, element): - assert iselement(element) - self._children.insert(index, element) - - ## - # Removes a matching subelement. Unlike the <b>find</b> methods, - # this method compares elements based on identity, not on tag - # value or contents. - # - # @param element What element to remove. - # @exception ValueError If a matching element could not be found. - # @exception AssertionError If the element is not a valid object. - - def remove(self, element): - assert iselement(element) - self._children.remove(element) - - ## - # (Deprecated) Returns all subelements. The elements are returned - # in document order. - # - # @return A list of subelements. - # @defreturn list of Element instances - - def getchildren(self): - import warnings - warnings.warn( - "This method will be removed in future versions. " - "Use 'list(elem)' or iteration over elem instead.", - DeprecationWarning - ) - return self._children - - ## - # Finds the first matching subelement, by tag name or path. - # - # @param path What element to look for. - # @return The first matching element, or None if no element was found. - # @defreturn Element or None - - def find(self, path): - return ElementPath.find(self, path) - - ## - # Finds text for the first matching subelement, by tag name or path. - # - # @param path What element to look for. - # @param default What to return if the element was not found. - # @return The text content of the first matching element, or the - # default value no element was found. Note that if the element - # has is found, but has no text content, this method returns an - # empty string. - # @defreturn string - - def findtext(self, path, default=None): - return ElementPath.findtext(self, path, default) - - ## - # Finds all matching subelements, by tag name or path. - # - # @param path What element to look for. - # @return A list or iterator containing all matching elements, - # in document order. - # @defreturn list of Element instances - - def findall(self, path): - return ElementPath.findall(self, path) - - ## - # Resets an element. This function removes all subelements, clears - # all attributes, and sets the text and tail attributes to None. - - def clear(self): - self.attrib.clear() - self._children = [] - self.text = self.tail = None - - ## - # Gets an element attribute. - # - # @param key What attribute to look for. - # @param default What to return if the attribute was not found. - # @return The attribute value, or the default value, if the - # attribute was not found. - # @defreturn string or None - - def get(self, key, default=None): - return self.attrib.get(key, default) - - ## - # Sets an element attribute. - # - # @param key What attribute to set. - # @param value The attribute value. - - def set(self, key, value): - self.attrib[key] = value - - ## - # Gets a list of attribute names. The names are returned in an - # arbitrary order (just like for an ordinary Python dictionary). - # - # @return A list of element attribute names. - # @defreturn list of strings - - def keys(self): - return self.attrib.keys() - - ## - # Gets element attributes, as a sequence. The attributes are - # returned in an arbitrary order. - # - # @return A list of (name, value) tuples for all attributes. - # @defreturn list of (string, string) tuples - - def items(self): - return self.attrib.items() - - ## - # Creates a tree iterator. The iterator loops over this element - # and all subelements, in document order, and returns all elements - # with a matching tag. - # <p> - # If the tree structure is modified during iteration, new or removed - # elements may or may not be included. To get a stable set, use the - # list() function on the iterator, and loop over the resulting list. - # - # @param tag What tags to look for (default is to return all elements). - # @return An iterator containing all the matching elements. - # @defreturn iterator - - def iter(self, tag=None): - if tag == "*": - tag = None - if tag is None or self.tag == tag: - yield self - for e in self._children: - for e in e.iter(tag): - yield e - - # compatibility (FIXME: preserve list behaviour too? see below) - getiterator = iter - - # def getiterator(self, tag=None): - # return list(tag) - - ## - # Creates a text iterator. The iterator loops over this element - # and all subelements, in document order, and returns all inner - # text. - # - # @return An iterator containing all inner text. - # @defreturn iterator - - def itertext(self): - if self.text: - yield self.text - for e in self: - for s in e.itertext(): - yield s - if e.tail: - yield e.tail - -# compatibility -_Element = _ElementInterface = Element - -## -# Subelement factory. This function creates an element instance, and -# appends it to an existing element. -# <p> -# The element name, attribute names, and attribute values can be -# either 8-bit ASCII strings or Unicode strings. -# -# @param parent The parent element. -# @param tag The subelement name. -# @param attrib An optional dictionary, containing element attributes. -# @param **extra Additional attributes, given as keyword arguments. -# @return An element instance. -# @defreturn Element - -def SubElement(parent, tag, attrib={}, **extra): - attrib = attrib.copy() - attrib.update(extra) - element = parent.makeelement(tag, attrib) - parent.append(element) - return element - -## -# Comment element factory. This factory function creates a special -# element that will be serialized as an XML comment by the standard -# serializer. -# <p> -# The comment string can be either an 8-bit ASCII string or a Unicode -# string. -# -# @param text A string containing the comment string. -# @return An element instance, representing a comment. -# @defreturn Element - -def Comment(text=None): - element = Element(Comment) - element.text = text - return element - -## -# PI element factory. This factory function creates a special element -# that will be serialized as an XML processing instruction by the standard -# serializer. -# -# @param target A string containing the PI target. -# @param text A string containing the PI contents, if any. -# @return An element instance, representing a PI. -# @defreturn Element - -def ProcessingInstruction(target, text=None): - element = Element(ProcessingInstruction) - element.text = target - if text: - element.text = element.text + " " + text - return element - -PI = ProcessingInstruction - -## -# QName wrapper. This can be used to wrap a QName attribute value, in -# order to get proper namespace handling on output. -# -# @param text A string containing the QName value, in the form {uri}local, -# or, if the tag argument is given, the URI part of a QName. -# @param tag Optional tag. If given, the first argument is interpreted as -# an URI, and this argument is interpreted as a local name. -# @return An opaque object, representing the QName. - -class QName(object): - def __init__(self, text_or_uri, tag=None): - if tag: - text_or_uri = "{%s}%s" % (text_or_uri, tag) - self.text = text_or_uri - def __str__(self): - return self.text - def __hash__(self): - return hash(self.text) - def __cmp__(self, other): - if isinstance(other, QName): - return cmp(self.text, other.text) - return cmp(self.text, other) - -# -------------------------------------------------------------------- - -## -# ElementTree wrapper class. This class represents an entire element -# hierarchy, and adds some extra support for serialization to and from -# standard XML. -# -# @param element Optional root element. -# @keyparam file Optional file handle or file name. If given, the -# tree is initialized with the contents of this XML file. - -class ElementTree(object): - - def __init__(self, element=None, file=None): - assert element is None or iselement(element) - self._root = element # first node - if file: - self.parse(file) - - ## - # Gets the root element for this tree. - # - # @return An element instance. - # @defreturn Element - - def getroot(self): - return self._root - - ## - # Replaces the root element for this tree. This discards the - # current contents of the tree, and replaces it with the given - # element. Use with care. - # - # @param element An element instance. - - def _setroot(self, element): - assert iselement(element) - self._root = element - - ## - # Loads an external XML document into this element tree. - # - # @param source A file name or file object. - # @keyparam parser An optional parser instance. If not given, the - # standard {@link XMLParser} parser is used. - # @return The document root element. - # @defreturn Element - - def parse(self, source, parser=None): - if not hasattr(source, "read"): - source = open(source, "rb") - if not parser: - parser = XMLParser(target=TreeBuilder()) - while 1: - data = source.read(32768) - if not data: - break - parser.feed(data) - self._root = parser.close() - return self._root - - ## - # Creates a tree iterator for the root element. The iterator loops - # over all elements in this tree, in document order. - # - # @param tag What tags to look for (default is to return all elements) - # @return An iterator. - # @defreturn iterator - - def iter(self, tag=None): - assert self._root is not None - return self._root.iter(tag) - - getiterator = iter - - ## - # Finds the first toplevel element with given tag. - # Same as getroot().find(path). - # - # @param path What element to look for. - # @return The first matching element, or None if no element was found. - # @defreturn Element or None - - def find(self, path): - assert self._root is not None - if path[:1] == "/": - path = "." + path - import warnings - warnings.warn( - "This search is broken in 1.3 and earlier; if you rely " - "on the current behaviour, change it to %r" % path, - FutureWarning - ) - return self._root.find(path) - - ## - # Finds the element text for the first toplevel element with given - # tag. Same as getroot().findtext(path). - # - # @param path What toplevel element to look for. - # @param default What to return if the element was not found. - # @return The text content of the first matching element, or the - # default value no element was found. Note that if the element - # has is found, but has no text content, this method returns an - # empty string. - # @defreturn string - - def findtext(self, path, default=None): - assert self._root is not None - if path[:1] == "/": - path = "." + path - import warnings - warnings.warn( - "This search is broken in 1.3 and earlier; if you rely " - "on the current behaviour, change it to %r" % path, - FutureWarning - ) - return self._root.findtext(path, default) - - ## - # Finds all toplevel elements with the given tag. - # Same as getroot().findall(path). - # - # @param path What element to look for. - # @return A list or iterator containing all matching elements, - # in document order. - # @defreturn list of Element instances - - def findall(self, path): - assert self._root is not None - if path[:1] == "/": - path = "." + path - import warnings - warnings.warn( - "This search is broken in 1.3 and earlier; if you rely " - "on the current behaviour, change it to %r" % path, - FutureWarning - ) - return self._root.findall(path) - - ## - # Writes the element tree to a file, as XML. - # - # @param file A file name, or a file object opened for writing. - # @keyparam encoding Optional output encoding (default is US-ASCII). - # @keyparam method Optional output method ("xml" or "html"; default - # is "xml". - # @keyparam xml_declaration Controls if an XML declaration should - # be added to the file. Use False for never, True for always, - # None for only if not US-ASCII or UTF-8. None is default. - - def write(self, file, - # keyword arguments - encoding="us-ascii", - xml_declaration=None, - default_namespace=None, - method=None): - assert self._root is not None - if not hasattr(file, "write"): - file = open(file, "wb") - write = file.write - if not method: - method = "xml" - if not encoding: - encoding = "us-ascii" - elif xml_declaration or (xml_declaration is None and - encoding not in ("utf-8", "us-ascii")): - write("<?xml version='1.0' encoding='%s'?>\n" % encoding) - if method == "text": - _serialize_text(write, self._root, encoding) - else: - qnames, namespaces = _namespaces( - self._root, encoding, default_namespace - ) - if method == "xml": - _serialize_xml( - write, self._root, encoding, qnames, namespaces - ) - elif method == "html": - _serialize_html( - write, self._root, encoding, qnames, namespaces - ) - else: - raise ValueError("unknown method %r" % method) - -# -------------------------------------------------------------------- -# serialization support - -def _namespaces(elem, encoding, default_namespace=None): - # identify namespaces used in this tree - - # maps qnames to *encoded* prefix:local names - qnames = {None: None} - - # maps uri:s to prefixes - namespaces = {} - if default_namespace: - namespaces[default_namespace] = "" - - def encode(text): - return text.encode(encoding) - - def add_qname(qname): - # calculate serialized qname representation - try: - if qname[:1] == "{": - uri, tag = qname[1:].split("}", 1) - prefix = namespaces.get(uri) - if prefix is None: - prefix = _namespace_map.get(uri) - if prefix is None: - prefix = "ns%d" % len(namespaces) - if prefix != "xml": - namespaces[uri] = prefix - if prefix: - qnames[qname] = encode("%s:%s" % (prefix, tag)) - else: - qnames[qname] = encode(tag) # default element - else: - if default_namespace: - # FIXME: can this be handled in XML 1.0? - raise ValueError( - "cannot use non-qualified names with " - "default_namespace option" - ) - qnames[qname] = encode(qname) - except TypeError: - _raise_serialization_error(qname) - - # populate qname and namespaces table - try: - iterate = elem.iter - except AttributeError: - iterate = elem.getiterator # cET compatibility - for elem in iterate(): - tag = elem.tag - if isinstance(tag, QName) and tag.text not in qnames: - add_qname(tag.text) - elif isinstance(tag, string_types): - if tag not in qnames: - add_qname(tag) - elif tag is not None and tag is not Comment and tag is not PI: - _raise_serialization_error(tag) - for key, value in elem.items(): - if isinstance(key, QName): - key = key.text - if key not in qnames: - add_qname(key) - if isinstance(value, QName) and value.text not in qnames: - add_qname(value.text) - text = elem.text - if isinstance(text, QName) and text.text not in qnames: - add_qname(text.text) - return qnames, namespaces - -def _serialize_xml(write, elem, encoding, qnames, namespaces): - tag = elem.tag - text = elem.text - if tag is Comment: - write("<!--%s-->" % _escape_cdata(text, encoding)) - elif tag is ProcessingInstruction: - write("<?%s?>" % _escape_cdata(text, encoding)) - else: - tag = qnames[tag] - if tag is None: - if text: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_xml(write, e, encoding, qnames, None) - else: - write("<" + tag) - items = elem.items() - if items or namespaces: - items = sorted(items) # lexical order - for k, v in items: - if isinstance(k, QName): - k = k.text - if isinstance(v, QName): - v = qnames[v.text] - else: - v = _escape_attrib(v, encoding) - write(" %s=\"%s\"" % (qnames[k], v)) - if namespaces: - items = namespaces.items() - items = sorted(items, key=lambda x: x[1]) # sort on prefix - for v, k in items: - if k: - k = ":" + k - write(" xmlns%s=\"%s\"" % ( - k.encode(encoding), - _escape_attrib(v, encoding) - )) - if text or len(elem): - write(">") - if text: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_xml(write, e, encoding, qnames, None) - write("</" + tag + ">") - else: - write(" />") - if elem.tail: - write(_escape_cdata(elem.tail, encoding)) - -HTML_EMPTY = ("area", "base", "basefont", "br", "col", "frame", "hr", - "img", "input", "isindex", "link", "meta" "param") - -try: - HTML_EMPTY = set(HTML_EMPTY) -except NameError: - pass - -def _serialize_html(write, elem, encoding, qnames, namespaces): - tag = elem.tag - text = elem.text - if tag is Comment: - write("<!--%s-->" % _escape_cdata(text, encoding)) - elif tag is ProcessingInstruction: - write("<?%s?>" % _escape_cdata(text, encoding)) - else: - tag = qnames[tag] - if tag is None: - if text: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_html(write, e, encoding, qnames, None) - else: - write("<" + tag) - items = elem.items() - if items or namespaces: - items = sorted(items) # lexical order - for k, v in items: - if isinstance(k, QName): - k = k.text - if isinstance(v, QName): - v = qnames[v.text] - else: - v = _escape_attrib_html(v, encoding) - # FIXME: handle boolean attributes - write(" %s=\"%s\"" % (qnames[k], v)) - if namespaces: - items = namespaces.items() - items = sorted(items, key=lambda x: x[1]) # sort on prefix - for v, k in items: - if k: - k = ":" + k - write(" xmlns%s=\"%s\"" % ( - k.encode(encoding), - _escape_attrib(v, encoding) - )) - write(">") - tag = tag.lower() - if text: - if tag == "script" or tag == "style": - write(_encode(text, encoding)) - else: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_html(write, e, encoding, qnames, None) - if tag not in HTML_EMPTY: - write("</" + tag + ">") - if elem.tail: - write(_escape_cdata(elem.tail, encoding)) - -def _serialize_text(write, elem, encoding): - for part in elem.itertext(): - write(part.encode(encoding)) - if elem.tail: - write(elem.tail.encode(encoding)) - -## -# Registers a namespace prefix. The registry is global, and any -# existing mapping for either the given prefix or the namespace URI -# will be removed. -# -# @param prefix Namespace prefix. -# @param uri Namespace uri. Tags and attributes in this namespace -# will be serialized with the given prefix, if at all possible. -# @raise ValueError If the prefix is reserved, or is otherwise -# invalid. - -def register_namespace(prefix, uri): - if re.match(r"ns\d+$", prefix): - raise ValueError("Prefix format reserved for internal use") - for k, v in _namespace_map.items(): - if k == uri or v == prefix: - del _namespace_map[k] - _namespace_map[uri] = prefix - -_namespace_map = { - # "well-known" namespace prefixes - "http://www.w3.org/XML/1998/namespace": "xml", - "http://www.w3.org/1999/xhtml": "html", - "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf", - "http://schemas.xmlsoap.org/wsdl/": "wsdl", - # xml schema - "http://www.w3.org/2001/XMLSchema": "xs", - "http://www.w3.org/2001/XMLSchema-instance": "xsi", - # dublic core - "http://purl.org/dc/elements/1.1/": "dc", -} - -def _raise_serialization_error(text): - raise TypeError( - "cannot serialize %r (type %s)" % (text, type(text).__name__) - ) - -def _encode(text, encoding): - try: - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_cdata(text, encoding): - # escape character data - try: - # it's worth avoiding do-nothing calls for strings that are - # shorter than 500 character, or so. assume that's, by far, - # the most common case in most applications. - if "&" in text: - text = text.replace("&", "&") - if "<" in text: - text = text.replace("<", "<") - if ">" in text: - text = text.replace(">", ">") - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_attrib(text, encoding): - # escape attribute value - try: - if "&" in text: - text = text.replace("&", "&") - if "<" in text: - text = text.replace("<", "<") - if ">" in text: - text = text.replace(">", ">") - if "\"" in text: - text = text.replace("\"", """) - if "\n" in text: - text = text.replace("\n", " ") - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_attrib_html(text, encoding): - # escape attribute value - try: - if "&" in text: - text = text.replace("&", "&") - if ">" in text: - text = text.replace(">", ">") - if "\"" in text: - text = text.replace("\"", """) - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -# -------------------------------------------------------------------- - -## -# Generates a string representation of an XML element, including all -# subelements. -# -# @param element An Element instance. -# @return An encoded string containing the XML data. -# @defreturn string - -def tostring(element, encoding=None, method=None): - class dummy: - pass - data = [] - file = dummy() - file.write = data.append - ElementTree(element).write(file, encoding, method=method) - return "".join(data) - -## -# Generates a string representation of an XML element, including all -# subelements. The string is returned as a sequence of string fragments. -# -# @param element An Element instance. -# @return A sequence object containing the XML data. -# @defreturn sequence -# @since 1.3 - -def tostringlist(element, encoding=None): - class dummy: - pass - data = [] - file = dummy() - file.write = data.append - ElementTree(element).write(file, encoding) - # FIXME: merge small fragments into larger parts - return data - -## -# Writes an element tree or element structure to sys.stdout. This -# function should be used for debugging only. -# <p> -# The exact output format is implementation dependent. In this -# version, it's written as an ordinary XML file. -# -# @param elem An element tree or an individual element. - -def dump(elem): - # debugging - if not isinstance(elem, ElementTree): - elem = ElementTree(elem) - elem.write(sys.stdout) - tail = elem.getroot().tail - if not tail or tail[-1] != "\n": - sys.stdout.write("\n") - -# -------------------------------------------------------------------- -# parsing - -## -# Parses an XML document into an element tree. -# -# @param source A filename or file object containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return An ElementTree instance - -def parse(source, parser=None): - tree = ElementTree() - tree.parse(source, parser) - return tree - -## -# Parses an XML document into an element tree incrementally, and reports -# what's going on to the user. -# -# @param source A filename or file object containing XML data. -# @param events A list of events to report back. If omitted, only "end" -# events are reported. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return A (event, elem) iterator. - -def iterparse(source, events=None, parser=None): - if not hasattr(source, "read"): - source = open(source, "rb") - if not parser: - parser = XMLParser(target=TreeBuilder()) - return _IterParseIterator(source, events, parser) - -class _IterParseIterator(object): - - def __init__(self, source, events, parser): - self._file = source - self._events = [] - self._index = 0 - self.root = self._root = None - self._parser = parser - # wire up the parser for event reporting - parser = self._parser._parser - append = self._events.append - if events is None: - events = ["end"] - for event in events: - if event == "start": - try: - parser.ordered_attributes = 1 - parser.specified_attributes = 1 - def handler(tag, attrib_in, event=event, append=append, - start=self._parser._start_list): - append((event, start(tag, attrib_in))) - parser.StartElementHandler = handler - except AttributeError: - def handler(tag, attrib_in, event=event, append=append, - start=self._parser._start): - append((event, start(tag, attrib_in))) - parser.StartElementHandler = handler - elif event == "end": - def handler(tag, event=event, append=append, - end=self._parser._end): - append((event, end(tag))) - parser.EndElementHandler = handler - elif event == "start-ns": - def handler(prefix, uri, event=event, append=append): - try: - uri = uri.encode("ascii") - except UnicodeError: - pass - append((event, (prefix or "", uri))) - parser.StartNamespaceDeclHandler = handler - elif event == "end-ns": - def handler(prefix, event=event, append=append): - append((event, None)) - parser.EndNamespaceDeclHandler = handler - - def __next__(self): - while 1: - try: - item = self._events[self._index] - except IndexError: - if self._parser is None: - self.root = self._root - raise StopIteration - # load event buffer - del self._events[:] - self._index = 0 - data = self._file.read(16384) - if data: - self._parser.feed(data) - else: - self._root = self._parser.close() - self._parser = None - else: - self._index = self._index + 1 - return item - - next = __next__ # Python 2 compatibility - - def __iter__(self): - return self - -## -# Parses an XML document from a string constant. This function can -# be used to embed "XML literals" in Python code. -# -# @param source A string containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return An Element instance. -# @defreturn Element - -def XML(text, parser=None): - if not parser: - parser = XMLParser(target=TreeBuilder()) - parser.feed(text) - return parser.close() - -## -# Parses an XML document from a string constant, and also returns -# a dictionary which maps from element id:s to elements. -# -# @param source A string containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return A tuple containing an Element instance and a dictionary. -# @defreturn (Element, dictionary) - -def XMLID(text, parser=None): - if not parser: - parser = XMLParser(target=TreeBuilder()) - parser.feed(text) - tree = parser.close() - ids = {} - for elem in tree.getiterator(): - id = elem.get("id") - if id: - ids[id] = elem - return tree, ids - -## -# Parses an XML document from a string constant. Same as {@link #XML}. -# -# @def fromstring(text) -# @param source A string containing XML data. -# @return An Element instance. -# @defreturn Element - -fromstring = XML - -## -# Parses an XML document from a sequence of string fragments. -# -# @param sequence A list or other sequence containing XML data fragments. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return An Element instance. -# @defreturn Element -# @since 1.3 - -def fromstringlist(sequence, parser=None): - if not parser: - parser = XMLParser(target=TreeBuilder()) - for text in sequence: - parser.feed(text) - return parser.close() - -# -------------------------------------------------------------------- - -## -# Generic element structure builder. This builder converts a sequence -# of {@link #TreeBuilder.start}, {@link #TreeBuilder.data}, and {@link -# #TreeBuilder.end} method calls to a well-formed element structure. -# <p> -# You can use this class to build an element structure using a custom XML -# parser, or a parser for some other XML-like format. -# -# @param element_factory Optional element factory. This factory -# is called to create new Element instances, as necessary. - -class TreeBuilder(object): - - def __init__(self, element_factory=None): - self._data = [] # data collector - self._elem = [] # element stack - self._last = None # last element - self._tail = None # true if we're after an end tag - if element_factory is None: - element_factory = Element - self._factory = element_factory - - ## - # Flushes the builder buffers, and returns the toplevel document - # element. - # - # @return An Element instance. - # @defreturn Element - - def close(self): - assert len(self._elem) == 0, "missing end tags" - assert self._last != None, "missing toplevel element" - return self._last - - def _flush(self): - if self._data: - if self._last is not None: - text = "".join(self._data) - if self._tail: - assert self._last.tail is None, "internal error (tail)" - self._last.tail = text - else: - assert self._last.text is None, "internal error (text)" - self._last.text = text - self._data = [] - - ## - # Adds text to the current element. - # - # @param data A string. This should be either an 8-bit string - # containing ASCII text, or a Unicode string. - - def data(self, data): - self._data.append(data) - - ## - # Opens a new element. - # - # @param tag The element name. - # @param attrib A dictionary containing element attributes. - # @return The opened element. - # @defreturn Element - - def start(self, tag, attrs): - self._flush() - self._last = elem = self._factory(tag, attrs) - if self._elem: - self._elem[-1].append(elem) - self._elem.append(elem) - self._tail = 0 - return elem - - ## - # Closes the current element. - # - # @param tag The element name. - # @return The closed element. - # @defreturn Element - - def end(self, tag): - self._flush() - self._last = self._elem.pop() - assert self._last.tag == tag,\ - "end tag mismatch (expected %s, got %s)" % ( - self._last.tag, tag) - self._tail = 1 - return self._last - -## -# Element structure builder for XML source data, based on the -# <b>expat</b> parser. -# -# @keyparam target Target object. If omitted, the builder uses an -# instance of the standard {@link #TreeBuilder} class. -# @keyparam html Predefine HTML entities. This flag is not supported -# by the current implementation. -# @keyparam encoding Optional encoding. If given, the value overrides -# the encoding specified in the XML file. -# @see #ElementTree -# @see #TreeBuilder - -class XMLParser(object): - - def __init__(self, html=0, target=None, encoding=None): - try: - from xml.parsers import expat - except ImportError: - try: - import pyexpat; expat = pyexpat - except ImportError: - raise ImportError( - "No module named expat; use SimpleXMLTreeBuilder instead" - ) - parser = expat.ParserCreate(encoding, "}") - if target is None: - target = TreeBuilder() - # underscored names are provided for compatibility only - self.parser = self._parser = parser - self.target = self._target = target - self._error = expat.error - self._names = {} # name memo cache - # callbacks - parser.DefaultHandlerExpand = self._default - parser.StartElementHandler = self._start - parser.EndElementHandler = self._end - parser.CharacterDataHandler = self._data - # let expat do the buffering, if supported - try: - self._parser.buffer_text = 1 - except AttributeError: - pass - # use new-style attribute handling, if supported - try: - self._parser.ordered_attributes = 1 - self._parser.specified_attributes = 1 - parser.StartElementHandler = self._start_list - except AttributeError: - pass - self._doctype = None - self.entity = {} - try: - self.version = "Expat %d.%d.%d" % expat.version_info - except AttributeError: - pass # unknown - - def _raiseerror(self, value): - err = ParseError(value) - err.code = value.code - err.position = value.lineno, value.offset - raise err - - if sys.version_info >= (3, 0): - def _fixtext(self, text): - return text - else: - def _fixtext(self, text): - # convert text string to ascii, if possible - try: - return text.encode("ascii") - except UnicodeError: - return text - - def _fixname(self, key): - # expand qname, and convert name string to ascii, if possible - try: - name = self._names[key] - except KeyError: - name = key - if "}" in name: - name = "{" + name - self._names[key] = name = self._fixtext(name) - return name - - def _start(self, tag, attrib_in): - fixname = self._fixname - fixtext = self._fixtext - tag = fixname(tag) - attrib = {} - for key, value in attrib_in.items(): - attrib[fixname(key)] = fixtext(value) - return self.target.start(tag, attrib) - - def _start_list(self, tag, attrib_in): - fixname = self._fixname - fixtext = self._fixtext - tag = fixname(tag) - attrib = {} - if attrib_in: - for i in range(0, len(attrib_in), 2): - attrib[fixname(attrib_in[i])] = fixtext(attrib_in[i+1]) - return self.target.start(tag, attrib) - - def _data(self, text): - return self.target.data(self._fixtext(text)) - - def _end(self, tag): - return self.target.end(self._fixname(tag)) - - def _default(self, text): - prefix = text[:1] - if prefix == "&": - # deal with undefined entities - try: - self.target.data(self.entity[text[1:-1]]) - except KeyError: - from xml.parsers import expat - err = expat.error( - "undefined entity %s: line %d, column %d" % - (text, self._parser.ErrorLineNumber, - self._parser.ErrorColumnNumber) - ) - err.code = 11 # XML_ERROR_UNDEFINED_ENTITY - err.lineno = self._parser.ErrorLineNumber - err.offset = self._parser.ErrorColumnNumber - raise err - elif prefix == "<" and text[:9] == "<!DOCTYPE": - self._doctype = [] # inside a doctype declaration - elif self._doctype is not None: - # parse doctype contents - if prefix == ">": - self._doctype = None - return - text = text.strip() - if not text: - return - self._doctype.append(text) - n = len(self._doctype) - if n > 2: - type = self._doctype[1] - if type == "PUBLIC" and n == 4: - name, type, pubid, system = self._doctype - elif type == "SYSTEM" and n == 3: - name, type, system = self._doctype - pubid = None - else: - return - if pubid: - pubid = pubid[1:-1] - if hasattr(self.target, "doctype"): - self.target.doctype(name, pubid, system[1:-1]) - self._doctype = None - - ## - # Feeds data to the parser. - # - # @param data Encoded data. - - def feed(self, data): - try: - self._parser.Parse(data, 0) - except self._error as v: - self._raiseerror(v) - - ## - # Finishes feeding data to the parser. - # - # @return An element structure. - # @defreturn Element - - def close(self): - try: - self._parser.Parse("", 1) # end of data - except self._error as v: - self._raiseerror(v) - tree = self.target.close() - del self.target, self._parser # get rid of circular references - return tree - -# compatibility -XMLTreeBuilder = XMLParser diff --git a/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py b/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py index 6f493c3a3..00a5a7039 100644 --- a/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py +++ b/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py @@ -9,7 +9,7 @@ sys.path.insert(0, os.path.abspath('.')) class DummyTestParser(Parser): - pass + supported = ('dummy',) extensions = ['source_parser'] diff --git a/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py b/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py index 0dff7e311..69898ed91 100644 --- a/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py +++ b/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py @@ -4,8 +4,9 @@ from docutils.parsers import Parser class TestSourceParser(Parser): - pass + supported = ('test',) def setup(app): - app.add_source_parser('.test', TestSourceParser) + app.add_source_suffix('.test', 'test') + app.add_source_parser(TestSourceParser) diff --git a/tests/roots/test-add_source_parser/conf.py b/tests/roots/test-add_source_parser/conf.py index 805f80fcc..eface21e8 100644 --- a/tests/roots/test-add_source_parser/conf.py +++ b/tests/roots/test-add_source_parser/conf.py @@ -9,7 +9,7 @@ sys.path.insert(0, os.path.abspath('.')) class DummyMarkdownParser(Parser): - pass + supported = ('markdown',) extensions = ['source_parser'] diff --git a/tests/roots/test-add_source_parser/source_parser.py b/tests/roots/test-add_source_parser/source_parser.py index 0dff7e311..69898ed91 100644 --- a/tests/roots/test-add_source_parser/source_parser.py +++ b/tests/roots/test-add_source_parser/source_parser.py @@ -4,8 +4,9 @@ from docutils.parsers import Parser class TestSourceParser(Parser): - pass + supported = ('test',) def setup(app): - app.add_source_parser('.test', TestSourceParser) + app.add_source_suffix('.test', 'test') + app.add_source_parser(TestSourceParser) diff --git a/tests/roots/test-config/conf.py b/tests/roots/test-config/conf.py index 4c2ea9fc5..0027d87f2 100644 --- a/tests/roots/test-config/conf.py +++ b/tests/roots/test-config/conf.py @@ -1,54 +1,3 @@ -from sphinx.config import string_classes, ENUM - -value1 = 123 # wrong type -value2 = 123 # lambda with wrong type -value3 = [] # lambda with correct type -value4 = True # child type -value5 = 3 # parent type -value6 = () # other sequence type, also raises -value7 = ['foo'] # explicitly permitted - - -class A(object): - pass - - -class B(A): - pass - - -class C(A): - pass - - -value8 = C() # sibling type - -# both have no default or permissible types -value9 = 'foo' -value10 = 123 -value11 = u'bar' -value12 = u'bar' -value13 = 'bar' -value14 = u'bar' -value15 = 'bar' -value16 = u'bar' - - -def setup(app): - app.add_config_value('value1', 'string', False) - app.add_config_value('value2', lambda conf: [], False) - app.add_config_value('value3', [], False) - app.add_config_value('value4', 100, False) - app.add_config_value('value5', False, False) - app.add_config_value('value6', [], False) - app.add_config_value('value7', 'string', False, [list]) - app.add_config_value('value8', B(), False) - app.add_config_value('value9', None, False) - app.add_config_value('value10', None, False) - app.add_config_value('value11', None, False, [str]) - app.add_config_value('value12', 'string', False) - app.add_config_value('value13', None, False, string_classes) - app.add_config_value('value14', None, False, string_classes) - app.add_config_value('value15', u'unicode', False) - app.add_config_value('value16', u'unicode', False) - app.add_config_value('value17', 'default', False, ENUM('default', 'one', 'two')) +project = 'Sphinx <Tests>' +release = '0.6alpha1' +templates_path = ['_templates'] diff --git a/tests/roots/test-ext-viewcode-find/conf.py b/tests/roots/test-ext-viewcode-find/conf.py new file mode 100644 index 000000000..3f5ddc175 --- /dev/null +++ b/tests/roots/test-ext-viewcode-find/conf.py @@ -0,0 +1,9 @@ +# -*- coding: utf-8 -*- + +import os +import sys + +extensions = ['sphinx.ext.viewcode'] +master_doc = 'index' +exclude_patterns = ['_build'] +viewcode_follow_imported_members = False diff --git a/tests/roots/test-ext-viewcode-find/index.rst b/tests/roots/test-ext-viewcode-find/index.rst new file mode 100644 index 000000000..7eb416ac3 --- /dev/null +++ b/tests/roots/test-ext-viewcode-find/index.rst @@ -0,0 +1,38 @@ +viewcode +======== + +.. py:module:: not_a_package + +.. py:function:: func1(a, b) + + This is func1 + +.. py:function:: not_a_package.submodule.func1(a, b) + + This is func1 + +.. py:module:: not_a_package.submodule + +.. py:class:: Class1 + + This is Class1 + +.. py:class:: Class3 + + This is Class3 + +.. py:class:: not_a_package.submodule.Class1 + + This is Class1 + +.. literalinclude:: not_a_package/__init__.py + :language: python + :pyobject: func1 + +.. literalinclude:: not_a_package/submodule.py + :language: python + :pyobject: func1 + +.. py:attribute:: not_a_package.submodule.Class3.class_attr + + This is the class attribute class_attr diff --git a/tests/roots/test-ext-viewcode-find/not_a_package/__init__.py b/tests/roots/test-ext-viewcode-find/not_a_package/__init__.py new file mode 100644 index 000000000..4a1d689e5 --- /dev/null +++ b/tests/roots/test-ext-viewcode-find/not_a_package/__init__.py @@ -0,0 +1,3 @@ +from __future__ import absolute_import + +from .submodule import func1, Class1 # NOQA diff --git a/tests/roots/test-ext-viewcode-find/not_a_package/submodule.py b/tests/roots/test-ext-viewcode-find/not_a_package/submodule.py new file mode 100644 index 000000000..fb697ab75 --- /dev/null +++ b/tests/roots/test-ext-viewcode-find/not_a_package/submodule.py @@ -0,0 +1,30 @@ +""" +submodule +""" +raise RuntimeError('This module should not get imported') + +def decorator(f): + return f + + +@decorator +def func1(a, b): + """ + this is func1 + """ + return a, b + + +@decorator +class Class1(object): + """ + this is Class1 + """ + + +class Class3(object): + """ + this is Class3 + """ + class_attr = 42 + """this is the class attribute class_attr""" diff --git a/tests/roots/test-ext-viewcode/conf.py b/tests/roots/test-ext-viewcode/conf.py index 08522791b..53ce4f7ce 100644 --- a/tests/roots/test-ext-viewcode/conf.py +++ b/tests/roots/test-ext-viewcode/conf.py @@ -3,7 +3,9 @@ import os import sys -sys.path.insert(0, os.path.abspath('.')) +source_dir = os.path.abspath('.') +if source_dir not in sys.path: + sys.path.insert(0, source_dir) extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] master_doc = 'index' exclude_patterns = ['_build'] diff --git a/tests/roots/test-footnotes/index.rst b/tests/roots/test-footnotes/index.rst index a714f9e22..226c868a9 100644 --- a/tests/roots/test-footnotes/index.rst +++ b/tests/roots/test-footnotes/index.rst @@ -39,7 +39,8 @@ The section with a reference to [AuthorYear]_ .. [AuthorYear] Author, Title, Year .. [1] Second -.. [#] Third +.. [#] Third [#]_ +.. [#] Footnote inside footnote The section with a reference to [#]_ ===================================== diff --git a/tests/roots/test-html_assets/conf.py b/tests/roots/test-html_assets/conf.py index a17e417a3..c61f0b42c 100644 --- a/tests/roots/test-html_assets/conf.py +++ b/tests/roots/test-html_assets/conf.py @@ -6,4 +6,6 @@ version = '1.4.4' html_static_path = ['static', 'subdir'] html_extra_path = ['extra', 'subdir'] +html_css_files = ['css/style.css', + ('https://example.com/custom.css', {'title': 'title', 'media': 'print'})] exclude_patterns = ['**/_build', '**/.htpasswd'] diff --git a/tests/roots/test-html_assets/extra/index.rst b/tests/roots/test-html_assets/extra/index.rst new file mode 100644 index 000000000..e69de29bb --- /dev/null +++ b/tests/roots/test-html_assets/extra/index.rst diff --git a/tests/roots/test-html_assets/static/index.rst b/tests/roots/test-html_assets/static/index.rst new file mode 100644 index 000000000..e69de29bb --- /dev/null +++ b/tests/roots/test-html_assets/static/index.rst diff --git a/tests/roots/test-html_assets/static/js/custom.js b/tests/roots/test-html_assets/static/js/custom.js new file mode 100644 index 000000000..e69de29bb --- /dev/null +++ b/tests/roots/test-html_assets/static/js/custom.js diff --git a/tests/roots/test-intl/refs.txt b/tests/roots/test-intl/refs.txt index b093dbe60..0f923a9e7 100644 --- a/tests/roots/test-intl/refs.txt +++ b/tests/roots/test-intl/refs.txt @@ -4,7 +4,7 @@ References Translation Tips ----------------- -.. _download Sphinx: https://pypi.python.org/pypi/sphinx +.. _download Sphinx: https://pypi.org/project/Sphinx/ .. _Docutils site: http://docutils.sourceforge.net/ .. _Sphinx site: http://sphinx-doc.org/ diff --git a/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.mo b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.mo Binary files differnew file mode 100644 index 000000000..6aa00f77a --- /dev/null +++ b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.mo diff --git a/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.po b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.po new file mode 100644 index 000000000..ee1f6c23f --- /dev/null +++ b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.po @@ -0,0 +1,2 @@ +msgid "Hello world" +msgstr "HELLO WORLD" diff --git a/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.mo b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.mo Binary files differnew file mode 100644 index 000000000..14c34d0b7 --- /dev/null +++ b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.mo diff --git a/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.po b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.po new file mode 100644 index 000000000..d376cf9bd --- /dev/null +++ b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.po @@ -0,0 +1,2 @@ +msgid "Hello sphinx" +msgstr "HELLO SPHINX" diff --git a/tests/roots/test-prolog/prolog_markdown_parser.py b/tests/roots/test-prolog/prolog_markdown_parser.py index f28c37b4e..56ce3cf5a 100644 --- a/tests/roots/test-prolog/prolog_markdown_parser.py +++ b/tests/roots/test-prolog/prolog_markdown_parser.py @@ -4,9 +4,12 @@ from docutils.parsers import Parser class DummyMarkdownParser(Parser): + supported = ('markdown',) + def parse(self, inputstring, document): document.rawsource = inputstring def setup(app): - app.add_source_parser('.md', DummyMarkdownParser) + app.add_source_suffix('.md', 'markdown') + app.add_source_parser(DummyMarkdownParser) diff --git a/tests/roots/test-root/conf.py b/tests/roots/test-root/conf.py index e2d8928fe..f96ea8821 100644 --- a/tests/roots/test-root/conf.py +++ b/tests/roots/test-root/conf.py @@ -12,7 +12,7 @@ from sphinx import addnodes sys.path.append(os.path.abspath('.')) extensions = ['sphinx.ext.autodoc', 'sphinx.ext.jsmath', 'sphinx.ext.todo', - 'sphinx.ext.coverage', 'sphinx.ext.extlinks', 'ext'] + 'sphinx.ext.coverage', 'sphinx.ext.extlinks'] jsmath_path = 'dummy.js' @@ -20,7 +20,6 @@ templates_path = ['_templates'] master_doc = 'contents' source_suffix = ['.txt', '.add', '.foo'] -source_parsers = {'.foo': 'parsermod.Parser'} project = 'Sphinx <Tests>' copyright = '2010-2016, Georg Brandl & Team' @@ -66,8 +65,6 @@ man_pages = [ 'Georg Brandl and someone else', 1), ] -value_from_conf_py = 84 - coverage_c_path = ['special/*.h'] coverage_c_regexes = {'function': r'^PyAPI_FUNC\(.*\)\s+([^_][\w_]+)'} @@ -95,11 +92,6 @@ def userdesc_parse(env, sig, signode): return x -def functional_directive(name, arguments, options, content, lineno, - content_offset, block_text, state, state_machine): - return [nodes.strong(text='from function: %s' % options['opt'])] - - class ClassDirective(Directive): option_spec = {'opt': lambda x: x} @@ -108,9 +100,11 @@ class ClassDirective(Directive): def setup(app): - app.add_config_value('value_from_conf_py', 42, False) - app.add_directive('funcdir', functional_directive, opt=lambda x: x) + import parsermod + app.add_directive('clsdir', ClassDirective) app.add_object_type('userdesc', 'userdescrole', '%s (userdesc)', userdesc_parse, objname='user desc') app.add_javascript('file://moo.js') + app.add_source_suffix('.foo', 'foo') + app.add_source_parser(parsermod.Parser) diff --git a/tests/roots/test-root/ext.py b/tests/roots/test-root/ext.py deleted file mode 100644 index 5665bea90..000000000 --- a/tests/roots/test-root/ext.py +++ /dev/null @@ -1,5 +0,0 @@ -# Test extension module - - -def setup(app): - app.add_config_value('value_from_ext', [], False) diff --git a/tests/roots/test-root/extapi.txt b/tests/roots/test-root/extapi.txt index 4728e3de1..56be6d8ce 100644 --- a/tests/roots/test-root/extapi.txt +++ b/tests/roots/test-root/extapi.txt @@ -3,8 +3,5 @@ Extension API tests Testing directives: -.. funcdir:: - :opt: Foo - .. clsdir:: :opt: Bar diff --git a/tests/roots/test-root/parsermod.py b/tests/roots/test-root/parsermod.py index dfc699944..de0849c80 100644 --- a/tests/roots/test-root/parsermod.py +++ b/tests/roots/test-root/parsermod.py @@ -3,6 +3,8 @@ from docutils.parsers import Parser class Parser(Parser): + supported = ('foo',) + def parse(self, input, document): section = nodes.section(ids=['id1']) section += nodes.title('Generated section', 'Generated section') diff --git a/tests/roots/test-stylesheets/conf.py b/tests/roots/test-stylesheets/conf.py index 0696e35c2..cae940080 100644 --- a/tests/roots/test-stylesheets/conf.py +++ b/tests/roots/test-stylesheets/conf.py @@ -6,7 +6,7 @@ templates_path = ['_templates'] def setup(app): - app.add_stylesheet('persistent.css') - app.add_stylesheet('default.css', title="Default") - app.add_stylesheet('alternate1.css', title="Alternate", alternate=True) - app.add_stylesheet('alternate2.css', alternate=True) + app.add_css_file('persistent.css') + app.add_css_file('default.css', title="Default") + app.add_css_file('alternate1.css', title="Alternate", rel="alternate stylesheet") + app.add_css_file('alternate2.css', rel="alternate stylesheet") diff --git a/tests/test_application.py b/tests/test_application.py index 075acc80c..efd55c487 100644 --- a/tests/test_application.py +++ b/tests/test_application.py @@ -11,7 +11,7 @@ import pytest from docutils import nodes -from sphinx.application import ExtensionError +from sphinx.errors import ExtensionError from sphinx.domains import Domain from sphinx.testing.util import strip_escseq from sphinx.util import logging @@ -58,34 +58,23 @@ def test_extension_in_blacklist(app, status, warning): assert msg.startswith("WARNING: the extension 'sphinxjp.themecore' was") -def test_domain_override(app, status, warning): - class A(Domain): - name = 'foo' - - class B(A): - name = 'foo' - - class C(Domain): - name = 'foo' - - # No domain know named foo. - with pytest.raises(ExtensionError) as excinfo: - app.override_domain(A) - assert 'domain foo not yet registered' in str(excinfo.value) - - assert app.add_domain(A) is None - assert app.override_domain(B) is None - with pytest.raises(ExtensionError) as excinfo: - app.override_domain(C) - assert 'new domain not a subclass of registered foo domain' in str(excinfo.value) - - @pytest.mark.sphinx(testroot='add_source_parser') def test_add_source_parser(app, status, warning): assert set(app.config.source_suffix) == set(['.rst', '.md', '.test']) - assert set(app.registry.get_source_parsers().keys()) == set(['*', '.md', '.test']) + + # .rst; only in :confval:`source_suffix` + assert '.rst' not in app.registry.get_source_parsers() + assert app.registry.source_suffix['.rst'] is None + + # .md; configured by :confval:`source_suffix` and :confval:`source_parsers` + assert '.md' in app.registry.get_source_parsers() + assert app.registry.source_suffix['.md'] == '.md' assert app.registry.get_source_parsers()['.md'].__name__ == 'DummyMarkdownParser' - assert app.registry.get_source_parsers()['.test'].__name__ == 'TestSourceParser' + + # .test; configured by API + assert app.registry.source_suffix['.test'] == 'test' + assert 'test' in app.registry.get_source_parsers() + assert app.registry.get_source_parsers()['test'].__name__ == 'TestSourceParser' @pytest.mark.sphinx(testroot='extensions') diff --git a/tests/test_build_epub.py b/tests/test_build_epub.py index 2f09f6d5a..24450089b 100644 --- a/tests/test_build_epub.py +++ b/tests/test_build_epub.py @@ -317,6 +317,34 @@ def test_epub_writing_mode(app): assert 'writing-mode: vertical-rl;' in css +@pytest.mark.sphinx('epub', testroot='html_assets') +def test_epub_assets(app): + app.builder.build_all() + + # epub_sytlesheets (same as html_css_files) + content = (app.outdir / 'index.xhtml').text() + assert ('<link rel="stylesheet" type="text/css" href="_static/css/style.css" />' + in content) + assert ('<link media="print" rel="stylesheet" title="title" type="text/css" ' + 'href="https://example.com/custom.css" />' in content) + + +@pytest.mark.sphinx('epub', testroot='html_assets', + confoverrides={'epub_css_files': ['css/epub.css']}) +def test_epub_css_files(app): + app.builder.build_all() + + # epub_css_files + content = (app.outdir / 'index.xhtml').text() + assert '<link rel="stylesheet" type="text/css" href="_static/css/epub.css" />' in content + + # files in html_css_files are not outputed + assert ('<link rel="stylesheet" type="text/css" href="_static/css/style.css" />' + not in content) + assert ('<link media="print" rel="stylesheet" title="title" type="text/css" ' + 'href="https://example.com/custom.css" />' not in content) + + @pytest.mark.sphinx('epub') def test_run_epubcheck(app): app.build() diff --git a/tests/test_build_html.py b/tests/test_build_html.py index bbfd0bc90..2b45720d1 100644 --- a/tests/test_build_html.py +++ b/tests/test_build_html.py @@ -186,7 +186,6 @@ def test_html_warnings(app, warning): (".//dd/p", r'Return spam\.'), ], 'extapi.html': [ - (".//strong", 'from function: Foo'), (".//strong", 'from class: Bar'), ], 'markup.html': [ @@ -1090,14 +1089,19 @@ def test_enumerable_node(app, cached_etree_parse, fname, expect): def test_html_assets(app): app.builder.build_all() + # exclude_path and its family + assert not (app.outdir / 'static' / 'index.html').exists() + assert not (app.outdir / 'extra' / 'index.html').exists() + # html_static_path assert not (app.outdir / '_static' / '.htaccess').exists() assert not (app.outdir / '_static' / '.htpasswd').exists() assert (app.outdir / '_static' / 'API.html').exists() assert (app.outdir / '_static' / 'API.html').text() == 'Sphinx-1.4.4' - assert (app.outdir / '_static' / 'css/style.css').exists() + assert (app.outdir / '_static' / 'css' / 'style.css').exists() + assert (app.outdir / '_static' / 'js' / 'custom.js').exists() assert (app.outdir / '_static' / 'rimg.png').exists() - assert not (app.outdir / '_static' / '_build/index.html').exists() + assert not (app.outdir / '_static' / '_build' / 'index.html').exists() assert (app.outdir / '_static' / 'background.png').exists() assert not (app.outdir / '_static' / 'subdir' / '.htaccess').exists() assert not (app.outdir / '_static' / 'subdir' / '.htpasswd').exists() @@ -1108,11 +1112,17 @@ def test_html_assets(app): assert (app.outdir / 'API.html_t').exists() assert (app.outdir / 'css/style.css').exists() assert (app.outdir / 'rimg.png').exists() - assert not (app.outdir / '_build/index.html').exists() + assert not (app.outdir / '_build' / 'index.html').exists() assert (app.outdir / 'background.png').exists() assert (app.outdir / 'subdir' / '.htaccess').exists() assert not (app.outdir / 'subdir' / '.htpasswd').exists() + # html_css_files + content = (app.outdir / 'index.html').text() + assert '<link rel="stylesheet" type="text/css" href="_static/css/style.css" />' in content + assert ('<link media="print" rel="stylesheet" title="title" type="text/css" ' + 'href="https://example.com/custom.css" />' in content) + @pytest.mark.sphinx('html', testroot='basic', confoverrides={'html_copy_source': False}) def test_html_copy_source(app): diff --git a/tests/test_build_html5.py b/tests/test_build_html5.py index 6f0338113..265c42cbd 100644 --- a/tests/test_build_html5.py +++ b/tests/test_build_html5.py @@ -94,7 +94,6 @@ def cached_etree_parse(): (".//dd/p", r'Return spam\.'), ], 'extapi.html': [ - (".//strong", 'from function: Foo'), (".//strong", 'from class: Bar'), ], 'markup.html': [ diff --git a/tests/test_build_latex.py b/tests/test_build_latex.py index c6aed1e3e..459ac9f1d 100644 --- a/tests/test_build_latex.py +++ b/tests/test_build_latex.py @@ -590,15 +590,8 @@ def test_footnote(app, status, warning): assert ('\\begin{footnote}[2]\\sphinxAtStartFootnote\nauto numbered\n%\n' '\\end{footnote}') in result assert '\\begin{footnote}[3]\\sphinxAtStartFootnote\nnamed\n%\n\\end{footnote}' in result - assert '{\\hyperref[\\detokenize{footnote:bar}]{\\sphinxcrossref{{[}bar{]}}}}' in result - assert ('\\bibitem[bar]{\\detokenize{bar}}' - '{\\phantomsection\\label{\\detokenize{footnote:bar}} ') in result - assert ('\\bibitem[bar]{\\detokenize{bar}}' - '{\\phantomsection\\label{\\detokenize{footnote:bar}} ' - '\ncite') in result - assert ('\\bibitem[bar]{\\detokenize{bar}}' - '{\\phantomsection\\label{\\detokenize{footnote:bar}} ' - '\ncite\n}') in result + assert '\\sphinxcite{footnote:bar}' in result + assert ('\\bibitem[bar]{footnote:bar}\ncite\n') in result assert '\\sphinxcaption{Table caption \\sphinxfootnotemark[4]' in result assert ('\\hline%\n\\begin{footnotetext}[4]\\sphinxAtStartFootnote\n' 'footnote in table caption\n%\n\\end{footnotetext}\\ignorespaces %\n' @@ -620,34 +613,32 @@ def test_reference_in_caption_and_codeblock_in_footnote(app, status, warning): print(status.getvalue()) print(warning.getvalue()) assert ('\\caption{This is the figure caption with a reference to ' - '\\label{\\detokenize{index:id2}}' - '{\\hyperref[\\detokenize{index:authoryear}]' - '{\\sphinxcrossref{{[}AuthorYear{]}}}}.}' in result) + '\\sphinxcite{index:authoryear}.}' in result) assert '\\chapter{The section with a reference to {[}AuthorYear{]}}' in result assert ('\\sphinxcaption{The table title with a reference' ' to {[}AuthorYear{]}}' in result) assert '\\paragraph{The rubric title with a reference to {[}AuthorYear{]}}' in result - assert ('\\chapter{The section with a reference to \\sphinxfootnotemark[4]}\n' + assert ('\\chapter{The section with a reference to \\sphinxfootnotemark[5]}\n' '\\label{\\detokenize{index:the-section-with-a-reference-to}}' - '%\n\\begin{footnotetext}[4]\\sphinxAtStartFootnote\n' + '%\n\\begin{footnotetext}[5]\\sphinxAtStartFootnote\n' 'Footnote in section\n%\n\\end{footnotetext}') in result assert ('\\caption{This is the figure caption with a footnote to ' - '\\sphinxfootnotemark[6].}\\label{\\detokenize{index:id27}}\\end{figure}\n' - '%\n\\begin{footnotetext}[6]\\sphinxAtStartFootnote\n' + '\\sphinxfootnotemark[7].}\\label{\\detokenize{index:id29}}\\end{figure}\n' + '%\n\\begin{footnotetext}[7]\\sphinxAtStartFootnote\n' 'Footnote in caption\n%\n\\end{footnotetext}')in result - assert ('\\sphinxcaption{footnote \\sphinxfootnotemark[7] in ' - 'caption of normal table}\\label{\\detokenize{index:id28}}') in result - assert ('\\caption{footnote \\sphinxfootnotemark[8] ' - 'in caption \\sphinxfootnotemark[9] of longtable\\strut}') in result - assert ('\\endlastfoot\n%\n\\begin{footnotetext}[8]\\sphinxAtStartFootnote\n' + assert ('\\sphinxcaption{footnote \\sphinxfootnotemark[8] in ' + 'caption of normal table}\\label{\\detokenize{index:id30}}') in result + assert ('\\caption{footnote \\sphinxfootnotemark[9] ' + 'in caption \\sphinxfootnotemark[10] of longtable\\strut}') in result + assert ('\\endlastfoot\n%\n\\begin{footnotetext}[9]\\sphinxAtStartFootnote\n' 'Foot note in longtable\n%\n\\end{footnotetext}\\ignorespaces %\n' - '\\begin{footnotetext}[9]\\sphinxAtStartFootnote\n' + '\\begin{footnotetext}[10]\\sphinxAtStartFootnote\n' 'Second footnote in caption of longtable\n') in result assert ('This is a reference to the code-block in the footnote:\n' '{\\hyperref[\\detokenize{index:codeblockinfootnote}]' '{\\sphinxcrossref{\\DUrole{std,std-ref}{I am in a footnote}}}}') in result assert ('&\nThis is one more footnote with some code in it %\n' - '\\begin{footnote}[10]\\sphinxAtStartFootnote\n' + '\\begin{footnote}[11]\\sphinxAtStartFootnote\n' 'Third footnote in longtable\n') in result assert ('\\end{sphinxVerbatim}\n%\n\\end{footnote}.\n') in result assert '\\begin{sphinxVerbatim}[commandchars=\\\\\\{\\}]' in result @@ -666,14 +657,12 @@ def test_latex_show_urls_is_inline(app, status, warning): 'footnote in bar\n%\n\\end{footnote} in bar.rst') in result assert ('Auto footnote number %\n\\begin{footnote}[1]\\sphinxAtStartFootnote\n' 'footnote in baz\n%\n\\end{footnote} in baz.rst') in result - assert ('\\phantomsection\\label{\\detokenize{index:id30}}' + assert ('\\phantomsection\\label{\\detokenize{index:id32}}' '{\\hyperref[\\detokenize{index:the-section' '-with-a-reference-to-authoryear}]' '{\\sphinxcrossref{The section with a reference to ' - '\\phantomsection\\label{\\detokenize{index:id1}}' - '{\\hyperref[\\detokenize{index:authoryear}]' - '{\\sphinxcrossref{{[}AuthorYear{]}}}}}}}') in result - assert ('\\phantomsection\\label{\\detokenize{index:id31}}' + '\\sphinxcite{index:authoryear}}}}') in result + assert ('\\phantomsection\\label{\\detokenize{index:id33}}' '{\\hyperref[\\detokenize{index:the-section-with-a-reference-to}]' '{\\sphinxcrossref{The section with a reference to }}}' in result) assert ('First footnote: %\n\\begin{footnote}[2]\\sphinxAtStartFootnote\n' @@ -682,13 +671,15 @@ def test_latex_show_urls_is_inline(app, status, warning): 'Second\n%\n\\end{footnote}') in result assert '\\sphinxhref{http://sphinx-doc.org/}{Sphinx} (http://sphinx-doc.org/)' in result assert ('Third footnote: %\n\\begin{footnote}[3]\\sphinxAtStartFootnote\n' - 'Third\n%\n\\end{footnote}') in result + 'Third \\sphinxfootnotemark[4]\n%\n\\end{footnote}%\n' + '\\begin{footnotetext}[4]\\sphinxAtStartFootnote\n' + 'Footnote inside footnote\n%\n\\end{footnotetext}\\ignorespaces') in result assert ('\\sphinxhref{http://sphinx-doc.org/~test/}{URL including tilde} ' '(http://sphinx-doc.org/\\textasciitilde{}test/)') in result assert ('\\item[{\\sphinxhref{http://sphinx-doc.org/}{URL in term} ' '(http://sphinx-doc.org/)}] \\leavevmode\nDescription' in result) - assert ('\\item[{Footnote in term \\sphinxfootnotemark[5]}] ' - '\\leavevmode%\n\\begin{footnotetext}[5]\\sphinxAtStartFootnote\n' + assert ('\\item[{Footnote in term \\sphinxfootnotemark[6]}] ' + '\\leavevmode%\n\\begin{footnotetext}[6]\\sphinxAtStartFootnote\n' 'Footnote in term\n%\n\\end{footnotetext}\\ignorespaces \n' 'Description') in result assert ('\\item[{\\sphinxhref{http://sphinx-doc.org/}{Term in deflist} ' @@ -711,13 +702,11 @@ def test_latex_show_urls_is_footnote(app, status, warning): 'footnote in bar\n%\n\\end{footnote} in bar.rst') in result assert ('Auto footnote number %\n\\begin{footnote}[2]\\sphinxAtStartFootnote\n' 'footnote in baz\n%\n\\end{footnote} in baz.rst') in result - assert ('\\phantomsection\\label{\\detokenize{index:id30}}' + assert ('\\phantomsection\\label{\\detokenize{index:id32}}' '{\\hyperref[\\detokenize{index:the-section-with-a-reference-to-authoryear}]' '{\\sphinxcrossref{The section with a reference ' - 'to \\phantomsection\\label{\\detokenize{index:id1}}' - '{\\hyperref[\\detokenize{index:authoryear}]' - '{\\sphinxcrossref{{[}AuthorYear{]}}}}}}}') in result - assert ('\\phantomsection\\label{\\detokenize{index:id31}}' + 'to \\sphinxcite{index:authoryear}}}}') in result + assert ('\\phantomsection\\label{\\detokenize{index:id33}}' '{\\hyperref[\\detokenize{index:the-section-with-a-reference-to}]' '{\\sphinxcrossref{The section with a reference to }}}') in result assert ('First footnote: %\n\\begin{footnote}[3]\\sphinxAtStartFootnote\n' @@ -728,22 +717,25 @@ def test_latex_show_urls_is_footnote(app, status, warning): '%\n\\begin{footnote}[4]\\sphinxAtStartFootnote\n' '\\sphinxnolinkurl{http://sphinx-doc.org/}\n%\n\\end{footnote}') in result assert ('Third footnote: %\n\\begin{footnote}[6]\\sphinxAtStartFootnote\n' - 'Third\n%\n\\end{footnote}') in result + 'Third \\sphinxfootnotemark[7]\n%\n\\end{footnote}%\n' + '\\begin{footnotetext}[7]\\sphinxAtStartFootnote\n' + 'Footnote inside footnote\n%\n' + '\\end{footnotetext}\\ignorespaces') in result assert ('\\sphinxhref{http://sphinx-doc.org/~test/}{URL including tilde}' '%\n\\begin{footnote}[5]\\sphinxAtStartFootnote\n' '\\sphinxnolinkurl{http://sphinx-doc.org/~test/}\n%\n\\end{footnote}') in result assert ('\\item[{\\sphinxhref{http://sphinx-doc.org/}' - '{URL in term}\\sphinxfootnotemark[8]}] ' - '\\leavevmode%\n\\begin{footnotetext}[8]\\sphinxAtStartFootnote\n' + '{URL in term}\\sphinxfootnotemark[9]}] ' + '\\leavevmode%\n\\begin{footnotetext}[9]\\sphinxAtStartFootnote\n' '\\sphinxnolinkurl{http://sphinx-doc.org/}\n%\n' '\\end{footnotetext}\\ignorespaces \nDescription') in result - assert ('\\item[{Footnote in term \\sphinxfootnotemark[10]}] ' - '\\leavevmode%\n\\begin{footnotetext}[10]\\sphinxAtStartFootnote\n' + assert ('\\item[{Footnote in term \\sphinxfootnotemark[11]}] ' + '\\leavevmode%\n\\begin{footnotetext}[11]\\sphinxAtStartFootnote\n' 'Footnote in term\n%\n\\end{footnotetext}\\ignorespaces \n' 'Description') in result assert ('\\item[{\\sphinxhref{http://sphinx-doc.org/}{Term in deflist}' - '\\sphinxfootnotemark[9]}] ' - '\\leavevmode%\n\\begin{footnotetext}[9]\\sphinxAtStartFootnote\n' + '\\sphinxfootnotemark[10]}] ' + '\\leavevmode%\n\\begin{footnotetext}[10]\\sphinxAtStartFootnote\n' '\\sphinxnolinkurl{http://sphinx-doc.org/}\n%\n' '\\end{footnotetext}\\ignorespaces \nDescription') in result assert ('\\sphinxurl{https://github.com/sphinx-doc/sphinx}\n' in result) @@ -764,13 +756,11 @@ def test_latex_show_urls_is_no(app, status, warning): 'footnote in bar\n%\n\\end{footnote} in bar.rst') in result assert ('Auto footnote number %\n\\begin{footnote}[1]\\sphinxAtStartFootnote\n' 'footnote in baz\n%\n\\end{footnote} in baz.rst') in result - assert ('\\phantomsection\\label{\\detokenize{index:id30}}' + assert ('\\phantomsection\\label{\\detokenize{index:id32}}' '{\\hyperref[\\detokenize{index:the-section-with-a-reference-to-authoryear}]' '{\\sphinxcrossref{The section with a reference ' - 'to \\phantomsection\\label{\\detokenize{index:id1}}' - '{\\hyperref[\\detokenize{index:authoryear}]' - '{\\sphinxcrossref{{[}AuthorYear{]}}}}}}}') in result - assert ('\\phantomsection\\label{\\detokenize{index:id31}}' + 'to \\sphinxcite{index:authoryear}}}}') in result + assert ('\\phantomsection\\label{\\detokenize{index:id33}}' '{\\hyperref[\\detokenize{index:the-section-with-a-reference-to}]' '{\\sphinxcrossref{The section with a reference to }}}' in result) assert ('First footnote: %\n\\begin{footnote}[2]\\sphinxAtStartFootnote\n' @@ -779,12 +769,14 @@ def test_latex_show_urls_is_no(app, status, warning): 'Second\n%\n\\end{footnote}') in result assert '\\sphinxhref{http://sphinx-doc.org/}{Sphinx}' in result assert ('Third footnote: %\n\\begin{footnote}[3]\\sphinxAtStartFootnote\n' - 'Third\n%\n\\end{footnote}') in result + 'Third \\sphinxfootnotemark[4]\n%\n\\end{footnote}%\n' + '\\begin{footnotetext}[4]\\sphinxAtStartFootnote\n' + 'Footnote inside footnote\n%\n\\end{footnotetext}\\ignorespaces') in result assert '\\sphinxhref{http://sphinx-doc.org/~test/}{URL including tilde}' in result assert ('\\item[{\\sphinxhref{http://sphinx-doc.org/}{URL in term}}] ' '\\leavevmode\nDescription') in result - assert ('\\item[{Footnote in term \\sphinxfootnotemark[5]}] ' - '\\leavevmode%\n\\begin{footnotetext}[5]\\sphinxAtStartFootnote\n' + assert ('\\item[{Footnote in term \\sphinxfootnotemark[6]}] ' + '\\leavevmode%\n\\begin{footnotetext}[6]\\sphinxAtStartFootnote\n' 'Footnote in term\n%\n\\end{footnotetext}\\ignorespaces \n' 'Description') in result assert ('\\item[{\\sphinxhref{http://sphinx-doc.org/}{Term in deflist}}] ' @@ -1240,3 +1232,16 @@ def test_latex_nested_enumerated_list(app, status, warning): assert r'\setcounter{enumii}{3}' in result assert r'\setcounter{enumiii}{9}' in result assert r'\setcounter{enumii}{2}' in result + + +@pytest.mark.sphinx('latex', testroot='footnotes') +def test_latex_thebibliography(app, status, warning): + app.builder.build_all() + + result = (app.outdir / 'Python.tex').text(encoding='utf8') + print(result) + assert ('\\begin{sphinxthebibliography}{AuthorYe}\n' + '\\bibitem[AuthorYear]{index:authoryear}\n' + 'Author, Title, Year\n' + '\\end{sphinxthebibliography}\n' in result) + assert '\\sphinxcite{index:authoryear}' in result diff --git a/tests/test_builder.py b/tests/test_builder.py new file mode 100644 index 000000000..d58091e8d --- /dev/null +++ b/tests/test_builder.py @@ -0,0 +1,56 @@ +# -*- coding: utf-8 -*- +""" + test_builder + ~~~~~~~~ + + Test the Builder class. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" +import pytest + + +@pytest.mark.sphinx('dummy', srcdir="test_builder") +def test_incremental_reading(app): + # first reading + updated = app.builder.read() + assert set(updated) == app.env.found_docs == set(app.env.all_docs) + + # test if exclude_patterns works ok + assert 'subdir/excluded' not in app.env.found_docs + + # before second reading, add, modify and remove source files + (app.srcdir / 'new.txt').write_text('New file\n========\n') + app.env.all_docs['contents'] = 0 # mark as modified + (app.srcdir / 'autodoc.txt').unlink() + + # second reading + updated = app.builder.read() + + # "includes" and "images" are in there because they contain references + # to nonexisting downloadable or image files, which are given another + # chance to exist + assert set(updated) == set(['contents', 'new', 'includes', 'images']) + assert 'autodoc' not in app.env.all_docs + assert 'autodoc' not in app.env.found_docs + + +@pytest.mark.sphinx('dummy') +def test_env_read_docs(app): + """By default, docnames are read in alphanumeric order""" + def on_env_read_docs_1(app, env, docnames): + pass + + app.connect('env-before-read-docs', on_env_read_docs_1) + + read_docnames = app.builder.read() + assert len(read_docnames) > 2 and read_docnames == sorted(read_docnames) + + def on_env_read_docs_2(app, env, docnames): + docnames.remove('images') + + app.connect('env-before-read-docs', on_env_read_docs_2) + + read_docnames = app.builder.read() + assert len(read_docnames) == 2 diff --git a/tests/test_config.py b/tests/test_config.py index be227819e..e3b79c835 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -11,15 +11,15 @@ """ import mock import pytest -from six import PY3, iteritems +from six import PY3 import sphinx -from sphinx.config import Config +from sphinx.config import Config, ENUM, string_classes, check_confval_types from sphinx.errors import ExtensionError, ConfigError, VersionRequirementError from sphinx.testing.path import path -@pytest.mark.sphinx(confoverrides={ +@pytest.mark.sphinx(testroot='config', confoverrides={ 'master_doc': 'master', 'nonexisting_value': 'True', 'latex_elements.docclass': 'scrartcl', @@ -74,36 +74,64 @@ def test_core_config(app, status, warning): assert cfg['project'] == cfg.project == 'Sphinx Tests' -def test_extension_values(app, status, warning): - cfg = app.config +def test_extension_values(): + config = Config() - # default value - assert cfg.value_from_ext == [] - # non-default value - assert cfg.value_from_conf_py == 84 + # check standard settings + assert config.master_doc == 'contents' - # no duplicate values allowed + # can't override it by add_config_value() with pytest.raises(ExtensionError) as excinfo: - app.add_config_value('html_title', 'x', True) + config.add('master_doc', 'index', 'env', None) assert 'already present' in str(excinfo.value) + + # add a new config value + config.add('value_from_ext', [], 'env', None) + assert config.value_from_ext == [] + + # can't override it by add_config_value() with pytest.raises(ExtensionError) as excinfo: - app.add_config_value('value_from_ext', 'x', True) + config.add('value_from_ext', [], 'env', None) assert 'already present' in str(excinfo.value) +def test_overrides(): + config = Config({'value1': '1', 'value2': 2, 'value6': {'default': 6}}, + {'value2': 999, 'value3': '999', 'value5.attr1': 999, 'value6.attr1': 999, + 'value7': 'abc,def,ghi', 'value8': 'abc,def,ghi'}) + config.add('value1', None, 'env', ()) + config.add('value2', None, 'env', ()) + config.add('value3', 0, 'env', ()) + config.add('value4', 0, 'env', ()) + config.add('value5', {'default': 0}, 'env', ()) + config.add('value6', {'default': 0}, 'env', ()) + config.add('value7', None, 'env', ()) + config.add('value8', [], 'env', ()) + config.init_values() + + assert config.value1 == '1' + assert config.value2 == 999 + assert config.value3 == 999 + assert config.value4 == 0 + assert config.value5 == {'attr1': 999} + assert config.value6 == {'default': 6, 'attr1': 999} + assert config.value7 == 'abc,def,ghi' + assert config.value8 == ['abc', 'def', 'ghi'] + + @mock.patch("sphinx.config.logger") def test_errors_warnings(logger, tempdir): # test the error for syntax errors in the config file (tempdir / 'conf.py').write_text(u'project = \n', encoding='ascii') with pytest.raises(ConfigError) as excinfo: - Config(tempdir, 'conf.py', {}, None) + Config.read(tempdir, {}, None) assert 'conf.py' in str(excinfo.value) # test the automatic conversion of 2.x only code in configs (tempdir / 'conf.py').write_text( u'# -*- coding: utf-8\n\nproject = u"Jägermeister"\n', encoding='utf-8') - cfg = Config(tempdir, 'conf.py', {}, None) + cfg = Config.read(tempdir, {}, None) cfg.init_values() assert cfg.project == u'Jägermeister' assert logger.called is False @@ -115,7 +143,7 @@ def test_errors_warnings(logger, tempdir): return (tempdir / 'conf.py').write_text( u'# -*- coding: latin-1\nproject = "fooä"\n', encoding='latin-1') - cfg = Config(tempdir, 'conf.py', {}, None) + cfg = Config.read(tempdir, {}, None) assert logger.warning.called is False cfg.check_unicode() @@ -174,7 +202,7 @@ def test_config_eol(logger, tempdir): configfile = tempdir / 'conf.py' for eol in (b'\n', b'\r\n'): configfile.write_bytes(b'project = "spam"' + eol) - cfg = Config(tempdir, 'conf.py', {}, None) + cfg = Config.read(tempdir, {}, None) cfg.init_values() assert cfg.project == u'spam' assert logger.called is False @@ -195,60 +223,81 @@ def test_builtin_conf(app, status, warning): 'warning') -# See roots/test-config/conf.py. -TYPECHECK_WARNINGS = { - 'value1': True, - 'value2': True, - 'value3': False, - 'value4': True, - 'value5': False, - 'value6': True, - 'value7': False, - 'value8': False, - 'value9': False, - 'value10': False, - 'value11': False if PY3 else True, - 'value12': False, - 'value13': False, - 'value14': False, - 'value15': False, - 'value16': False, -} - - -@pytest.mark.parametrize("key,should", iteritems(TYPECHECK_WARNINGS)) -@pytest.mark.sphinx(testroot='config') -def test_check_types(warning, key, should): - warn = warning.getvalue() - if should: - assert key in warn, ( - 'override on "%s" should raise a type warning' % key - ) - else: - assert key not in warn, ( - 'override on "%s" should NOT raise a type warning' % key - ) - - -@pytest.mark.sphinx(testroot='config') -def test_check_enum(app, status, warning): - assert "The config value `value17` has to be a one of ('default', 'one', 'two'), " \ - not in warning.getvalue() - - -@pytest.mark.sphinx(testroot='config', confoverrides={'value17': 'invalid'}) -def test_check_enum_failed(app, status, warning): - assert "The config value `value17` has to be a one of ('default', 'one', 'two'), " \ - "but `invalid` is given." in warning.getvalue() - - -@pytest.mark.sphinx(testroot='config', confoverrides={'value17': ['one', 'two']}) -def test_check_enum_for_list(app, status, warning): - assert "The config value `value17` has to be a one of ('default', 'one', 'two'), " \ - not in warning.getvalue() - - -@pytest.mark.sphinx(testroot='config', confoverrides={'value17': ['one', 'two', 'invalid']}) -def test_check_enum_for_list_failed(app, status, warning): - assert "The config value `value17` has to be a one of ('default', 'one', 'two'), " \ - "but `['one', 'two', 'invalid']` is given." in warning.getvalue() +# example classes for type checking +class A(object): + pass + + +class B(A): + pass + + +class C(A): + pass + + +# name, default, annotation, actual, warned +TYPECHECK_WARNINGS = [ + ('value1', 'string', None, 123, True), # wrong type + ('value2', lambda _: [], None, 123, True), # lambda with wrong type + ('value3', lambda _: [], None, [], False), # lambda with correct type + ('value4', 100, None, True, True), # child type + ('value5', False, None, True, False), # parent type + ('value6', [], None, (), True), # other sequence type + ('value7', 'string', [list], ['foo'], False), # explicit type annotation + ('value8', B(), None, C(), False), # sibling type + ('value9', None, None, 'foo', False), # no default or no annotations + ('value10', None, None, 123, False), # no default or no annotations + ('value11', None, [str], u'bar', False if PY3 else True), # str vs unicode + ('value12', 'string', None, u'bar', False), # str vs unicode + ('value13', None, string_classes, 'bar', False), # string_classes + ('value14', None, string_classes, u'bar', False), # string_classes + ('value15', u'unicode', None, 'bar', False), # str vs unicode + ('value16', u'unicode', None, u'bar', False), # str vs unicode +] + + +@mock.patch("sphinx.config.logger") +@pytest.mark.parametrize("name,default,annotation,actual,warned", TYPECHECK_WARNINGS) +def test_check_types(logger, name, default, annotation, actual, warned): + config = Config({name: actual}) + config.add(name, default, 'env', annotation or ()) + config.init_values() + check_confval_types(None, config) + assert logger.warning.called == warned + + +@mock.patch("sphinx.config.logger") +def test_check_enum(logger): + config = Config() + config.add('value', 'default', False, ENUM('default', 'one', 'two')) + config.init_values() + check_confval_types(None, config) + logger.warning.assert_not_called() # not warned + + +@mock.patch("sphinx.config.logger") +def test_check_enum_failed(logger): + config = Config({'value': 'invalid'}) + config.add('value', 'default', False, ENUM('default', 'one', 'two')) + config.init_values() + check_confval_types(None, config) + logger.warning.assert_called() + + +@mock.patch("sphinx.config.logger") +def test_check_enum_for_list(logger): + config = Config({'value': ['one', 'two']}) + config.add('value', 'default', False, ENUM('default', 'one', 'two')) + config.init_values() + check_confval_types(None, config) + logger.warning.assert_not_called() # not warned + + +@mock.patch("sphinx.config.logger") +def test_check_enum_for_list_failed(logger): + config = Config({'value': ['one', 'two', 'invalid']}) + config.add('value', 'default', False, ENUM('default', 'one', 'two')) + config.init_values() + check_confval_types(None, config) + logger.warning.assert_called() diff --git a/tests/test_directive_code.py b/tests/test_directive_code.py index 5b48b1d61..cb4a1ef22 100644 --- a/tests/test_directive_code.py +++ b/tests/test_directive_code.py @@ -17,7 +17,7 @@ from sphinx.config import Config from sphinx.directives.code import LiteralIncludeReader from sphinx.testing.util import etree_parse -DUMMY_CONFIG = Config(None, None, {}, '') +DUMMY_CONFIG = Config({}, {}) @pytest.fixture(scope='module') diff --git a/tests/test_docutilsconf.py b/tests/test_docutilsconf.py index 2332dcf50..989edc6a8 100644 --- a/tests/test_docutilsconf.py +++ b/tests/test_docutilsconf.py @@ -15,6 +15,7 @@ import sys import pytest from sphinx.testing.path import path +from sphinx.util.docutils import patch_docutils def regex_count(expr, result): @@ -23,7 +24,9 @@ def regex_count(expr, result): @pytest.mark.sphinx('html', testroot='docutilsconf', freshenv=True, docutilsconf='') def test_html_with_default_docutilsconf(app, status, warning): - app.builder.build(['contents']) + with patch_docutils(app.confdir): + app.builder.build(['contents']) + result = (app.outdir / 'contents.html').text(encoding='utf-8') assert regex_count(r'<th class="field-name">', result) == 1 @@ -39,7 +42,9 @@ def test_html_with_default_docutilsconf(app, status, warning): '\n') ) def test_html_with_docutilsconf(app, status, warning): - app.builder.build(['contents']) + with patch_docutils(app.confdir): + app.builder.build(['contents']) + result = (app.outdir / 'contents.html').text(encoding='utf-8') assert regex_count(r'<th class="field-name">', result) == 0 @@ -50,25 +55,29 @@ def test_html_with_docutilsconf(app, status, warning): @pytest.mark.sphinx('html', testroot='docutilsconf') def test_html(app, status, warning): - app.builder.build(['contents']) + with patch_docutils(app.confdir): + app.builder.build(['contents']) assert warning.getvalue() == '' @pytest.mark.sphinx('latex', testroot='docutilsconf') def test_latex(app, status, warning): - app.builder.build(['contents']) + with patch_docutils(app.confdir): + app.builder.build(['contents']) assert warning.getvalue() == '' @pytest.mark.sphinx('man', testroot='docutilsconf') def test_man(app, status, warning): - app.builder.build(['contents']) + with patch_docutils(app.confdir): + app.builder.build(['contents']) assert warning.getvalue() == '' @pytest.mark.sphinx('texinfo', testroot='docutilsconf') def test_texinfo(app, status, warning): - app.builder.build(['contents']) + with patch_docutils(app.confdir): + app.builder.build(['contents']) @pytest.mark.sphinx('html', testroot='docutilsconf', @@ -87,4 +96,5 @@ def test_docutils_source_link_with_nonascii_file(app, status, warning): 'nonascii filename not supported on this filesystem encoding: ' '%s', FILESYSTEMENCODING) - app.builder.build_all() + with patch_docutils(app.confdir): + app.builder.build_all() diff --git a/tests/test_domain_std.py b/tests/test_domain_std.py index dce7a5ddf..57d0bf185 100644 --- a/tests/test_domain_std.py +++ b/tests/test_domain_std.py @@ -17,6 +17,7 @@ from sphinx.domains.std import StandardDomain def test_process_doc_handle_figure_caption(): env = mock.Mock(domaindata={}) + env.app.registry.enumerable_nodes = {} figure_node = nodes.figure( '', nodes.caption('caption text', 'caption text'), @@ -40,6 +41,7 @@ def test_process_doc_handle_figure_caption(): def test_process_doc_handle_table_title(): env = mock.Mock(domaindata={}) + env.app.registry.enumerable_nodes = {} table_node = nodes.table( '', nodes.title('title text', 'title text'), @@ -63,6 +65,7 @@ def test_process_doc_handle_table_title(): def test_get_full_qualified_name(): env = mock.Mock(domaindata={}) + env.app.registry.enumerable_nodes = {} domain = StandardDomain(env) # normal references diff --git a/tests/test_environment.py b/tests/test_environment.py index 11364e89f..1ab60b539 100644 --- a/tests/test_environment.py +++ b/tests/test_environment.py @@ -12,38 +12,15 @@ import pytest from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.builders.latex import LaTeXBuilder -from sphinx.testing.util import SphinxTestApp, path -app = env = None - -@pytest.fixture(scope='module', autouse=True) -def setup_module(rootdir, sphinx_test_tempdir): - global app, env - srcdir = sphinx_test_tempdir / 'root-envtest' - if not srcdir.exists(): - (rootdir / 'test-root').copytree(srcdir) - app = SphinxTestApp(srcdir=srcdir) - env = app.env - yield - app.cleanup() - - -# Tests are run in the order they appear in the file, therefore we can -# afford to not run update() in the setup but in its own test - -def test_first_update(): - updated = env.update(app.config, app.srcdir, app.doctreedir) - assert set(updated) == env.found_docs == set(env.all_docs) - # test if exclude_patterns works ok - assert 'subdir/excluded' not in env.found_docs - - -def test_images(): +@pytest.mark.sphinx('dummy') +def test_images(app): + app.build() assert ('image file not readable: foo.png' in app._warning.getvalue()) - tree = env.get_doctree('images') + tree = app.env.get_doctree('images') htmlbuilder = StandaloneHTMLBuilder(app) htmlbuilder.set_environment(app.env) htmlbuilder.init() @@ -67,44 +44,10 @@ def test_images(): 'svgimg.pdf', 'img.foo.png']) -def test_second_update(): - # delete, add and "edit" (change saved mtime) some files and update again - env.all_docs['contents'] = 0 - root = path(app.srcdir) - # important: using "autodoc" because it is the last one to be included in - # the contents.txt toctree; otherwise section numbers would shift - (root / 'autodoc.txt').unlink() - (root / 'new.txt').write_text('New file\n========\n') - updated = env.update(app.config, app.srcdir, app.doctreedir) - # "includes" and "images" are in there because they contain references - # to nonexisting downloadable or image files, which are given another - # chance to exist - assert set(updated) == set(['contents', 'new', 'includes', 'images']) - assert 'autodoc' not in env.all_docs - assert 'autodoc' not in env.found_docs - - -def test_env_read_docs(): - """By default, docnames are read in alphanumeric order""" - def on_env_read_docs_1(app, env, docnames): - pass - - app.connect('env-before-read-docs', on_env_read_docs_1) - - read_docnames = env.update(app.config, app.srcdir, app.doctreedir) - assert len(read_docnames) > 2 and read_docnames == sorted(read_docnames) - - def on_env_read_docs_2(app, env, docnames): - docnames.remove('images') - - app.connect('env-before-read-docs', on_env_read_docs_2) - - read_docnames = env.update(app.config, app.srcdir, app.doctreedir) - assert len(read_docnames) == 2 - - -def test_object_inventory(): - refs = env.domaindata['py']['objects'] +@pytest.mark.sphinx('dummy') +def test_object_inventory(app): + app.build() + refs = app.env.domaindata['py']['objects'] assert 'func_without_module' in refs assert refs['func_without_module'] == ('objects', 'function') @@ -121,8 +64,8 @@ def test_object_inventory(): assert 'func_in_module' not in refs assert 'func_noindex' not in refs - assert env.domaindata['py']['modules']['mod'] == \ + assert app.env.domaindata['py']['modules']['mod'] == \ ('objects', 'Module synopsis.', 'UNIX', False) - assert env.domains['py'].data is env.domaindata['py'] - assert env.domains['c'].data is env.domaindata['c'] + assert app.env.domains['py'].data is app.env.domaindata['py'] + assert app.env.domains['c'].data is app.env.domaindata['c'] diff --git a/tests/test_ext_doctest.py b/tests/test_ext_doctest.py index d19d5b4e5..4aad0d274 100644 --- a/tests/test_ext_doctest.py +++ b/tests/test_ext_doctest.py @@ -8,6 +8,7 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ +import os import pytest from packaging.specifiers import InvalidSpecifier from packaging.version import InvalidVersion @@ -68,7 +69,7 @@ def test_reporting_with_autodoc(app, status, warning, capfd): written = [] app.builder._warn_out = written.append app.builder.build_all() - lines = '\n'.join(written).split('\n') + lines = '\n'.join(written).replace(os.sep, '/').split('\n') failures = [l for l in lines if l.startswith('File')] expected = [ 'File "dir/inner.rst", line 1, in default', diff --git a/tests/test_ext_graphviz.py b/tests/test_ext_graphviz.py index 3efab6b56..9b53876db 100644 --- a/tests/test_ext_graphviz.py +++ b/tests/test_ext_graphviz.py @@ -22,24 +22,26 @@ def test_graphviz_png_html(app, status, warning): app.builder.build_all() content = (app.outdir / 'index.html').text() - html = (r'<div class="figure" .*?>\s*<img .*?/>\s*<p class="caption">' + html = (r'<div class="figure" .*?>\s*' + r'<div class="graphviz"><img .*?/></div>\s*<p class="caption">' r'<span class="caption-text">caption of graph</span>.*</p>\s*</div>') assert re.search(html, content, re.S) - html = 'Hello <img .*?/>\n graphviz world' + html = 'Hello <div class="graphviz"><img .*?/></div>\n graphviz world' assert re.search(html, content, re.S) html = '<img src=".*?" alt="digraph {\n bar -> baz\n}" />' assert re.search(html, content, re.M) - html = (r'<div class="figure align-right" .*?>\s*<img .*?/>\s*<p class="caption">' + html = (r'<div class="figure align-right" .*?>\s*' + r'<div class="graphviz"><img .*?/></div>\s*<p class="caption">' r'<span class="caption-text">on right</span>.*</p>\s*</div>') assert re.search(html, content, re.S) html = (r'<div align=\"center\" class=\"align-center\">' - r'<img src=\".*\.png\" alt=\"digraph foo {\n' + r'<div class="graphviz"><img src=\".*\.png\" alt=\"digraph foo {\n' r'centered\n' - r'}\" />\n</div>') + r'}\" /></div>\n</div>') assert re.search(html, content, re.S) @@ -52,34 +54,34 @@ def test_graphviz_svg_html(app, status, warning): content = (app.outdir / 'index.html').text() html = (r'<div class=\"figure\" .*?>\n' - r'<object data=\".*\.svg\".*>\n' - r'\s+<p class=\"warning\">digraph foo {\n' + r'<div class="graphviz"><object data=\".*\.svg\".*>\n' + r'\s*<p class=\"warning\">digraph foo {\n' r'bar -> baz\n' - r'}</p></object>\n' + r'}</p></object></div>\n' r'<p class=\"caption\"><span class=\"caption-text\">' r'caption of graph</span>.*</p>\n</div>') assert re.search(html, content, re.S) - html = (r'Hello <object.*>\n' - r'\s+<p class=\"warning\">graph</p></object>\n' + html = (r'Hello <div class="graphviz"><object.*>\n' + r'\s*<p class=\"warning\">graph</p></object></div>\n' r' graphviz world') assert re.search(html, content, re.S) html = (r'<div class=\"figure align-right\" .*\>\n' - r'<object data=\".*\.svg\".*>\n' - r'\s+<p class=\"warning\">digraph bar {\n' + r'<div class="graphviz"><object data=\".*\.svg\".*>\n' + r'\s*<p class=\"warning\">digraph bar {\n' r'foo -> bar\n' - r'}</p></object>\n' + r'}</p></object></div>\n' r'<p class=\"caption\"><span class=\"caption-text\">' r'on right</span>.*</p>\n' r'</div>') assert re.search(html, content, re.S) html = (r'<div align=\"center\" class=\"align-center\">' - r'<object data=\".*\.svg\".*>\n' - r'\s+<p class=\"warning\">digraph foo {\n' + r'<div class="graphviz"><object data=\".*\.svg\".*>\n' + r'\s*<p class=\"warning\">digraph foo {\n' r'centered\n' - r'}</p></object>\n' + r'}</p></object></div>\n' r'</div>') assert re.search(html, content, re.S) diff --git a/tests/test_ext_inheritance_diagram.py b/tests/test_ext_inheritance_diagram.py index ad106e6c6..8456cda4a 100644 --- a/tests/test_ext_inheritance_diagram.py +++ b/tests/test_ext_inheritance_diagram.py @@ -19,14 +19,34 @@ from sphinx.ext.inheritance_diagram import InheritanceException, import_classes @pytest.mark.sphinx('html', testroot='ext-inheritance_diagram') @pytest.mark.usefixtures('if_graphviz_found') -def test_inheritance_diagram_html(app, status, warning): +def test_inheritance_diagram_png_html(app, status, warning): app.builder.build_all() content = (app.outdir / 'index.html').text() pattern = ('<div class="figure" id="id1">\n' + '<div class="graphviz">' '<img src="_images/inheritance-\\w+.png" alt="Inheritance diagram of test.Foo" ' - 'class="inheritance"/>\n<p class="caption"><span class="caption-text">' + 'class="inheritance"/></div>\n<p class="caption"><span class="caption-text">' + 'Test Foo!</span><a class="headerlink" href="#id1" ' + 'title="Permalink to this image">\xb6</a></p>') + assert re.search(pattern, content, re.M) + + +@pytest.mark.sphinx('html', testroot='ext-inheritance_diagram', + confoverrides={'graphviz_output_format': 'svg'}) +@pytest.mark.usefixtures('if_graphviz_found') +def test_inheritance_diagram_svg_html(app, status, warning): + app.builder.build_all() + + content = (app.outdir / 'index.html').text() + + pattern = ('<div class="figure" id="id1">\n' + '<div class="graphviz">' + '<object data="_images/inheritance-\\w+.svg" ' + 'type="image/svg\\+xml" class="inheritance">\n' + '<p class=\"warning\">Inheritance diagram of test.Foo</p>' + '</object></div>\n<p class="caption"><span class="caption-text">' 'Test Foo!</span><a class="headerlink" href="#id1" ' 'title="Permalink to this image">\xb6</a></p>') assert re.search(pattern, content, re.M) @@ -62,8 +82,9 @@ def test_inheritance_diagram_latex_alias(app, status, warning): content = (app.outdir / 'index.html').text() pattern = ('<div class="figure" id="id1">\n' + '<div class="graphviz">' '<img src="_images/inheritance-\\w+.png" alt="Inheritance diagram of test.Foo" ' - 'class="inheritance"/>\n<p class="caption"><span class="caption-text">' + 'class="inheritance"/></div>\n<p class="caption"><span class="caption-text">' 'Test Foo!</span><a class="headerlink" href="#id1" ' 'title="Permalink to this image">\xb6</a></p>') assert re.search(pattern, content, re.M) diff --git a/tests/test_ext_napoleon_docstring.py b/tests/test_ext_napoleon_docstring.py index 5d0cbcf39..a4d127d0d 100644 --- a/tests/test_ext_napoleon_docstring.py +++ b/tests/test_ext_napoleon_docstring.py @@ -57,15 +57,21 @@ Sample namedtuple subclass .. attribute:: attr1 - *Arbitrary type* -- Quick description of attr1 + Quick description of attr1 + + :type: Arbitrary type .. attribute:: attr2 - *Another arbitrary type* -- Quick description of attr2 + Quick description of attr2 + + :type: Another arbitrary type .. attribute:: attr3 - *Type* -- Adds a newline after the type + Adds a newline after the type + + :type: Type """ self.assertEqual(expected, actual) @@ -265,6 +271,44 @@ class GoogleDocstringTest(BaseDocstringTest): """ )] + def test_sphinx_admonitions(self): + admonition_map = { + 'Attention': 'attention', + 'Caution': 'caution', + 'Danger': 'danger', + 'Error': 'error', + 'Hint': 'hint', + 'Important': 'important', + 'Note': 'note', + 'Tip': 'tip', + 'Todo': 'todo', + 'Warning': 'warning', + 'Warnings': 'warning', + } + config = Config() + for section, admonition in admonition_map.items(): + # Multiline + actual = str(GoogleDocstring(("{}:\n" + " this is the first line\n" + "\n" + " and this is the second line\n" + ).format(section), config)) + expect = (".. {}::\n" + "\n" + " this is the first line\n" + " \n" + " and this is the second line\n" + ).format(admonition) + self.assertEqual(expect, actual) + + # Single line + actual = str(GoogleDocstring(("{}:\n" + " this is a single line\n" + ).format(section), config)) + expect = (".. {}:: this is a single line\n" + ).format(admonition) + self.assertEqual(expect, actual) + def test_docstrings(self): config = Config( napoleon_use_param=False, @@ -322,7 +366,9 @@ Attributes: expected = """\ .. attribute:: in_attr - :class:`numpy.ndarray` -- super-dooper attribute + super-dooper attribute + + :type: :class:`numpy.ndarray` """ self.assertEqual(expected, actual) @@ -335,7 +381,9 @@ Attributes: expected = """\ .. attribute:: in_attr - *numpy.ndarray* -- super-dooper attribute + super-dooper attribute + + :type: numpy.ndarray """ self.assertEqual(expected, actual) @@ -916,6 +964,28 @@ Parameters: actual = str(GoogleDocstring(docstring, config)) self.assertEqual(expected, actual) + def test_custom_generic_sections(self): + + docstrings = (("""\ +Really Important Details: + You should listen to me! +""", """.. rubric:: Really Important Details + +You should listen to me! +"""), + ("""\ +Sooper Warning: + Stop hitting yourself! +""", """:Warns: **Stop hitting yourself!** +""")) + + testConfig = Config(napoleon_custom_sections=['Really Important Details', + ('Sooper Warning', 'warns')]) + + for docstring, expected in docstrings: + actual = str(GoogleDocstring(docstring, testConfig)) + self.assertEqual(expected, actual) + class NumpyDocstringTest(BaseDocstringTest): docstrings = [( @@ -1070,6 +1140,46 @@ class NumpyDocstringTest(BaseDocstringTest): """ )] + def test_sphinx_admonitions(self): + admonition_map = { + 'Attention': 'attention', + 'Caution': 'caution', + 'Danger': 'danger', + 'Error': 'error', + 'Hint': 'hint', + 'Important': 'important', + 'Note': 'note', + 'Tip': 'tip', + 'Todo': 'todo', + 'Warning': 'warning', + 'Warnings': 'warning', + } + config = Config() + for section, admonition in admonition_map.items(): + # Multiline + actual = str(NumpyDocstring(("{}\n" + "{}\n" + " this is the first line\n" + "\n" + " and this is the second line\n" + ).format(section, '-' * len(section)), config)) + expect = (".. {}::\n" + "\n" + " this is the first line\n" + " \n" + " and this is the second line\n" + ).format(admonition) + self.assertEqual(expect, actual) + + # Single line + actual = str(NumpyDocstring(("{}\n" + "{}\n" + " this is a single line\n" + ).format(section, '-' * len(section)), config)) + expect = (".. {}:: this is a single line\n" + ).format(admonition) + self.assertEqual(expect, actual) + def test_docstrings(self): config = Config( napoleon_use_param=False, diff --git a/tests/test_ext_viewcode.py b/tests/test_ext_viewcode.py index 3b7dbdafc..4676f488f 100644 --- a/tests/test_ext_viewcode.py +++ b/tests/test_ext_viewcode.py @@ -60,3 +60,46 @@ def test_linkcode(app, status, warning): assert 'http://foobar/js/' in stuff assert 'http://foobar/c/' in stuff assert 'http://foobar/cpp/' in stuff + + +@pytest.mark.sphinx(testroot='ext-viewcode-find') +def test_local_source_files(app, status, warning): + def find_source(app, modname): + if modname == 'not_a_package': + source = (app.srcdir / 'not_a_package/__init__.py').text() + tags = { + 'func1': ('def', 3, 3), + 'Class1': ('class', 3, 3), + 'not_a_package.submodule.func1': ('def', 3, 3), + 'not_a_package.submodule.Class1': ('class', 3, 3), + } + else: + source = (app.srcdir / 'not_a_package/submodule.py').text() + tags = { + 'not_a_package.submodule.func1': ('def', 11, 15), + 'Class1': ('class', 19, 22), + 'not_a_package.submodule.Class1': ('class', 19, 22), + 'Class3': ('class', 25, 30), + 'not_a_package.submodule.Class3.class_attr': ('other', 29, 29), + } + return (source, tags) + + app.connect('viewcode-find-source', find_source) + app.builder.build_all() + + warnings = re.sub(r'\\+', '/', warning.getvalue()) + assert re.findall( + r"index.rst:\d+: WARNING: Object named 'func1' not found in include " + + r"file .*/not_a_package/__init__.py'", + warnings + ) + + result = (app.outdir / 'index.html').text(encoding='utf-8') + assert result.count('href="_modules/not_a_package.html#func1"') == 1 + assert result.count('href="_modules/not_a_package.html#not_a_package.submodule.func1"') == 1 + assert result.count('href="_modules/not_a_package/submodule.html#Class1"') == 1 + assert result.count('href="_modules/not_a_package/submodule.html#Class3"') == 1 + assert result.count('href="_modules/not_a_package/submodule.html#not_a_package.submodule.Class1"') == 1 + + assert result.count('href="_modules/not_a_package/submodule.html#not_a_package.submodule.Class3.class_attr"') == 1 + assert result.count('This is the class attribute class_attr') == 1 diff --git a/tests/test_intl.py b/tests/test_intl.py index 85b65dab3..11e14ef84 100644 --- a/tests/test_intl.py +++ b/tests/test_intl.py @@ -588,7 +588,8 @@ def test_gettext_buildr_ignores_only_directive(app): def test_gettext_dont_rebuild_mo(make_app, app_params, build_mo): # --- don't rebuild by .mo mtime def get_number_of_update_targets(app_): - updated = app_.env.update(app_.config, app_.srcdir, app_.doctreedir) + app_.env.find_files(app_.config, app_.builder) + _, updated, _ = app_.env.get_outdated_files(config_changed=False) return len(updated) args, kwargs = app_params @@ -769,12 +770,14 @@ def test_html_rebuild_mo(app): app.build() # --- rebuild by .mo mtime app.builder.build_update() - updated = app.env.update(app.config, app.srcdir, app.doctreedir) + app.env.find_files(app.config, app.builder) + _, updated, _ = app.env.get_outdated_files(config_changed=False) assert len(updated) == 0 mtime = (app.srcdir / 'xx' / 'LC_MESSAGES' / 'bom.mo').stat().st_mtime (app.srcdir / 'xx' / 'LC_MESSAGES' / 'bom.mo').utime((mtime + 5, mtime + 5)) - updated = app.env.update(app.config, app.srcdir, app.doctreedir) + app.env.find_files(app.config, app.builder) + _, updated, _ = app.env.get_outdated_files(config_changed=False) assert len(updated) == 1 diff --git a/tests/test_locale.py b/tests/test_locale.py new file mode 100644 index 000000000..9b1921bd6 --- /dev/null +++ b/tests/test_locale.py @@ -0,0 +1,66 @@ +# -*- coding: utf-8 -*- +""" + test_locale + ~~~~~~~~~~ + + Test locale. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import pytest + +from sphinx import locale + + +@pytest.fixture(autouse=True) +def cleanup_translations(): + yield + locale.translators.clear() + + +def test_init(rootdir): + # not initialized yet + _ = locale.get_translation('myext') + assert _('Hello world') == 'Hello world' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' + + # load locale1 + locale.init([rootdir / 'test-locale' / 'locale1'], 'en', 'myext') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' + + # load a catalog to unrelated namespace + locale.init([rootdir / 'test-locale' / 'locale2'], 'en', 'myext', 'mynamespace') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'Hello sphinx' # nothing changed here + assert _('Hello reST') == 'Hello reST' + + # load locale2 in addition + locale.init([rootdir / 'test-locale' / 'locale2'], 'en', 'myext') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'HELLO SPHINX' + assert _('Hello reST') == 'Hello reST' + + +def test_init_with_unknown_language(rootdir): + locale.init([rootdir / 'test-locale' / 'locale1'], 'unknown', 'myext') + _ = locale.get_translation('myext') + assert _('Hello world') == 'Hello world' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' + + +def test_add_message_catalog(app, rootdir): + app.config.language = 'en' + app.add_message_catalog('myext', rootdir / 'test-locale' / 'locale1') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' diff --git a/tests/test_markup.py b/tests/test_markup.py index 37a1a721b..802202639 100644 --- a/tests/test_markup.py +++ b/tests/test_markup.py @@ -162,12 +162,20 @@ def get_verifier(verify, verify_re): '\\sphinxmenuselection{a \\(\\rightarrow\\) b}', ), ( - # interpolation of ampersands in guilabel/menuselection + # interpolation of ampersands in menuselection + 'verify', + ':menuselection:`&Foo -&&- &Bar`', + (u'<p><span class="menuselection"><span class="accelerator">F</span>oo ' + '-&- <span class="accelerator">B</span>ar</span></p>'), + r'\sphinxmenuselection{\sphinxaccelerator{F}oo -\&- \sphinxaccelerator{B}ar}', + ), + ( + # interpolation of ampersands in guilabel 'verify', ':guilabel:`&Foo -&&- &Bar`', (u'<p><span class="guilabel"><span class="accelerator">F</span>oo ' '-&- <span class="accelerator">B</span>ar</span></p>'), - r'\sphinxmenuselection{\sphinxaccelerator{F}oo -\&- \sphinxaccelerator{B}ar}', + r'\sphinxguilabel{\sphinxaccelerator{F}oo -\&- \sphinxaccelerator{B}ar}', ), ( # non-interpolation of dashes in option role diff --git a/tests/test_roles.py b/tests/test_roles.py new file mode 100644 index 000000000..13b4e194d --- /dev/null +++ b/tests/test_roles.py @@ -0,0 +1,81 @@ +# -*- coding: utf-8 -*- +""" + test_roles + ~~~~~~~~~~ + + Test sphinx.roles + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from docutils import nodes +from mock import Mock + +from sphinx.roles import emph_literal_role +from sphinx.testing.util import assert_node + + +def test_samp(): + # normal case + text = 'print 1+{variable}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, ("print 1+", + [nodes.emphasis, "variable"])],)) + assert msg == [] + + # two emphasis items + text = 'print {1}+{variable}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, ("print ", + [nodes.emphasis, "1"], + "+", + [nodes.emphasis, "variable"])],)) + assert msg == [] + + # empty curly brace + text = 'print 1+{}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, "print 1+{}"],)) + assert msg == [] + + # half-opened variable + text = 'print 1+{variable' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, "print 1+{variable"],)) + assert msg == [] + + # nested + text = 'print 1+{{variable}}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, ("print 1+", + [nodes.emphasis, "{variable"], + "}")],)) + assert msg == [] + + # emphasized item only + text = '{variable}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, nodes.emphasis, "variable"],)) + assert msg == [] + + # escaping + text = r'print 1+\{variable}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, "print 1+{variable}"],)) + assert msg == [] + + # escaping (2) + text = r'print 1+\{{variable}\}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, ("print 1+{", + [nodes.emphasis, "variable"], + "}")],)) + assert msg == [] + + # escape a backslash + text = r'print 1+\\{variable}' + ret, msg = emph_literal_role('samp', text, text, 0, Mock()) + assert_node(ret, ([nodes.literal, ("print 1+\\", + [nodes.emphasis, "variable"])],)) + assert msg == [] diff --git a/tests/test_util_docutils.py b/tests/test_util_docutils.py new file mode 100644 index 000000000..31a1d9bd2 --- /dev/null +++ b/tests/test_util_docutils.py @@ -0,0 +1,34 @@ +# -*- coding: utf-8 -*- +""" + test_util_docutils + ~~~~~~~~~~~~~~~~~~ + + Tests util.utils functions. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from docutils import nodes + +from sphinx.util.docutils import docutils_namespace, register_node + + +def test_register_node(): + class custom_node(nodes.Element): + pass + + with docutils_namespace(): + register_node(custom_node) + + # check registered + assert hasattr(nodes.GenericNodeVisitor, 'visit_custom_node') + assert hasattr(nodes.GenericNodeVisitor, 'depart_custom_node') + assert hasattr(nodes.SparseNodeVisitor, 'visit_custom_node') + assert hasattr(nodes.SparseNodeVisitor, 'depart_custom_node') + + # check unregistered outside namespace + assert not hasattr(nodes.GenericNodeVisitor, 'visit_custom_node') + assert not hasattr(nodes.GenericNodeVisitor, 'depart_custom_node') + assert not hasattr(nodes.SparseNodeVisitor, 'visit_custom_node') + assert not hasattr(nodes.SparseNodeVisitor, 'depart_custom_node') diff --git a/tests/test_util_inspect.py b/tests/test_util_inspect.py index 6a3e25b6e..0c7c7667d 100644 --- a/tests/test_util_inspect.py +++ b/tests/test_util_inspect.py @@ -346,6 +346,24 @@ def test_dictionary_sorting(): assert description == "{'a': 1, 'b': 4, 'c': 3, 'd': 2}" +def test_set_sorting(): + set_ = set("gfedcba") + description = inspect.object_description(set_) + if PY3: + assert description == "{'a', 'b', 'c', 'd', 'e', 'f', 'g'}" + else: + assert description == "set(['a', 'b', 'c', 'd', 'e', 'f', 'g'])" + + +def test_set_sorting_fallback(): + set_ = set((None, 1)) + description = inspect.object_description(set_) + if PY3: + assert description in ("{1, None}", "{None, 1}") + else: + assert description in ("set([1, None])", "set([None, 1])") + + def test_dict_customtype(): class CustomType(object): def __init__(self, value): diff --git a/utils/release-checklist b/utils/release-checklist index cd3044a51..84cbb3829 100644 --- a/utils/release-checklist +++ b/utils/release-checklist @@ -13,7 +13,7 @@ for stable releases * ``make clean`` * ``python setup.py release bdist_wheel sdist`` * ``twine upload dist/<Sphinx-X.Y.Z files> --sign --identity [your GPG key]`` -* open https://pypi.python.org/pypi/Sphinx and check there are no obvious errors +* open https://pypi.org/project/Sphinx/ and check there are no obvious errors * ``git tag vX.Y.Z`` * ``python utils/bump_version.py --in-develop X.Y.Zb0`` (ex. 1.5.3b0) * Check diff by ``git diff`` @@ -40,7 +40,7 @@ for first beta releases * ``make clean`` * ``python setup.py release bdist_wheel sdist`` * ``twine upload dist/<Sphinx-X.Y.Z files> --sign --identity [your GPG key]`` -* open https://pypi.python.org/pypi/Sphinx and check there are no obvious errors +* open https://pypi.org/project/Sphinx/ and check there are no obvious errors * ``git tag vX.Y.0b1`` * ``python utils/bump_version.py --in-develop X.Y.0b2`` (ex. 1.6.0b2) * Check diff by ``git diff`` @@ -53,6 +53,7 @@ for first beta releases * Check diff by ``git diff`` * ``git commit -am 'Bump version'`` * ``git push origin master`` +* open https://github.com/sphinx-doc/sphinx/settings/branches and make ``X.Y`` branch protected * Update `sphinx-doc-translations <https://github.com/sphinx-doc/sphinx-doc-translations>`_ * Add new version/milestone to tracker categories * Write announcement and send to sphinx-dev, sphinx-users and python-announce @@ -69,7 +70,7 @@ for other beta releases * ``make clean`` * ``python setup.py release bdist_wheel sdist`` * ``twine upload dist/<Sphinx-X.Y.Z files> --sign --identity [your GPG key]`` -* open https://pypi.python.org/pypi/Sphinx and check there are no obvious errors +* open https://pypi.org/project/Sphinx/ and check there are no obvious errors * ``git tag vX.Y.0bN`` * ``python utils/bump_version.py --in-develop X.Y.0bM`` (ex. 1.6.0b3) * Check diff by `git diff`` @@ -98,7 +99,7 @@ for major releases * ``make clean`` * ``python setup.py release bdist_wheel sdist`` * ``twine upload dist/<Sphinx-X.Y.Z files> --sign --identity [your GPG key]`` -* open https://pypi.python.org/pypi/Sphinx and check there are no obvious errors +* open https://pypi.org/project/Sphinx/ and check there are no obvious errors * ``git tag vX.Y.0`` * ``python utils/bump_version.py --in-develop X.Y.1b0`` (ex. 1.6.1b0) * Check diff by ``git diff`` @@ -107,6 +108,7 @@ for major releases * ``git checkout master`` * ``git merge X.Y`` * ``git push origin master`` +* open https://github.com/sphinx-doc/sphinx/settings/branches and make ``A.B`` branch *not* protected * ``git checkout A.B`` (checkout old stable) * Run ``git tag A.B`` to paste a tag instead branch * Run ``git push origin :A.B --tags`` to remove old stable branch |
