diff options
189 files changed, 3955 insertions, 4428 deletions
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 4fbf6ff2e..a863f2888 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,5 +1,13 @@ Subject: <what happen when you do on which document project> +<!-- + Important: This is a list of issues for Sphinx, not a forum. + If you'd like to post a question, please move to sphinx-users group. + https://groups.google.com/forum/#!forum/sphinx-users + + Thanks, +--> + ### Problem - <Detail of problem> @@ -49,6 +49,7 @@ Other contributors, listed alphabetically, are: * Will Maier -- directory HTML builder * Jacob Mason -- websupport library (GSOC project) * Glenn Matthews -- python domain signature improvements +* Kurt McKee -- documentation updates * Roland Meister -- epub builder * Ezio Melotti -- collapsible sidebar JavaScript * Bruce Mitchener -- Minor epub improvement @@ -1,3 +1,66 @@ +Release 1.8.0 (in development) +============================== + +Dependencies +------------ + +Incompatible changes +-------------------- + +* #4460: extensions which stores any data to environment should return the + version of its env data structure as metadata. In detail, please see + :ref:`ext-metadata`. +* Sphinx expects source parser modules to have supported file formats as + ``Parser.supported`` attribute +* The default value of :confval:`epub_author` and :confval:`epub_publisher` are + changed from ``'unknown'`` to the value of :confval:`author`. This is same as + a ``conf.py`` file sphinx-build generates. + +Deprecated +---------- + +* :confval:`source_parsers` is deprecated + instead. +* ``Application.import_object()`` is deprecated +* Drop function based directive support. For now, Sphinx only supports class + based directives. +* ``Sphinx.add_source_parser()`` has changed; the *suffix* argument has + been deprecated +* ``sphinx.util.docutils.directive_helper()`` is deprecated +* ``sphinx.cmdline`` is deprecated +* All ``env.update()``, ``env._read_serial()`` and ``env._read_parallel()`` are + deprecated +* ``sphinx.locale.l_()`` is deprecated + +For more details, see `deprecation APIs list +<http://www.sphinx-doc.org/en/master/extdev/index.html#deprecated-apis>`_ + +Features added +-------------- + +* Add :event:`config-inited` event +* Add ``sphinx.config.Any`` to represent the config value accepts any type of + value +* :confval:`source_suffix` allows a mapping fileext to file types +* Add :confval:`author` as a configuration value +* #2852: imgconverter: Support to convert GIF to PNG +* ``sphinx-build`` command supports i18n console output +* Add ``app.add_message_catalog()`` and ``sphinx.locale.get_translations()`` to + support translation for 3rd party extensions + +Bugs fixed +---------- + +* i18n: message catalogs were reset on each initialization + +Testing +-------- + +Features removed +---------------- + +* ``sphinx.ext.pngmath`` extension + Release 1.7.2 (in development) ============================== @@ -181,6 +244,8 @@ Features added * #4271: sphinx-build supports an option called ``-j auto`` to adjust numbers of processes automatically. +* Napoleon: added option to specify custom section tags. + Features removed ---------------- @@ -366,6 +366,7 @@ Books produced using Sphinx https://www.packtpub.com/application-development/expert-python-programming * "Expert Python Programming" (Japanese translation): https://www.amazon.co.jp/dp/4048686291/ +* "The Hitchhiker's Guide to Python": http://docs.python-guide.org/en/latest/ * "LassoGuide": http://www.lassosoft.com/Lasso-Documentation * "Learning Sphinx" (in Japanese): https://www.oreilly.co.jp/books/9784873116488/ @@ -104,36 +104,6 @@ smartypants.py license:: such damage. ---------------------------------------------------------------------- -The ElementTree package, included in this distribution in -test/etree13, is available under the following license: - ----------------------------------------------------------------------- -The ElementTree toolkit is - -Copyright (c) 1999-2007 by Fredrik Lundh - -By obtaining, using, and/or copying this software and/or its -associated documentation, you agree that you have read, understood, -and will comply with the following terms and conditions: - -Permission to use, copy, modify, and distribute this software and its -associated documentation for any purpose and without fee is hereby -granted, provided that the above copyright notice appears in all -copies, and that both that copyright notice and this permission notice -appear in supporting documentation, and that the name of Secret Labs -AB or the author not be used in advertising or publicity pertaining to -distribution of the software without specific, written prior -permission. - -SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO -THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- ABILITY -AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE -FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES -WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN -ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT -OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ----------------------------------------------------------------------- - The included JQuery JavaScript library is available under the MIT license: @@ -3,14 +3,6 @@ PYTHON ?= python .PHONY: all all: clean-pyc clean-backupfiles style-check type-check test -.PHONY: style-check -style-check: - @flake8 - -.PHONY: type-check -type-check: - mypy sphinx/ - .PHONY: clean clean: clean-pyc clean-pycache clean-patchfiles clean-backupfiles clean-generated clean-testfiles clean-buildfiles clean-mypyfiles @@ -59,22 +51,22 @@ clean-buildfiles: clean-mypyfiles: rm -rf .mypy_cache/ +.PHONY: style-check +style-check: + @flake8 + +.PHONY: type-check +type-check: + mypy sphinx/ + .PHONY: pylint pylint: @pylint --rcfile utils/pylintrc sphinx -.PHONY: reindent -reindent: - @echo "This target no longer does anything and will be removed imminently" - .PHONY: test test: @$(PYTHON) -m pytest -v $(TEST) -.PHONY: test-async -test-async: - @echo "This target no longer does anything and will be removed imminently" - .PHONY: covertest covertest: @$(PYTHON) -m pytest -v --cov=sphinx --junitxml=.junit.xml $(TEST) @@ -86,6 +78,6 @@ build: .PHONY: docs docs: ifndef target - $(info You need to give a provide a target variable, e.g. `make docs target=html`.) + $(info You need to give a provide a target variable, e.g. `make docs target=html`.) endif - $(MAKE) -C doc $(target) + $(MAKE) -C doc $(target) diff --git a/README.rst b/README.rst index 82f61710b..0d29c3984 100644 --- a/README.rst +++ b/README.rst @@ -81,6 +81,15 @@ Documentation is available from `sphinx-doc.org`__. __ http://www.sphinx-doc.org/ +Get in touch +============ + +- Report bugs, suggest features or view the source code `on GitHub`_. +- For less well defined questions or ideas, use the `mailing list`_. + +.. _on GitHub: https://github.com/sphinx-doc/sphinx +.. _mailing list: https://groups.google.com/forum/#!forum/sphinx-users + Testing ======= diff --git a/doc/more.png b/doc/_static/more.png Binary files differindex 97553a8b7..97553a8b7 100644 --- a/doc/more.png +++ b/doc/_static/more.png diff --git a/doc/_templates/index.html b/doc/_templates/index.html index 5a8a2f025..032636d94 100644 --- a/doc/_templates/index.html +++ b/doc/_templates/index.html @@ -49,7 +49,7 @@ <table class="contentstable"> <tr> <td> - <p class="biglink"><a class="biglink" href="{{ pathto("tutorial") }}">{%trans%}First steps with Sphinx{%endtrans%}</a><br/> + <p class="biglink"><a class="biglink" href="{{ pathto("usage/quickstart") }}">{%trans%}First steps with Sphinx{%endtrans%}</a><br/> <span class="linkdescr">{%trans%}overview of basic tasks{%endtrans%}</span></p> </td><td> {%- if hasdoc('search') %}<p class="biglink"><a class="biglink" href="{{ pathto("search") }}">{%trans%}Search page{%endtrans%}</a><br/> @@ -75,8 +75,8 @@ <p>{%trans%} You can also download PDF/EPUB versions of the Sphinx documentation: a <a href="https://media.readthedocs.org/pdf/sphinx/stable/sphinx.pdf">PDF version</a> generated from - the LaTeX Sphinx produces, and - a <a href="https://media.readthedocs.org/epub/sphinx/stable/sphinx.epub">EPUB version</a>. + the LaTeX Sphinx producer, and + an <a href="https://media.readthedocs.org/epub/sphinx/stable/sphinx.epub">EPUB version</a>. {%endtrans%} </p> @@ -98,7 +98,7 @@ <p>{%trans%}There is a <a href="http://docs.sphinx-users.jp/">Japanese translation</a> of this documentation, thanks to the Japanese Sphinx user group.{%endtrans%}</p> <p>{%trans%}A Japanese book about Sphinx has been published by O'Reilly: - <a href="http://www.oreilly.co.jp/books/9784873116488/">Sphinxをはじめよう / + <a href="https://www.oreilly.co.jp/books/9784873116488/">Sphinxをはじめよう / Learning Sphinx</a>.{%endtrans%}</p> <!-- <p><img src="{{ pathto("_static/bookcover.png", 1) }}"/></p> --> diff --git a/doc/_templates/indexsidebar.html b/doc/_templates/indexsidebar.html index b07ef2033..668070579 100644 --- a/doc/_templates/indexsidebar.html +++ b/doc/_templates/indexsidebar.html @@ -1,4 +1,4 @@ -<p class="logo">A <a href="http://pocoo.org/"> +<p class="logo">A <a href="https://www.pocoo.org/"> <img src="{{ pathto("_static/pocoo.png", 1) }}" alt="Pocoo" /></a> {%trans%}project{%endtrans%}</p> @@ -19,9 +19,9 @@ Index</a>, or install it with:{%endtrans%}</p> <h3>{%trans%}Questions? Suggestions?{%endtrans%}</h3> -<p>{%trans%}Join the <a href="http://groups.google.com/group/sphinx-users">sphinx-users</a> mailing list on Google Groups:{%endtrans%}</p> +<p>{%trans%}Join the <a href="https://groups.google.com/group/sphinx-users">sphinx-users</a> mailing list on Google Groups:{%endtrans%}</p> <div class="subscribeformwrapper"> -<form action="http://groups.google.com/group/sphinx-users/boxsubscribe" +<form action="https://groups.google.com/group/sphinx-users/boxsubscribe" class="subscribeform"> <input type="text" name="email" value="your@email" onfocus="$(this).val('');" /> diff --git a/doc/builders.rst b/doc/builders.rst index ff56881af..ffaa1e14f 100644 --- a/doc/builders.rst +++ b/doc/builders.rst @@ -207,16 +207,10 @@ The builder's "name" must be given to the **-b** command-line option of .. autoattribute:: supported_image_types Note that a direct PDF builder is being provided by `rinohtype`_. The builder's -name is ``rinoh``. Refer to the `rinohtype manual`_ for details. There is also -PDF builder using ReportLab in `rst2pdf`_ version 0.12 or greater. However, -rst2pdf is no longer being actively maintained and suffers from some problems -when used with recent Sphinx versions. See the `rst2pdf manual`_ for usage -instructions. +name is ``rinoh``. Refer to the `rinohtype manual`_ for details. .. _rinohtype: https://github.com/brechtm/rinohtype .. _rinohtype manual: http://www.mos6581.org/rinohtype/quickstart.html#sphinx-builder -.. _rst2pdf: https://github.com/rst2pdf/rst2pdf -.. _rst2pdf manual: http://ralsina.me/static/manual.pdf .. module:: sphinx.builders.text .. class:: TextBuilder diff --git a/doc/config.rst b/doc/config.rst index 830fb69a0..7bcae99f5 100644 --- a/doc/config.rst +++ b/doc/config.rst @@ -90,12 +90,32 @@ General configuration .. confval:: source_suffix - The file name extension, or list of extensions, of source files. Only files - with this suffix will be read as sources. Default is ``'.rst'``. + The file extensions of source files. Sphinx considers the files with this + suffix as sources. This value can be a dictionary mapping file extensions + to file types. For example:: + + source_suffix = { + '.rst': 'restructuredtext', + '.txt': 'restructuredtext', + '.md': 'markdown', + } + + By default, Sphinx only supports ``'restrcturedtext'`` file type. You can + add a new file type using source parser extensions. Please read a document + of the extension to know what file type the extension supports. + + This also allows a list of file extensions. In that case, Sphinx conciders + that all they are ``'restructuredtext'``. Default is + ``{'.rst': 'restructuredtext'}``. + + .. note:: file extensions have to start with dot (like ``.rst``). .. versionchanged:: 1.3 Can now be a list of extensions. + .. versionchanged:: 1.8 + Support file type mapping + .. confval:: source_encoding The encoding of all reST source files. The recommended encoding, and the @@ -123,6 +143,10 @@ General configuration .. versionadded:: 1.3 + .. deprecated:: 1.8 + Now Sphinx provides an API :meth:`Sphinx.add_source_parser` to register + a source parser. Please use it instead. + .. confval:: master_doc The document name of the "master" document, that is, the document that @@ -451,6 +475,10 @@ Project information The documented project's name. +.. confval:: author + + The author name(s) of the document. The default value is ``'unknown'``. + .. confval:: copyright A copyright statement in the style ``'2008, Author Name'``. @@ -1191,16 +1219,21 @@ that use Sphinx's HTMLWriter class. .. versionadded:: 1.3 -.. confval:: htmlhelp_basename - - Output file base name for HTML help builder. Default is ``'pydoc'``. - .. confval:: html_experimental_html5_writer Output is processed with HTML5 writer. This feature needs docutils 0.13 or newer. Default is ``False``. .. versionadded:: 1.6 +.. _htmlhelp-options: + +Options for HTML help output +----------------------------- + +.. confval:: htmlhelp_basename + + Output file base name for HTML help builder. Default is ``'pydoc'``. + .. _applehelp-options: Options for Apple Help output @@ -1410,8 +1443,8 @@ the `Dublin Core metadata <http://dublincore.org/>`_. .. confval:: epub_author - The author of the document. This is put in the Dublin Core metadata. The - default value is ``'unknown'``. + The author of the document. This is put in the Dublin Core metadata. It + defaults to the :confval:`author` option. .. confval:: epub_contributor @@ -1432,8 +1465,8 @@ the `Dublin Core metadata <http://dublincore.org/>`_. .. confval:: epub_publisher The publisher of the document. This is put in the Dublin Core metadata. You - may use any sensible string, e.g. the project homepage. The default value is - ``'unknown'``. + may use any sensible string, e.g. the project homepage. The defaults to the + :confval:`author` option. .. confval:: epub_copyright diff --git a/doc/contents.rst b/doc/contents.rst index ab8d09eb0..886861fba 100644 --- a/doc/contents.rst +++ b/doc/contents.rst @@ -7,8 +7,10 @@ Sphinx documentation contents .. toctree:: :maxdepth: 2 + usage/installation + usage/quickstart + intro - tutorial man/index rest markup/index @@ -33,6 +35,7 @@ Sphinx documentation contents authors + Indices and tables ================== diff --git a/doc/develop.rst b/doc/develop.rst index 19ca81ef9..af0302367 100644 --- a/doc/develop.rst +++ b/doc/develop.rst @@ -12,7 +12,7 @@ Sphinx is a maintained by a group of volunteers. We value every contribution! * The mailing list for development is at `Google Groups <https://groups.google.com/forum/#!forum/sphinx-dev>`_. * There is also the #sphinx-doc IRC channel on `freenode - <http://freenode.net/>`_. + <https://freenode.net/>`_. For more about our development process and methods, see the :doc:`devguide`. @@ -106,37 +106,37 @@ own extensions. .. _aafigure: https://launchpad.net/aafigure .. _gnuplot: http://www.gnuplot.info/ -.. _paver: http://paver.github.io/paver/ -.. _Sword: http://www.crosswire.org/sword/ +.. _paver: https://paver.readthedocs.io/en/latest/ +.. _Sword: https://www.crosswire.org/sword/ .. _Lilypond: http://lilypond.org/ .. _sdedit: http://sdedit.sourceforge.net/ -.. _Trac: http://trac.edgewall.org -.. _TracLinks: http://trac.edgewall.org/wiki/TracLinks -.. _OmegaT: http://www.omegat.org/ +.. _Trac: https://trac.edgewall.org/ +.. _TracLinks: https://trac.edgewall.org/wiki/TracLinks +.. _OmegaT: https://omegat.org/ .. _PlantUML: http://plantuml.com/ -.. _PyEnchant: http://www.rfk.id.au/software/pyenchant/ +.. _PyEnchant: https://pythonhosted.org/pyenchant/ .. _sadisplay: https://bitbucket.org/estin/sadisplay/wiki/Home .. _blockdiag: http://blockdiag.com/en/ .. _seqdiag: http://blockdiag.com/en/ .. _actdiag: http://blockdiag.com/en/ .. _nwdiag: http://blockdiag.com/en/ -.. _Google Analytics: http://www.google.com/analytics/ +.. _Google Analytics: https://www.google.com/analytics/ .. _Google Chart: https://developers.google.com/chart/ .. _Google Maps: https://www.google.com/maps .. _Google style: https://google.github.io/styleguide/pyguide.html .. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt .. _hyphenator: https://github.com/mnater/hyphenator -.. _exceltable: http://pythonhosted.org/sphinxcontrib-exceltable/ +.. _exceltable: https://pythonhosted.org/sphinxcontrib-exceltable/ .. _YouTube: http://www.youtube.com/ -.. _ClearQuest: http://www-03.ibm.com/software/products/en/clearquest -.. _Zope interfaces: http://docs.zope.org/zope.interface/README.html -.. _slideshare: http://www.slideshare.net/ +.. _ClearQuest: https://www.ibm.com/us-en/marketplace/rational-clearquest +.. _Zope interfaces: https://zopeinterface.readthedocs.io/en/latest/README.html +.. _slideshare: https://www.slideshare.net/ .. _TikZ/PGF LaTeX package: https://sourceforge.net/projects/pgf/ -.. _MATLAB: http://www.mathworks.com/products/matlab/ +.. _MATLAB: https://www.mathworks.com/products/matlab.html .. _swf: https://bitbucket.org/klorenz/sphinxcontrib-swf .. _findanything: https://bitbucket.org/klorenz/sphinxcontrib-findanything .. _cmakedomain: https://bitbucket.org/klorenz/sphinxcontrib-cmakedomain -.. _GNU Make: http://www.gnu.org/software/make/ +.. _GNU Make: https://www.gnu.org/software/make/ .. _makedomain: https://bitbucket.org/klorenz/sphinxcontrib-makedomain .. _inlinesyntaxhighlight: https://sphinxcontrib-inlinesyntaxhighlight.readthedocs.io/ .. _CMake: https://cmake.org diff --git a/doc/ext/autodoc.rst b/doc/ext/autodoc.rst index 09098f39c..0bddba5c5 100644 --- a/doc/ext/autodoc.rst +++ b/doc/ext/autodoc.rst @@ -398,7 +398,7 @@ There are also new config values that you can set: .. confval:: autodoc_inherit_docstrings This value controls the docstrings inheritance. - If set to True the cocstring for classes or methods, if not explicitly set, + If set to True the docstring for classes or methods, if not explicitly set, is inherited form parents. The default is ``True``. diff --git a/doc/ext/example_google.py b/doc/ext/example_google.py index 5b4fa58df..4f6abacdf 100644 --- a/doc/ext/example_google.py +++ b/doc/ext/example_google.py @@ -29,7 +29,7 @@ Todo: * You have to also use ``sphinx.ext.todo`` extension .. _Google Python Style Guide: - http://google.github.io/styleguide/pyguide.html + https://google.github.io/styleguide/pyguide.html """ diff --git a/doc/ext/graphviz.rst b/doc/ext/graphviz.rst index ef0483da7..8b9fff90b 100644 --- a/doc/ext/graphviz.rst +++ b/doc/ext/graphviz.rst @@ -8,7 +8,7 @@ .. versionadded:: 0.6 -This extension allows you to embed `Graphviz <http://graphviz.org/>`_ graphs in +This extension allows you to embed `Graphviz <https://graphviz.org/>`_ graphs in your documents. It adds these directives: diff --git a/doc/ext/intersphinx.rst b/doc/ext/intersphinx.rst index fb0114fc4..b8ae104b3 100644 --- a/doc/ext/intersphinx.rst +++ b/doc/ext/intersphinx.rst @@ -38,7 +38,10 @@ Behind the scenes, this works as follows: specified individually, e.g. if the docs should be buildable without Internet access. -To use intersphinx linking, add ``'sphinx.ext.intersphinx'`` to your +Configuring Intersphinx +----------------------- + +To use Intersphinx linking, add ``'sphinx.ext.intersphinx'`` to your :confval:`extensions` config value, and use these new config values to activate linking: @@ -84,7 +87,7 @@ linking: To add links to modules and objects in the Python standard library documentation, use:: - intersphinx_mapping = {'python': ('https://docs.python.org/3.4', None)} + intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} This will download the corresponding :file:`objects.inv` file from the Internet and generate links to the pages under the given URI. The downloaded @@ -94,12 +97,12 @@ linking: A second example, showing the meaning of a non-``None`` value of the second tuple item:: - intersphinx_mapping = {'python': ('https://docs.python.org/3.4', + intersphinx_mapping = {'python': ('https://docs.python.org/3', 'python-inv.txt')} This will read the inventory from :file:`python-inv.txt` in the source directory, but still generate links to the pages under - ``https://docs.python.org/3.4``. It is up to you to update the inventory file + ``https://docs.python.org/3``. It is up to you to update the inventory file as new objects are added to the Python documentation. **Multiple target for the inventory** @@ -113,7 +116,7 @@ linking: this to specify mirror sites for server downtime of the primary inventory:: - intersphinx_mapping = {'python': ('https://docs.python.org/3.4', + intersphinx_mapping = {'python': ('https://docs.python.org/3', (None, 'python-inv.txt'))} .. confval:: intersphinx_cache_limit @@ -132,3 +135,13 @@ linking: timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds. + +Showing all links of an Intersphinx mapping file +------------------------------------------------ + +To show all Intersphinx links and their targets of an Intersphinx mapping file, run +``python -msphinx.ext.intersphinx url-or-path``. This is helpful when searching for the root cause of a broken +Intersphinx link in a documentation project. The following example prints the Intersphinx mapping of the Python 3 +documentation:: + + $ python -msphinx.ext.intersphinx https://docs.python.org/3/objects.inv diff --git a/doc/ext/linkcode.rst b/doc/ext/linkcode.rst index 05d2cc6db..f00345fca 100644 --- a/doc/ext/linkcode.rst +++ b/doc/ext/linkcode.rst @@ -44,4 +44,4 @@ function that returns an URL based on the object. if not info['module']: return None filename = info['module'].replace('.', '/') - return "http://somesite/sourcerepo/%s.py" % filename + return "https://somesite/sourcerepo/%s.py" % filename diff --git a/doc/ext/math.rst b/doc/ext/math.rst index 4097bb29e..3e4f94c6c 100644 --- a/doc/ext/math.rst +++ b/doc/ext/math.rst @@ -248,7 +248,7 @@ Sphinx. __ https://cdjns.com - __ http://docs.mathjax.org/en/latest/start.html + __ https://docs.mathjax.org/en/latest/start.html The path can be absolute or relative; if it is relative, it is relative to the ``_static`` directory of the built docs. @@ -258,7 +258,7 @@ Sphinx. documentation set on one server, it is advisable to install MathJax in a shared location. - You can also give a full ``http://`` URL different from the CDN URL. + You can also give a full ``https://`` URL different from the CDN URL. :mod:`sphinx.ext.jsmath` -- Render math via JavaScript @@ -284,9 +284,9 @@ package jsMath_. It provides this config value: a shared location. -.. _dvipng: http://savannah.nongnu.org/projects/dvipng/ +.. _dvipng: https://savannah.nongnu.org/projects/dvipng/ .. _dvisvgm: http://dvisvgm.bplaced.net/ .. _MathJax: https://www.mathjax.org/ .. _jsMath: http://www.math.union.edu/~dpvc/jsmath/ -.. _preview-latex package: http://www.gnu.org/software/auctex/preview-latex.html -.. _AmSMath LaTeX package: http://www.ams.org/publications/authors/tex/amslatex +.. _preview-latex package: https://www.gnu.org/software/auctex/preview-latex.html +.. _AmSMath LaTeX package: https://www.ams.org/publications/authors/tex/amslatex diff --git a/doc/ext/napoleon.rst b/doc/ext/napoleon.rst index f7e9081f7..6c5f0d61a 100644 --- a/doc/ext/napoleon.rst +++ b/doc/ext/napoleon.rst @@ -52,9 +52,9 @@ source code files. .. _ReStructuredText: http://docutils.sourceforge.net/rst.html .. _docstrings: https://www.python.org/dev/peps/pep-0287/ .. _Google Python Style Guide: - http://google.github.io/styleguide/pyguide.html + https://google.github.io/styleguide/pyguide.html .. _Google: - http://google.github.io/styleguide/pyguide.html#Comments + https://google.github.io/styleguide/pyguide.html#Comments .. _NumPy: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt .. _Khan Academy: @@ -63,8 +63,8 @@ source code files. Getting Started --------------- -1. After :doc:`setting up Sphinx <../tutorial>` to build your docs, enable - napoleon in the Sphinx `conf.py` file:: +1. After :doc:`setting up Sphinx </usage/quickstart>` to build your docs, + enable napoleon in the Sphinx `conf.py` file:: # conf.py @@ -267,7 +267,7 @@ sure that "sphinx.ext.napoleon" is enabled in `conf.py`:: napoleon_use_rtype = True .. _Google style: - http://google-styleguide.googlecode.com/svn/trunk/pyguide.html + https://google.github.io/styleguide/pyguide.html .. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt diff --git a/doc/ext/thirdparty.rst b/doc/ext/thirdparty.rst index 40c24246a..20dda9ff1 100644 --- a/doc/ext/thirdparty.rst +++ b/doc/ext/thirdparty.rst @@ -6,8 +6,9 @@ repository. It is open for anyone who wants to maintain an extension publicly; just send a short message asking for write permissions. There are also several extensions hosted elsewhere. The `Sphinx extension -survey <https://sphinxext-survey.readthedocs.io/>`__ contains a -comprehensive list. +survey <https://sphinxext-survey.readthedocs.io/>`__ and `awesome-sphinxdoc +<https://github.com/yoloseem/awesome-sphinxdoc>`__ contains a comprehensive +list. If you write an extension that you think others will find useful or you think should be included as a part of Sphinx, please write to the project mailing diff --git a/doc/extdev/appapi.rst b/doc/extdev/appapi.rst index ce856d031..584cf375b 100644 --- a/doc/extdev/appapi.rst +++ b/doc/extdev/appapi.rst @@ -7,9 +7,9 @@ Application API :synopsis: Application class and extensibility interface. -Each Sphinx extension is a Python module with at least a :func:`setup` function. -This function is called at initialization time with one argument, the -application object representing the Sphinx process. +Each Sphinx extension is a Python module with at least a :func:`setup` +function. This function is called at initialization time with one argument, +the application object representing the Sphinx process. .. class:: Sphinx @@ -23,387 +23,79 @@ These methods are usually called in an extension's ``setup()`` function. Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext` package. -.. method:: Sphinx.setup_extension(name) - - Load the extension given by the module *name*. Use this if your extension - needs the features provided by another extension. - -.. method:: Sphinx.add_builder(builder) - - Register a new builder. *builder* must be a class that inherits from - :class:`~sphinx.builders.Builder`. - -.. method:: Sphinx.add_config_value(name, default, rebuild) - - Register a configuration value. This is necessary for Sphinx to recognize - new values and set default values accordingly. The *name* should be prefixed - with the extension name, to avoid clashes. The *default* value can be any - Python object. The string value *rebuild* must be one of those values: - - * ``'env'`` if a change in the setting only takes effect when a document is - parsed -- this means that the whole environment must be rebuilt. - * ``'html'`` if a change in the setting needs a full rebuild of HTML - documents. - * ``''`` if a change in the setting will not need any special rebuild. - - .. versionchanged:: 0.4 - If the *default* value is a callable, it will be called with the config - object as its argument in order to get the default value. This can be - used to implement config values whose default depends on other values. - - .. versionchanged:: 0.6 - Changed *rebuild* from a simple boolean (equivalent to ``''`` or - ``'env'``) to a string. However, booleans are still accepted and - converted internally. - -.. method:: Sphinx.add_domain(domain) - - Make the given *domain* (which must be a class; more precisely, a subclass of - :class:`~sphinx.domains.Domain`) known to Sphinx. - - .. versionadded:: 1.0 - -.. method:: Sphinx.override_domain(domain) - - Make the given *domain* class known to Sphinx, assuming that there is already - a domain with its ``.name``. The new domain must be a subclass of the - existing one. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_index_to_domain(domain, index) - - Add a custom *index* class to the domain named *domain*. *index* must be a - subclass of :class:`~sphinx.domains.Index`. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_event(name) - - Register an event called *name*. This is needed to be able to emit it. - -.. method:: Sphinx.set_translator(name, translator_class) - - Register or override a Docutils translator class. This is used to register - a custom output translator or to replace a builtin translator. - This allows extensions to use custom translator and define custom - nodes for the translator (see :meth:`add_node`). - - .. versionadded:: 1.3 - -.. method:: Sphinx.add_node(node, **kwds) - - Register a Docutils node class. This is necessary for Docutils internals. - It may also be used in the future to validate nodes in the parsed documents. - - Node visitor functions for the Sphinx HTML, LaTeX, text and manpage writers - can be given as keyword arguments: the keyword should be one or more of - ``'html'``, ``'latex'``, ``'text'``, ``'man'``, ``'texinfo'`` or any other - supported translators, the value a 2-tuple of ``(visit, depart)`` methods. - ``depart`` can be ``None`` if the ``visit`` function raises - :exc:`docutils.nodes.SkipNode`. Example: - - .. code-block:: python - - class math(docutils.nodes.Element): pass - - def visit_math_html(self, node): - self.body.append(self.starttag(node, 'math')) - def depart_math_html(self, node): - self.body.append('</math>') - - app.add_node(math, html=(visit_math_html, depart_math_html)) - - Obviously, translators for which you don't specify visitor methods will choke - on the node when encountered in a document to translate. - - .. versionchanged:: 0.5 - Added the support for keyword arguments giving visit functions. - -.. method:: Sphinx.add_enumerable_node(node, figtype, title_getter=None, **kwds) - - Register a Docutils node class as a numfig target. Sphinx numbers the node - automatically. And then the users can refer it using :rst:role:`numref`. - - *figtype* is a type of enumerable nodes. Each figtypes have individual - numbering sequences. As a system figtypes, ``figure``, ``table`` and - ``code-block`` are defined. It is able to add custom nodes to these - default figtypes. It is also able to define new custom figtype if new - figtype is given. - - *title_getter* is a getter function to obtain the title of node. It takes - an instance of the enumerable node, and it must return its title as string. - The title is used to the default title of references for :rst:role:`ref`. - By default, Sphinx searches ``docutils.nodes.caption`` or - ``docutils.nodes.title`` from the node as a title. - - Other keyword arguments are used for node visitor functions. See the - :meth:`Sphinx.add_node` for details. - - .. versionadded:: 1.4 - -.. method:: Sphinx.add_directive(name, func, content, arguments, **options) - Sphinx.add_directive(name, directiveclass) - - Register a Docutils directive. *name* must be the prospective directive - name. There are two possible ways to write a directive: - - * In the docutils 0.4 style, *obj* is the directive function. *content*, - *arguments* and *options* are set as attributes on the function and - determine whether the directive has content, arguments and options, - respectively. **This style is deprecated.** - - * In the docutils 0.5 style, *directiveclass* is the directive class. It - must already have attributes named *has_content*, *required_arguments*, - *optional_arguments*, *final_argument_whitespace* and *option_spec* that - correspond to the options for the function way. See `the Docutils docs - <http://docutils.sourceforge.net/docs/howto/rst-directives.html>`_ for - details. - - The directive class must inherit from the class - ``docutils.parsers.rst.Directive``. - - For example, the (already existing) :rst:dir:`literalinclude` directive would - be added like this: - - .. code-block:: python - - from docutils.parsers.rst import directives - add_directive('literalinclude', literalinclude_directive, - content = 0, arguments = (1, 0, 0), - linenos = directives.flag, - language = directives.unchanged, - encoding = directives.encoding) - - .. versionchanged:: 0.6 - Docutils 0.5-style directive classes are now supported. - -.. method:: Sphinx.add_directive_to_domain(domain, name, func, content, arguments, **options) - Sphinx.add_directive_to_domain(domain, name, directiveclass) - - Like :meth:`add_directive`, but the directive is added to the domain named - *domain*. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_role(name, role) - - Register a Docutils role. *name* must be the role name that occurs in the - source, *role* the role function (see the `Docutils documentation - <http://docutils.sourceforge.net/docs/howto/rst-roles.html>`_ on details). - -.. method:: Sphinx.add_role_to_domain(domain, name, role) - - Like :meth:`add_role`, but the role is added to the domain named *domain*. - - .. versionadded:: 1.0 - -.. method:: Sphinx.add_generic_role(name, nodeclass) - - Register a Docutils role that does nothing but wrap its contents in the - node given by *nodeclass*. - - .. versionadded:: 0.6 - -.. method:: Sphinx.add_object_type(directivename, rolename, indextemplate='', parse_node=None, \ - ref_nodeclass=None, objname='', doc_field_types=[]) - - This method is a very convenient way to add a new :term:`object` type that - can be cross-referenced. It will do this: - - * Create a new directive (called *directivename*) for documenting an object. - It will automatically add index entries if *indextemplate* is nonempty; if - given, it must contain exactly one instance of ``%s``. See the example - below for how the template will be interpreted. - * Create a new role (called *rolename*) to cross-reference to these - object descriptions. - * If you provide *parse_node*, it must be a function that takes a string and - a docutils node, and it must populate the node with children parsed from - the string. It must then return the name of the item to be used in - cross-referencing and index entries. See the :file:`conf.py` file in the - source for this documentation for an example. - * The *objname* (if not given, will default to *directivename*) names the - type of object. It is used when listing objects, e.g. in search results. - - For example, if you have this call in a custom Sphinx extension:: - - app.add_object_type('directive', 'dir', 'pair: %s; directive') - - you can use this markup in your documents:: - - .. rst:directive:: function - - Document a function. - - <...> - - See also the :rst:dir:`function` directive. - - For the directive, an index entry will be generated as if you had prepended :: - - .. index:: pair: function; directive - - The reference node will be of class ``literal`` (so it will be rendered in a - proportional font, as appropriate for code) unless you give the - *ref_nodeclass* argument, which must be a docutils node class. Most useful - are ``docutils.nodes.emphasis`` or ``docutils.nodes.strong`` -- you can also - use ``docutils.nodes.generated`` if you want no further text decoration. If - the text should be treated as literal (e.g. no smart quote replacement), but - not have typewriter styling, use ``sphinx.addnodes.literal_emphasis`` or - ``sphinx.addnodes.literal_strong``. - - For the role content, you have the same syntactical possibilities as for - standard Sphinx roles (see :ref:`xref-syntax`). - - This method is also available under the deprecated alias - ``add_description_unit``. - -.. method:: Sphinx.add_crossref_type(directivename, rolename, indextemplate='', ref_nodeclass=None, objname='') - - This method is very similar to :meth:`add_object_type` except that the - directive it generates must be empty, and will produce no output. - - That means that you can add semantic targets to your sources, and refer to - them using custom roles instead of generic ones (like :rst:role:`ref`). - Example call:: - - app.add_crossref_type('topic', 'topic', 'single: %s', docutils.nodes.emphasis) - - Example usage:: - - .. topic:: application API - - The application API - ------------------- - - <...> - - See also :topic:`this section <application API>`. - - (Of course, the element following the ``topic`` directive needn't be a - section.) - -.. method:: Sphinx.add_transform(transform) - - Add the standard docutils :class:`Transform` subclass *transform* to the list - of transforms that are applied after Sphinx parses a reST document. - -.. method:: Sphinx.add_post_transform(transform) - - Add the standard docutils :class:`Transform` subclass *transform* to the list - of transforms that are applied before Sphinx writes a document. - -.. method:: Sphinx.add_javascript(filename) - - Add *filename* to the list of JavaScript files that the default HTML template - will include. The filename must be relative to the HTML static path, see - :confval:`the docs for the config value <html_static_path>`. A full URI with - scheme, like ``http://example.org/foo.js``, is also supported. - - .. versionadded:: 0.5 - -.. method:: Sphinx.add_stylesheet(filename, alternate=None, title=None) - - Add *filename* to the list of CSS files that the default HTML template will - include. Like for :meth:`add_javascript`, the filename must be relative to - the HTML static path, or a full URI with scheme. - - .. versionadded:: 1.0 - - .. versionchanged:: 1.6 - Optional ``alternate`` and/or ``title`` attributes can be supplied with - the *alternate* (of boolean type) and *title* (a string) arguments. The - default is no title and *alternate* = ``False`` (see `this explanation - <https://developer.mozilla.org/en-US/docs/Web/CSS/Alternative_style_sheets>`_). - -.. method:: Sphinx.add_latex_package(packagename, options=None) +.. currentmodule:: sphinx.application - Add *packagename* to the list of packages that LaTeX source code will include. - If you provide *options*, it will be taken to `\usepackage` declaration. +.. automethod:: Sphinx.setup_extension(name) - .. code-block:: python +.. automethod:: Sphinx.require_sphinx(version) - app.add_latex_package('mypackage') # => \usepackage{mypackage} - app.add_latex_package('mypackage', 'foo,bar') # => \usepackage[foo,bar]{mypackage} +.. automethod:: Sphinx.connect(event, callback) - .. versionadded:: 1.3 +.. automethod:: Sphinx.disconnect(listener_id) -.. method:: Sphinx.add_lexer(alias, lexer) +.. automethod:: Sphinx.add_builder(builder) - Use *lexer*, which must be an instance of a Pygments lexer class, to - highlight code blocks with the given language *alias*. +.. automethod:: Sphinx.add_config_value(name, default, rebuild) - .. versionadded:: 0.6 +.. automethod:: Sphinx.add_event(name) -.. method:: Sphinx.add_autodocumenter(cls) +.. automethod:: Sphinx.set_translator(name, translator_class) - Add *cls* as a new documenter class for the :mod:`sphinx.ext.autodoc` - extension. It must be a subclass of :class:`sphinx.ext.autodoc.Documenter`. - This allows to auto-document new types of objects. See the source of the - autodoc module for examples on how to subclass :class:`Documenter`. +.. automethod:: Sphinx.add_node(node, \*\*kwds) - .. XXX add real docs for Documenter and subclassing +.. automethod:: Sphinx.add_enumerable_node(node, figtype, title_getter=None, \*\*kwds) - .. versionadded:: 0.6 +.. method:: Sphinx.add_directive(name, func, content, arguments, \*\*options) +.. automethod:: Sphinx.add_directive(name, directiveclass) -.. method:: Sphinx.add_autodoc_attrgetter(type, getter) +.. automethod:: Sphinx.add_role(name, role) - Add *getter*, which must be a function with an interface compatible to the - :func:`getattr` builtin, as the autodoc attribute getter for objects that are - instances of *type*. All cases where autodoc needs to get an attribute of a - type are then handled by this function instead of :func:`getattr`. +.. automethod:: Sphinx.add_generic_role(name, nodeclass) - .. versionadded:: 0.6 +.. automethod:: Sphinx.add_domain(domain) -.. method:: Sphinx.add_search_language(cls) +.. automethod:: Sphinx.override_domain(domain) - Add *cls*, which must be a subclass of :class:`sphinx.search.SearchLanguage`, - as a support language for building the HTML full-text search index. The - class must have a *lang* attribute that indicates the language it should be - used for. See :confval:`html_search_language`. +.. method:: Sphinx.add_directive_to_domain(domain, name, func, content, arguments, \*\*options) +.. automethod:: Sphinx.add_directive_to_domain(domain, name, directiveclass) - .. versionadded:: 1.1 +.. automethod:: Sphinx.add_role_to_domain(domain, name, role) -.. method:: Sphinx.add_source_parser(suffix, parser) +.. automethod:: Sphinx.add_index_to_domain(domain, index) - Register a parser class for specified *suffix*. +.. automethod:: Sphinx.add_object_type(directivename, rolename, indextemplate='', parse_node=None, ref_nodeclass=None, objname='', doc_field_types=[]) - .. versionadded:: 1.4 +.. automethod:: Sphinx.add_crossref_type(directivename, rolename, indextemplate='', ref_nodeclass=None, objname='') -.. method:: Sphinx.add_html_theme(name, theme_path) +.. automethod:: Sphinx.add_transform(transform) - Register a HTML Theme. The *name* is a name of theme, and *path* is a - full path to the theme (refs: :ref:`distribute-your-theme`). +.. automethod:: Sphinx.add_post_transform(transform) - .. versionadded:: 1.6 +.. automethod:: Sphinx.add_javascript(filename) -.. method:: Sphinx.add_env_collector(collector) +.. automethod:: Sphinx.add_stylesheet(filename, alternate=None, title=None) - Register an environment collector class (refs: :ref:`collector-api`) +.. automethod:: Sphinx.add_latex_package(packagename, options=None) - .. versionadded:: 1.6 +.. automethod:: Sphinx.add_lexer(alias, lexer) -.. method:: Sphinx.require_sphinx(version) +.. automethod:: Sphinx.add_autodocumenter(cls) - Compare *version* (which must be a ``major.minor`` version string, - e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build - when it is too old. +.. automethod:: Sphinx.add_autodoc_attrgetter(type, getter) - .. versionadded:: 1.0 +.. automethod:: Sphinx.add_search_language(cls) -.. method:: Sphinx.connect(event, callback) +.. automethod:: Sphinx.add_source_suffix(suffix, filetype) - Register *callback* to be called when *event* is emitted. For details on - available core events and the arguments of callback functions, please see - :ref:`events`. +.. automethod:: Sphinx.add_source_parser(parser) - The method returns a "listener ID" that can be used as an argument to - :meth:`disconnect`. +.. automethod:: Sphinx.add_env_collector(collector) -.. method:: Sphinx.disconnect(listener_id) +.. automethod:: Sphinx.add_html_theme(name, theme_path) - Unregister callback *listener_id*. +.. automethod:: Sphinx.add_message_catalog(catalog, locale_dir) +.. automethod:: Sphinx.is_parallel_allowed(typ) .. exception:: ExtensionError @@ -414,18 +106,11 @@ package. Emitting events --------------- -.. method:: Sphinx.emit(event, *arguments) - - Emit *event* and pass *arguments* to the callback functions. Return the - return values of all callbacks as a list. Do not emit core Sphinx events - in extensions! - -.. method:: Sphinx.emit_firstresult(event, *arguments) +.. class:: Sphinx - Emit *event* and pass *arguments* to the callback functions. Return the - result of the first callback that doesn't return ``None``. + .. automethod:: emit(event, \*arguments) - .. versionadded:: 0.5 + .. automethod:: emit_firstresult(event, \*arguments) Producing messages / logging @@ -500,6 +185,12 @@ handlers to the events. Example: Emitted when the builder object has been created. It is available as ``app.builder``. +.. event:: config-inited (app, config) + + Emitted when the config object has been initialized. + + .. versionadded:: 1.8 + .. event:: env-get-outdated (app, env, added, changed, removed) Emitted when the environment determines which source files have changed and @@ -614,7 +305,7 @@ handlers to the events. Example: .. event:: env-check-consistency (env) - Emmited when Consistency checks phase. You can check consistency of + Emitted when Consistency checks phase. You can check consistency of metadata for whole of documents. .. versionadded:: 1.6 @@ -674,28 +365,15 @@ Checking the Sphinx version Use this to adapt your extension to API changes in Sphinx. -.. data:: version_info - - A tuple of five elements; for Sphinx version 1.2.1 beta 3 this would be - ``(1, 2, 1, 'beta', 3)``. - - .. versionadded:: 1.2 - Before version 1.2, check the string ``sphinx.__version__``. +.. autodata:: version_info The Config object ----------------- -.. module:: sphinx.config +.. currentmodule:: sphinx.config -.. class:: Config - - The config object makes the values of all config values available as - attributes. - - It is available as the ``config`` attribute on the application and - environment objects. For example, to get the value of :confval:`language`, - use either ``app.config.language`` or ``env.config.language``. +.. autoclass:: Config .. _template-bridge: @@ -716,38 +394,12 @@ Exceptions .. module:: sphinx.errors -.. exception:: SphinxError - - This is the base class for "nice" exceptions. When such an exception is - raised, Sphinx will abort the build and present the exception category and - message to the user. - - Extensions are encouraged to derive from this exception for their custom - errors. - - Exceptions *not* derived from :exc:`SphinxError` are treated as unexpected - and shown to the user with a part of the traceback (and the full traceback - saved in a temporary file). - - .. attribute:: category - - Description of the exception "category", used in converting the exception - to a string ("category: message"). Should be set accordingly in - subclasses. - -.. exception:: ConfigError - - Used for erroneous values or nonsensical combinations of configuration - values. - -.. exception:: ExtensionError - - Used for errors in setting up extensions. +.. autoexception:: SphinxError -.. exception:: ThemeError +.. autoexception:: ConfigError - Used for errors to do with themes. +.. autoexception:: ExtensionError -.. exception:: VersionRequirementError +.. autoexception:: ThemeError - Raised when the docs require a higher Sphinx version than the current one. +.. autoexception:: VersionRequirementError diff --git a/doc/extdev/builderapi.rst b/doc/extdev/builderapi.rst index b8ff0595b..2c2cf12e3 100644 --- a/doc/extdev/builderapi.rst +++ b/doc/extdev/builderapi.rst @@ -17,6 +17,9 @@ Builder API .. autoattribute:: format .. autoattribute:: epilog .. autoattribute:: supported_image_types + .. autoattribute:: supported_remote_images + .. autoattribute:: supported_data_uri_images + .. autoattribute:: default_translator_class These methods are predefined and will be called from the application: diff --git a/doc/extdev/i18n.rst b/doc/extdev/i18n.rst new file mode 100644 index 000000000..c8c54da36 --- /dev/null +++ b/doc/extdev/i18n.rst @@ -0,0 +1,17 @@ +.. _i18n-api: + +i18n API +======== + +.. currentmodule:: sphinx.locale + +.. autofunction:: init + +.. autofunction:: init_console + +.. autofunction:: get_translation + +.. autofunction:: _ + +.. autofunction:: __ + diff --git a/doc/extdev/index.rst b/doc/extdev/index.rst index ace43fd71..062467dbc 100644 --- a/doc/extdev/index.rst +++ b/doc/extdev/index.rst @@ -52,6 +52,8 @@ Note that it is still necessary to register the builder using .. _entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins +.. _ext-metadata: + Extension metadata ------------------ @@ -63,6 +65,11 @@ as metadata of the extension. Metadata keys currently recognized are: * ``'version'``: a string that identifies the extension version. It is used for extension version requirement checking (see :confval:`needs_extensions`) and informational purposes. If not given, ``"unknown version"`` is substituted. +* ``'env_version'``: an integer that identifies the version of env data + structure if the extension stores any data to environment. It is used to + detect the data structure has been changed from last build. The extensions + have to increment the version when data structure has changed. If not given, + Sphinx considers the extension does not stores any data to environment. * ``'parallel_read_safe'``: a boolean that specifies if parallel reading of source files can be used when the extension is loaded. It defaults to ``False``, i.e. you have to explicitly specify your extension to be @@ -86,6 +93,7 @@ APIs used for writing extensions parserapi nodes logging + i18n Deprecated APIs --------------- @@ -105,6 +113,67 @@ The following is a list of deprecated interface. - (will be) Removed - Alternatives + * - :confval:`source_parsers` + - 1.8 + - 3.0 + - :meth:`~sphinx.application.Sphinx.add_source_parser()` + + * - ``Sphinx.import_object()`` + - 1.8 + - 3.0 + - ``sphinx.util.import_object()`` + + * - ``suffix`` argument of + :meth:`~sphinx.application.Sphinx.add_source_parser()` + - 1.8 + - 3.0 + - :meth:`~sphinx.application.Sphinx.add_source_suffix()` + + * - ``sphinx.util.docutils.directive_helper()`` + - 1.8 + - 3.0 + - ``Directive`` class of docutils + + * - ``sphinx.cmdline`` + - 1.8 + - 3.0 + - ``sphinx.cmd.build`` + + * - ``BuildEnvironment.update()`` + - 1.8 + - 3.0 + - ``Builder.read()`` + + * - ``BuildEnvironment._read_serial()`` + - 1.8 + - 3.0 + - ``Builder.read()`` + + * - ``BuildEnvironment._read_parallel()`` + - 1.8 + - 3.0 + - ``Builder.read()`` + + * - ``sphinx.locale.l_()`` + - 1.8 + - 3.0 + - :func:`sphinx.locale._()` + + * - ``sphinx.locale.lazy_gettext()`` + - 1.8 + - 3.0 + - :func:`sphinx.locale._()` + + * - ``sphinx.locale.mygettext()`` + - 1.8 + - 3.0 + - :func:`sphinx.locale._()` + + * - ``sphinx.util.copy_static_entry()`` + - 1.5 + - 3.0 + - ``sphinx.util.fileutil.copy_asset()`` + * - ``sphinx.build_main()`` - 1.7 - 2.0 diff --git a/doc/extdev/logging.rst b/doc/extdev/logging.rst index 50110c2a4..2e735e286 100644 --- a/doc/extdev/logging.rst +++ b/doc/extdev/logging.rst @@ -3,18 +3,11 @@ Logging API =========== -.. function:: sphinx.util.logging.getLogger(name) +.. currentmodule:: sphinx.util.logging - Returns a logger wrapped by :class:`SphinxLoggerAdapter` with the specified *name*. +.. autofunction:: getLogger(name) - Example usage:: - - from sphinx.util import logging # Load on top of python's logging module - - logger = logging.getLogger(__name__) - logger.info('Hello, this is an extension!') - -.. class:: SphinxLoggerAdapter(logging.LoggerAdapter) +.. autoclass:: SphinxLoggerAdapter(logging.LoggerAdapter) .. method:: SphinxLoggerAdapter.error(level, msg, *args, **kwargs) .. method:: SphinxLoggerAdapter.critical(level, msg, *args, **kwargs) @@ -62,16 +55,6 @@ Logging API colored as ``"darkgray"``, and debug2 level ones are ``"lightgray"``. The others are not colored. -.. function:: pending_logging() - - Marks all logs as pending:: - - with pending_logging(): - logger.warning('Warning message!') # not flushed yet - some_long_process() - - # the warning is flushed here - -.. function:: pending_warnings() +.. autofunction:: pending_logging() - Marks warning logs as pending. Similar to :func:`pending_logging`. +.. autofunction:: pending_warnings() diff --git a/doc/extdev/parserapi.rst b/doc/extdev/parserapi.rst index 7008a15a8..e71661b2e 100644 --- a/doc/extdev/parserapi.rst +++ b/doc/extdev/parserapi.rst @@ -3,6 +3,35 @@ Parser API ========== +`The docutils documentation describes`__ parsers as follows: + + The Parser analyzes the input document and creates a node tree + representation. + +__ http://docutils.sourceforge.net/docs/dev/hacking.html#parsing-the-document + +In Sphinx, the parser modules works as same as docutils. The parsers are +registered to Sphinx by extensions using Application APIs; +:meth:`Sphinx.add_source_suffix()` and :meth:`Sphinx.add_source_parsers()`. + +The *source suffix* is a mapping from file suffix to file type. For example, +``.rst`` file is mapped to ``'restructuredtext'`` type. Sphinx uses the +file type to looking for parsers from registered list. On searching, +Sphinx refers to the ``Parser.supported`` attribute and picks up a parser +which contains the file type in the attribute. + +The users can override the source suffix mappings using +:confval:`source_suffix` like following:: + + # a mapping from file suffix to file types + source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', + } + +You should indicate file types your parser supports. This will allow users +to configure their settings appropriately. + .. module:: sphinx.parsers .. autoclass:: Parser diff --git a/doc/faq.rst b/doc/faq.rst index fe3173749..8798fd5db 100644 --- a/doc/faq.rst +++ b/doc/faq.rst @@ -11,12 +11,9 @@ How do I... ... create PDF files without LaTeX? `rinohtype`_ provides a PDF builder that can be used as a drop-in - replacement for the LaTeX builder. Alternatively, you can use `rst2pdf`_ - version 0.12 or greater which comes with built-in Sphinx integration. See - the :ref:`builders` section for details. + replacement for the LaTeX builder. .. _rinohtype: https://github.com/brechtm/rinohtype - .. _rst2pdf: https://github.com/rst2pdf/rst2pdf ... get section numbers? They are automatic in LaTeX output; for HTML, give a ``:numbered:`` option to @@ -77,7 +74,7 @@ PyPI Jannis Leidel wrote a `setuptools command <https://pypi.python.org/pypi/Sphinx-PyPI-upload>`_ that automatically uploads Sphinx documentation to the PyPI package documentation area at - http://pythonhosted.org/. + https://pythonhosted.org/. GitHub Pages Directories starting with underscores are ignored by default which breaks @@ -106,7 +103,7 @@ Google Analytics {% block footer %} {{ super() }} - <div class="footer">This page uses <a href="http://analytics.google.com/"> + <div class="footer">This page uses <a href="https://analytics.google.com/"> Google Analytics</a> to collect statistics. You can disable it by blocking the JavaScript coming from www.google-analytics.com. <script type="text/javascript"> @@ -122,7 +119,7 @@ Google Analytics {% endblock %} -.. _api role: http://git.savannah.gnu.org/cgit/kenozooid.git/tree/doc/extapi.py +.. _api role: https://git.savannah.gnu.org/cgit/kenozooid.git/tree/doc/extapi.py .. _xhtml to reST: http://docutils.sourceforge.net/sandbox/xhtml2rest/xhtml2rest.py diff --git a/doc/glossary.rst b/doc/glossary.rst index a92e52b98..acac3a590 100644 --- a/doc/glossary.rst +++ b/doc/glossary.rst @@ -86,3 +86,7 @@ Glossary source directory The directory which, including its subdirectories, contains all source files for one Sphinx project. + + reStructuredText + An easy-to-read, what-you-see-is-what-you-get plaintext markup syntax and + parser system. diff --git a/doc/install.rst b/doc/install.rst deleted file mode 100644 index 2eb44b809..000000000 --- a/doc/install.rst +++ /dev/null @@ -1,155 +0,0 @@ -:orphan: - -Installing Sphinx -================= - -Since Sphinx is written in the Python language, you need to install Python -(the required version is at least 2.7) and Sphinx. - -Sphinx packages are available on the `Python Package Index -<https://pypi.python.org/pypi/Sphinx>`_. - -You can also download a snapshot from the Git repository: - -* as a `.tar.gz <https://github.com/sphinx-doc/sphinx/archive/master.tar.gz>`__ - file or -* as a `.zip <https://github.com/sphinx-doc/sphinx/archive/master.zip>`_ file - -There are introductions for several environments: - -.. contents:: - :depth: 1 - :local: - :backlinks: none - - -Debian/Ubuntu: Install Sphinx using packaging system ----------------------------------------------------- - -You may install using this command if you use Debian/Ubuntu. - -.. code-block:: bash - - $ apt-get install python-sphinx - - -Other Linux distributions -------------------------- - -Most Linux distributions have Sphinx in their package repositories. Usually the -package is called "python-sphinx", "python-Sphinx" or "sphinx". Be aware that -there are two other packages with "sphinx" in their name: a speech recognition -toolkit (CMU Sphinx) and a full-text search database (Sphinx search). - - -Mac OS X: Install Sphinx using MacPorts ---------------------------------------- - -If you use Mac OS X `MacPorts <http://www.macports.org/>`_, use this command to -install all necessary software. - -.. code-block:: bash - - $ sudo port install py27-sphinx - -To set up the executable paths, use the ``port select`` command: - -.. code-block:: bash - - $ sudo port select --set python python27 - $ sudo port select --set sphinx py27-sphinx - -Type :command:`which sphinx-quickstart` to check if the installation was -successful. - - -Windows: Install Python and Sphinx ----------------------------------- - -Install Python -^^^^^^^^^^^^^^ - -Most Windows users do not have Python, so we begin with the installation of -Python itself. If you have already installed Python, please skip this section. - -Go to https://www.python.org/, the main download site for Python. Look at the left -sidebar and under "Quick Links", click "Windows Installer" to download. - -.. image:: pythonorg.png - -.. note:: - - Currently, Python offers two major versions, 2.x and 3.x. Sphinx 1.5 can run - under Python 2.7, 3.4, 3.5, 3.6, with the recommended version being 2.7. This - chapter assumes you have installed Python 2.7. - -Follow the Windows installer for Python. - -.. image:: installpython.jpg - -After installation, you better add the Python executable directories to the -environment variable ``PATH`` in order to run Python and package commands such -as ``sphinx-build`` easily from the Command Prompt. - -* Right-click the "My Computer" icon and choose "Properties" -* Click the "Environment Variables" button under the "Advanced" tab - -* If "Path" (or "PATH") is already an entry in the "System variables" list, edit - it. If it is not present, add a new variable called "PATH". - -* Add these paths, separating entries by ";": - - - ``C:\Python27`` -- this folder contains the main Python executable - - ``C:\Python27\Scripts`` -- this folder will contain executables added by - Python packages installed with pip (see below) - - This is for Python 2.7. If you use another version of - Python or installed to a non-default location, change the digits "27" - accordingly. - -* Now run the **Command Prompt**. After command prompt window appear, type - ``python`` and Enter. If the Python installation was successful, the - installed Python version is printed, and you are greeted by the prompt - ``>>>``. Type ``Ctrl+Z`` and Enter to quit. - - -Install the pip command -^^^^^^^^^^^^^^^^^^^^^^^ - -Python has a very useful :command:`pip` command which can download and install -3rd-party libraries with a single command. This is provided by the -Python Packaging Authority(PyPA): -https://groups.google.com/forum/#!forum/pypa-dev - -To install pip, download https://bootstrap.pypa.io/get-pip.py and -save it somewhere. After download, invoke the command prompt, go to the -directory with ``get-pip.py`` and run this command: - -.. code-block:: bat - - C:\> python get-pip.py - -Now :command:`pip` command is installed. From there we can go to the Sphinx -install. - -.. note:: - - ``pip`` has been contained in the Python official installation after version - of Python-3.4.0 or Python-2.7.9. - - -Installing Sphinx with pip --------------------------- - -If you finished the installation of pip, type this line in the command prompt: - -.. code-block:: bat - - C:\> pip install sphinx - -After installation, type :command:`sphinx-build -h` on the command prompt. If -everything worked fine, you will get a Sphinx version number and a list of -options for this command. - -That it. Installation is over. Head to :doc:`tutorial` to make a Sphinx -project. diff --git a/doc/installpython.jpg b/doc/installpython.jpg Binary files differdeleted file mode 100644 index fff4630ae..000000000 --- a/doc/installpython.jpg +++ /dev/null diff --git a/doc/intl.rst b/doc/intl.rst index 870d06240..ac4f47079 100644 --- a/doc/intl.rst +++ b/doc/intl.rst @@ -13,7 +13,7 @@ in itself. See the :ref:`intl-options` for details on configuration. :width: 100% Workflow visualization of translations in Sphinx. (The stick-figure is taken - from an `XKCD comic <http://xkcd.com/779/>`_.) + from an `XKCD comic <https://xkcd.com/779/>`_.) .. contents:: :local: @@ -317,7 +317,7 @@ There is `sphinx translation page`_ for Sphinx-1.3 documentation. .. rubric:: Footnotes .. [1] See the `GNU gettext utilities - <http://www.gnu.org/software/gettext/manual/gettext.html#Introduction>`_ + <https://www.gnu.org/software/gettext/manual/gettext.html#Introduction>`_ for details on that software suite. .. [2] Because nobody expects the Spanish Inquisition! @@ -326,4 +326,4 @@ There is `sphinx translation page`_ for Sphinx-1.3 documentation. .. _`sphinx-intl`: https://pypi.python.org/pypi/sphinx-intl .. _Transifex: https://www.transifex.com/ .. _`sphinx translation page`: https://www.transifex.com/sphinx-doc/sphinx-doc-1_3/ -.. _`Transifex Client v0.8 — Transifex documentation`: http://docs.transifex.com/developer/client/ +.. _`Transifex Client v0.8 — Transifex documentation`: https://docs.transifex.com/client/introduction/ diff --git a/doc/intro.rst b/doc/intro.rst index a789145fe..e2a23f95a 100644 --- a/doc/intro.rst +++ b/doc/intro.rst @@ -8,7 +8,7 @@ you have a directory containing a bunch of reST-formatted documents (and possibly subdirectories of docs in there as well), Sphinx can generate a nicely-organized arrangement of HTML files (in some other directory) for easy browsing and navigation. But from the same source, it can also generate a PDF -file using LaTeX, `rinohtype`_ or `rst2pdf`_ (see :ref:`builders`). +file using LaTeX. The focus is on hand-written documentation, rather than auto-generated API docs. Though there is support for that kind of documentation as well (which is @@ -21,7 +21,6 @@ also `Write the docs <https://write-the-docs.readthedocs.io/>`_, written by Eric Holscher. .. _rinohtype: https://github.com/brechtm/rinohtype -.. _rst2pdf: https://github.com/rst2pdf/rst2pdf Conversion from other systems ----------------------------- @@ -34,7 +33,7 @@ to reStructuredText/Sphinx from other documentation systems. * For converting the old Python docs to Sphinx, a converter was written which can be found at `the Python SVN repository - <http://svn.python.org/projects/doctools/converter>`_. It contains generic + <https://svn.python.org/projects/doctools/converter/>`_. It contains generic code to convert Python-doc-style LaTeX markup to Sphinx reST. * Marcin Wojdyr has written a script to convert Docbook to reST with Sphinx @@ -43,7 +42,7 @@ to reStructuredText/Sphinx from other documentation systems. * Christophe de Vienne wrote a tool to convert from Open/LibreOffice documents to Sphinx: `odt2sphinx <https://pypi.python.org/pypi/odt2sphinx/>`_. -* To convert different markups, `Pandoc <http://pandoc.org/>`_ is +* To convert different markups, `Pandoc <https://pandoc.org/>`_ is a very helpful tool. @@ -70,5 +69,5 @@ highlighting support, you must also install the Pygments_ library. Usage ----- -See :doc:`tutorial` for an introduction. It also contains links to more -advanced sections in this manual for the topics it discusses. +See :doc:`/usage/quickstart` for an introduction. It also contains links to +more advanced sections in this manual for the topics it discusses. diff --git a/doc/man/sphinx-apidoc.rst b/doc/man/sphinx-apidoc.rst index 9a13f1401..8f75d6f7c 100644 --- a/doc/man/sphinx-apidoc.rst +++ b/doc/man/sphinx-apidoc.rst @@ -4,7 +4,7 @@ sphinx-apidoc Synopsis -------- -**sphinx-apidoc** [*options*] -o <*outputdir*> <*sourcedir*> [*pathnames* ...] +**sphinx-apidoc** [*OPTIONS*] -o <*OUTPUT_PATH*> <*MODULE_PATH*> [*EXCLUDE_PATTERN*, ...] Description ----------- @@ -13,9 +13,10 @@ Description that, using the :rst:dir:`autodoc` extension, document a whole package in the style of other automatic API documentation tools. -*sourcedir* is the path to a Python package to document, and *outputdir* is the -directory where the generated sources are placed. Any *pathnames* given are -paths to be excluded from the generation. +*MODULE_PATH* is the path to a Python package to document, and *OUTPUT_PATH* is +the directory where the generated sources are placed. Any *EXCLUDE_PATTERN*\s +given are `fnmatch-style <fnmatch>`_ file and/or directory patterns that will +be excluded from generation. .. warning:: @@ -31,7 +32,7 @@ Options .. program:: sphinx-apidoc -.. option:: -o <outputdir> +.. option:: -o <OUTPUT_PATH> Directory to place the output files. If it does not exist, it is created. @@ -51,7 +52,7 @@ Options Suffix for the source files generated. Defaults to ``rst``. -.. option:: -d <maxdepth> +.. option:: -d <MAXDEPTH> Maximum depth for the generated table of contents file. @@ -130,3 +131,5 @@ See also -------- :manpage:`sphinx-build(1)`, :manpage:`sphinx-autogen(1)` + +.. _fnmatch: https://docs.python.org/3/library/fnmatch.html diff --git a/doc/man/sphinx-quickstart.rst b/doc/man/sphinx-quickstart.rst index c4bbc531b..1d7a99add 100644 --- a/doc/man/sphinx-quickstart.rst +++ b/doc/man/sphinx-quickstart.rst @@ -20,7 +20,7 @@ Options .. option:: -q, --quiet - Quiet mode that will skips interactive wizard to specify options. + Quiet mode that will skip interactive wizard to specify options. This option requires `-p`, `-a` and `-v` options. .. option:: -h, --help, --version @@ -112,13 +112,17 @@ Options Enable `sphinx.ext.viewcode` extension. +.. option:: --ext-githubpages + + Enable `sphinx.ext.githubpages` extension. + .. option:: --extensions=EXTENSIONS - Enable arbitary extensions. + Enable arbitrary extensions. .. rubric:: Makefile and Batchfile Creation Options -.. option:: --use-make-mode, --no-use-make-mode +.. option:: --use-make-mode (-m), --no-use-make-mode (-M) :file:`Makefile/make.bat` uses (or doesn't use) :ref:`make-mode <make_mode>`. Default is ``use``, which generates a more concise :file:`Makefile/make.bat`. diff --git a/doc/markdown.rst b/doc/markdown.rst index 75a35b9d0..e94a821da 100644 --- a/doc/markdown.rst +++ b/doc/markdown.rst @@ -9,7 +9,7 @@ Markdown support text formatting syntax. It exists in many syntactically different *flavors*. To support Markdown-based documentation, Sphinx can use -`recommonmark <http://recommonmark.readthedocs.io/en/latest/index.html>`__. +`recommonmark <https://recommonmark.readthedocs.io/en/latest/index.html>`__. recommonmark is a Docutils bridge to `CommonMark-py <https://github.com/rtfd/CommonMark-py>`__, a Python package for parsing the `CommonMark <http://commonmark.org/>`__ Markdown flavor. @@ -42,4 +42,4 @@ To configure your Sphinx project for Markdown support, proceed as follows: source_suffix = ['.rst', '.md'] #. You can further configure recommonmark to allow custom syntax that standard CommonMark doesn't support. Read more in - the `recommonmark documentation <http://recommonmark.readthedocs.io/en/latest/auto_structify.html>`__. + the `recommonmark documentation <https://recommonmark.readthedocs.io/en/latest/auto_structify.html>`__. diff --git a/doc/pythonorg.png b/doc/pythonorg.png Binary files differdeleted file mode 100644 index cf9ccbbdb..000000000 --- a/doc/pythonorg.png +++ /dev/null diff --git a/doc/rest.rst b/doc/rest.rst index fbc3f2254..b059b873b 100644 --- a/doc/rest.rst +++ b/doc/rest.rst @@ -194,7 +194,7 @@ Hyperlinks External links ^^^^^^^^^^^^^^ -Use ```Link text <http://example.com/>`_`` for inline web links. If the link +Use ```Link text <https://domain.invalid/>`_`` for inline web links. If the link text should be the web address, you don't need special markup at all, the parser finds links and mail addresses in ordinary text. @@ -205,7 +205,7 @@ You can also separate the link and the target definition (:duref:`ref This is a paragraph that contains `a link`_. - .. _a link: http://example.com/ + .. _a link: https://domain.invalid/ Internal links diff --git a/doc/templating.rst b/doc/templating.rst index 66f72716b..78d5e3c45 100644 --- a/doc/templating.rst +++ b/doc/templating.rst @@ -62,7 +62,7 @@ following contents:: {% extends "!layout.html" %} {% block rootrellink %} - <li><a href="http://project.invalid/">Project Homepage</a> »</li> + <li><a href="https://project.invalid/">Project Homepage</a> »</li> {{ super() }} {% endblock %} @@ -70,8 +70,8 @@ By prefixing the name of the overridden template with an exclamation mark, Sphinx will load the layout template from the underlying HTML theme. **Important**: If you override a block, call ``{{ super() }}`` somewhere to -render the block's content in the extended template -- unless you don't want -that content to show up. +render the block's original content in the extended template -- unless you +don't want that content to show up. Working with the builtin templates diff --git a/doc/theming.rst b/doc/theming.rst index 32aef20fa..67dd6adc2 100644 --- a/doc/theming.rst +++ b/doc/theming.rst @@ -135,7 +135,7 @@ These themes are: :confval:`html_sidebars` for its use. .. _Alabaster theme: https://pypi.python.org/pypi/alabaster - .. _installation page: http://alabaster.readthedocs.io/en/latest/installation.html + .. _installation page: https://alabaster.readthedocs.io/en/latest/installation.html * **classic** -- This is the classic theme, which looks like `the Python 2 documentation <https://docs.python.org/2/>`_. It can be customized via @@ -422,3 +422,11 @@ Third Party Themes .. versionchanged:: 1.4 **sphinx_rtd_theme** has become optional. + + +Besides this, there are a lot of third party themes. You can find them on +PyPI__, GitHub__, sphinx-themes.org__ and so on. + +.. __: https://pypi.python.org/pypi?:action=browse&c=599 +.. __: https://github.com/search?utf8=%E2%9C%93&q=sphinx+theme&type= +.. __: https://sphinx-themes.org/ diff --git a/doc/usage/installation.rst b/doc/usage/installation.rst new file mode 100644 index 000000000..7ccdd677f --- /dev/null +++ b/doc/usage/installation.rst @@ -0,0 +1,179 @@ +================= +Installing Sphinx +================= + +.. contents:: + :depth: 1 + :local: + :backlinks: none + +Overview +-------- + +Sphinx is written in `Python`__ and supports both Python 2.7 and Python 3.3+. +We recommend the latter. + +__ http://docs.python-guide.org/en/latest/ + + +Linux +----- + +Debian/Ubuntu +~~~~~~~~~~~~~ + +Install either ``python3-sphinx`` (Python 3) or ``python-sphinx`` (Python 2) +using :command:`apt-get`: + +.. code-block:: bash + + $ apt-get install python3-sphinx + +If it not already present, this will install Python for you. + +RHEL, CentOS +~~~~~~~~~~~~ + +Install ``python-sphinx`` using :command:`yum`: + +.. code-block:: bash + + $ yum install python-sphinx + +If it not already present, this will install Python for you. + +Other distributions +~~~~~~~~~~~~~~~~~~~ + +Most Linux distributions have Sphinx in their package repositories. Usually +the package is called ``python3-sphinx``, ``python-sphinx`` or ``sphinx``. Be +aware that there are at least two other packages with ``sphinx`` in their name: +a speech recognition toolkit (*CMU Sphinx*) and a full-text search database +(*Sphinx search*). + + +macOS +----- + +Sphinx can be installed using `Homebrew`__ or `MacPorts`__. + +__ https://brew.sh/ +__ https://www.macports.org/ + +Homebrew +~~~~~~~~ + +.. code-block:: bash + + $ brew install sphinx-doc + +For more information, refer to the `package overview`__. + +__ http://formulae.brew.sh/formula/sphinx-doc + +MacPorts +~~~~~~~~ + +Install either ``python36-sphinx`` (Python 3) or ``python27-sphinx`` (Python 2) +using :command:`port`: + +.. code-block:: bash + + $ sudo port install py36-sphinx + +To set up the executable paths, use the ``port select`` command: + +.. code-block:: bash + + $ sudo port select --set python python36 + $ sudo port select --set sphinx py36-sphinx + +For more information, refer to the `package overview`__. + +__ https://www.macports.org/ports.php?by=library&substr=py36-sphinx + + +Windows +------- + +.. todo:: Could we start packaging this? + +Most Windows users do not have Python installed by default, so we begin with +the installation of Python itself. If you are unsure, open the *Command +Prompt* (:kbd:`Control-r` and type :command:`cmd`). Once the command prompt is +open, type :command:`python --version` and press Enter. If Python is +available, you will see the version of Python printed to the screen. If you do +not have Python installed, refer to the `Hitchhikers Guide to Python's`__ +Python on Windows installation guides. You can install either `Python 3`__ or +`Python 2.7`__. Python 3 is recommended. + +Once Python is installed, you can install Sphinx using :command:`pip`. Refer +to the :ref:`pip installation instructions <install-pypi>` below for more +information. + +__ http://docs.python-guide.org/en/latest/ +__ http://docs.python-guide.org/en/latest/starting/install3/win/ +__ http://docs.python-guide.org/en/latest/starting/install/win/ + + +.. _install-pypi: + +Installation from PyPI +---------------------- + +Sphinx packages are published on the `Python Package Index +<https://pypi.python.org/pypi/Sphinx>`_. The preferred tool for installing +packages from *PyPI* is :command:`pip`. This tool is provided with all modern +versions of Python. + +On Linux or MacOS, you should open your terminal and run the following command. + +.. code-block:: shell + + $ pip install -U sphinx + +On Windows, you should open *Command Prompt* (:kbd:`Control-r` and type +:command:`cmd`) and run the same command. + +.. code-block:: bat + + C:\> pip install -U sphinx + +After installation, type :command:`sphinx-build --version` on the command +prompt. If everything worked fine, you will see the version number for the +Sphinx package you just installed. + +Installation from *PyPI* also allows you to install the latest development +release. You will not generally need (or want) to do this, but it can be +useful if you see a possible bug in the latest stable release. To do this, use +the ``--pre`` flag. + +.. code-block:: shell + + $ pip install -U --pre sphinx + + +Installation from source +------------------------ + +You can install Sphinx directly from a clone of the `Git repository`__. This +can be done either by cloning the repo and installing from the local clone, on +simply installing directly via :command:`git`. + +.. code-block:: shell + + $ git clone https://github.com/sphinx-doc/sphinx + $ cd sphinx + $ pip install . + +.. code-block:: shell + + $ pip install git+https://github.com/sphinx-doc/sphinx + +You can also download a snapshot of the Git repo in either `tar.gz`__ or +`zip`__ format. Once downloaded and extracted, these can be installed with +:command:`pip` as above. + +__ https://github.com/sphinx-doc/sphinx +__ https://github.com/sphinx-doc/sphinx/archive/master.tar.gz +__ https://github.com/sphinx-doc/sphinx/archive/master.zip diff --git a/doc/tutorial.rst b/doc/usage/quickstart.rst index 243d44d03..650410333 100644 --- a/doc/tutorial.rst +++ b/doc/usage/quickstart.rst @@ -1,39 +1,33 @@ -.. highlight:: rst +=============== +Getting Started +=============== -First Steps with Sphinx -======================= - -This document is meant to give a tutorial-like overview of all common tasks -while using Sphinx. - -The green arrows designate "more info" links leading to advanced sections about -the described task. - - -Install Sphinx --------------- - -Install Sphinx, either from a distribution package or from -`PyPI <https://pypi.python.org/pypi/Sphinx>`_ with :: - - $ pip install Sphinx +Once Sphinx is :doc:`installed </usage/installation>`, you can proceed with +setting up your first Sphinx project. To ease the process of getting started, +Sphinx provides a tool, :program:`sphinx-quickstart`, which will generate a +documentation source directory and populate it with some defaults. We're going +to use the :program:`sphinx-quickstart` tool here, though it's use by no means +necessary. Setting up the documentation sources ------------------------------------ -The root directory of a Sphinx collection of reStructuredText document sources -is called the :term:`source directory`. This directory also contains the Sphinx -configuration file :file:`conf.py`, where you can configure all aspects of how -Sphinx reads your sources and builds your documentation. [#]_ +The root directory of a Sphinx collection of :term:`reStructuredText` document +sources is called the :term:`source directory`. This directory also contains +the Sphinx configuration file :file:`conf.py`, where you can configure all +aspects of how Sphinx reads your sources and builds your documentation. [#]_ Sphinx comes with a script called :program:`sphinx-quickstart` that sets up a source directory and creates a default :file:`conf.py` with the most useful -configuration values from a few questions it asks you. Just run :: +configuration values from a few questions it asks you. To use this, run: + +.. code-block:: shell $ sphinx-quickstart -and answer its questions. (Be sure to say yes to the "autodoc" extension.) +Answer each question asked. Be sure to say yes to the ``autodoc`` extension, as +we will use this later. There is also an automatic "API documentation" generator called :program:`sphinx-apidoc`; see :doc:`/man/sphinx-apidoc` for details. @@ -45,15 +39,15 @@ Defining document structure Let's assume you've run :program:`sphinx-quickstart`. It created a source directory with :file:`conf.py` and a master document, :file:`index.rst` (if you accepted the defaults). The main function of the :term:`master document` is to -serve as a welcome page, and to contain the root of the "table of contents tree" -(or *toctree*). This is one of the main things that Sphinx adds to +serve as a welcome page, and to contain the root of the "table of contents +tree" (or *toctree*). This is one of the main things that Sphinx adds to reStructuredText, a way to connect multiple files to a single hierarchy of documents. .. sidebar:: reStructuredText directives - ``toctree`` is a reStructuredText :dfn:`directive`, a very versatile piece of - markup. Directives can have arguments, options and content. + ``toctree`` is a reStructuredText :dfn:`directive`, a very versatile piece + of markup. Directives can have arguments, options and content. *Arguments* are given directly after the double colon following the directive's name. Each directive decides whether it can have arguments, and @@ -68,45 +62,52 @@ documents. A common gotcha with directives is that **the first line of the content must be indented to the same level as the options are**. +The ``toctree`` directive initially is empty, and looks like so: -The toctree directive initially is empty, and looks like this:: +.. code-block:: rest .. toctree:: :maxdepth: 2 -You add documents listing them in the *content* of the directive:: +You add documents listing them in the *content* of the directive: + +.. code-block:: rest .. toctree:: :maxdepth: 2 - intro - tutorial + usage/installation + usage/quickstart ... -This is exactly how the toctree for this documentation looks. The documents to -include are given as :term:`document name`\ s, which in short means that you -leave off the file name extension and use slashes as directory separators. +This is exactly how the ``toctree`` for this documentation looks. The +documents to include are given as :term:`document name`\ s, which in short +means that you leave off the file name extension and use forward slashes +(``/``) as directory separators. |more| Read more about :ref:`the toctree directive <toctree-directive>`. -You can now create the files you listed in the toctree and add content, and -their section titles will be inserted (up to the "maxdepth" level) at the place -where the toctree directive is placed. Also, Sphinx now knows about the order -and hierarchy of your documents. (They may contain ``toctree`` directives -themselves, which means you can create deeply nested hierarchies if necessary.) +You can now create the files you listed in the ``toctree`` and add content, and +their section titles will be inserted (up to the ``maxdepth`` level) at the +place where the ``toctree`` directive is placed. Also, Sphinx now knows about +the order and hierarchy of your documents. (They may contain ``toctree`` +directives themselves, which means you can create deeply nested hierarchies if +necessary.) Adding content -------------- -In Sphinx source files, you can use most features of standard reStructuredText. -There are also several features added by Sphinx. For example, you can add -cross-file references in a portable way (which works for all output types) using -the :rst:role:`ref` role. +In Sphinx source files, you can use most features of standard +:term:`reStructuredText`. There are also several features added by Sphinx. +For example, you can add cross-file references in a portable way (which works +for all output types) using the :rst:role:`ref` role. For an example, if you are viewing the HTML version you can look at the source for this document -- use the "Show Source" link in the sidebar. +.. todo:: Update the below link when we add new guides on these. + |more| See :ref:`rst-primer` for a more in-depth introduction to reStructuredText and :ref:`sphinxmarkup` for a full list of markup added by Sphinx. @@ -116,8 +117,9 @@ Running the build ----------------- Now that you have added some files and content, let's make a first build of the -docs. A build is started with the :program:`sphinx-build` program, called like -this:: +docs. A build is started with the :program:`sphinx-build` program: + +.. code-block:: shell $ sphinx-build -b html sourcedir builddir @@ -130,13 +132,15 @@ Sphinx will build HTML files. options that :program:`sphinx-build` supports. However, :program:`sphinx-quickstart` script creates a :file:`Makefile` and a -:file:`make.bat` which make life even easier for you: with them you only need -to run :: +:file:`make.bat` which make life even easier for you. These can be executed by +running :command:`make` with the name of the builder. For example. + +.. code-block:: shell $ make html -to build HTML docs in the build directory you chose. Execute ``make`` without -an argument to see which targets are available. +This will build HTML docs in the build directory you chose. Execute +:command:`make` without an argument to see which targets are available. .. admonition:: How do I generate PDF documents? @@ -145,6 +149,8 @@ an argument to see which targets are available. toolchain for you. +.. todo:: Move this whole section into a guide on rST or directives + Documenting objects ------------------- @@ -155,7 +161,9 @@ descriptions of these objects. The most prominent domain is the Python domain. For example, to document Python's built-in function ``enumerate()``, you would add this to one of your -source files:: +source files. + +.. code-block:: restructuredtext .. py:function:: enumerate(sequence[, start=0]) @@ -174,7 +182,9 @@ describe, the content is the documentation for it. Multiple signatures can be given, each in its own line. The Python domain also happens to be the default domain, so you don't need to -prefix the markup with the domain name:: +prefix the markup with the domain name. + +.. code-block:: restructuredtext .. function:: enumerate(sequence[, start=0]) @@ -182,18 +192,20 @@ prefix the markup with the domain name:: does the same job if you keep the default setting for the default domain. -There are several more directives for documenting other types of Python objects, -for example :rst:dir:`py:class` or :rst:dir:`py:method`. There is also a -cross-referencing :dfn:`role` for each of these object types. This markup will -create a link to the documentation of ``enumerate()``:: +There are several more directives for documenting other types of Python +objects, for example :rst:dir:`py:class` or :rst:dir:`py:method`. There is +also a cross-referencing :dfn:`role` for each of these object types. This +markup will create a link to the documentation of ``enumerate()``. + +:: The :py:func:`enumerate` function can be used for ... And here is the proof: A link to :func:`enumerate`. Again, the ``py:`` can be left out if the Python domain is the default one. It -doesn't matter which file contains the actual documentation for ``enumerate()``; -Sphinx will find it and create a link to it. +doesn't matter which file contains the actual documentation for +``enumerate()``; Sphinx will find it and create a link to it. Each domain will have special rules for how the signatures can look like, and make the formatted output look pretty, or add specific features like links to @@ -206,42 +218,45 @@ directives/roles. Basic configuration ------------------- -Earlier we mentioned that the :file:`conf.py` file controls how Sphinx processes -your documents. In that file, which is executed as a Python source file, you -assign configuration values. For advanced users: since it is executed by -Sphinx, you can do non-trivial tasks in it, like extending :data:`sys.path` or -importing a module to find out the version you are documenting. +Earlier we mentioned that the :file:`conf.py` file controls how Sphinx +processes your documents. In that file, which is executed as a Python source +file, you assign configuration values. For advanced users: since it is +executed by Sphinx, you can do non-trivial tasks in it, like extending +:data:`sys.path` or importing a module to find out the version you are +documenting. The config values that you probably want to change are already put into the :file:`conf.py` by :program:`sphinx-quickstart` and initially commented out -(with standard Python syntax: a ``#`` comments the rest of the line). To change -the default value, remove the hash sign and modify the value. To customize a -config value that is not automatically added by :program:`sphinx-quickstart`, -just add an additional assignment. +(with standard Python syntax: a ``#`` comments the rest of the line). To +change the default value, remove the hash sign and modify the value. To +customize a config value that is not automatically added by +:program:`sphinx-quickstart`, just add an additional assignment. -Keep in mind that the file uses Python syntax for strings, numbers, lists and so -on. The file is saved in UTF-8 by default, as indicated by the encoding +Keep in mind that the file uses Python syntax for strings, numbers, lists and +so on. The file is saved in UTF-8 by default, as indicated by the encoding declaration in the first line. If you use non-ASCII characters in any string value, you need to use Python Unicode strings (like ``project = u'Exposé'``). |more| See :ref:`build-config` for documentation of all available config values. +.. todo:: Move this entire doc to a different section + Autodoc ------- When documenting Python code, it is common to put a lot of documentation in the source files, in documentation strings. Sphinx supports the inclusion of docstrings from your modules with an :dfn:`extension` (an extension is a Python -module that provides additional features for Sphinx projects) called "autodoc". +module that provides additional features for Sphinx projects) called *autodoc*. -In order to use autodoc, you need to activate it in :file:`conf.py` by putting -the string ``'sphinx.ext.autodoc'`` into the list assigned to the +In order to use *autodoc*, you need to activate it in :file:`conf.py` by +putting the string ``'sphinx.ext.autodoc'`` into the list assigned to the :confval:`extensions` config value. Then, you have a few additional directives at your disposal. -For example, to document the function ``io.open()``, reading its -signature and docstring from the source file, you'd write this:: +For example, to document the function ``io.open()``, reading its signature and +docstring from the source file, you'd write this:: .. autofunction:: io.open @@ -251,7 +266,7 @@ options for the auto directives, like :: .. automodule:: io :members: -autodoc needs to import your modules in order to extract the docstrings. +*autodoc* needs to import your modules in order to extract the docstrings. Therefore, you must add the appropriate path to :py:data:`sys.path` in your :file:`conf.py`. @@ -261,17 +276,20 @@ Therefore, you must add the appropriate path to :py:data:`sys.path` in your modules have side effects on import, these will be executed by ``autodoc`` when ``sphinx-build`` is run. - If you document scripts (as opposed to library modules), make sure their main - routine is protected by a ``if __name__ == '__main__'`` condition. + If you document scripts (as opposed to library modules), make sure their + main routine is protected by a ``if __name__ == '__main__'`` condition. |more| See :mod:`sphinx.ext.autodoc` for the complete description of the features of autodoc. + +.. todo:: Move this doc to another section + Intersphinx ----------- -Many Sphinx documents including the `Python documentation`_ are published on the -internet. When you want to make links to such documents from your +Many Sphinx documents including the `Python documentation`_ are published on +the internet. When you want to make links to such documents from your documentation, you can do it with :mod:`sphinx.ext.intersphinx`. .. _Python documentation: https://docs.python.org/3 @@ -290,8 +308,8 @@ cross-reference that has no matching target in the current documentation set, will be looked up in the documentation sets configured in :confval:`intersphinx_mapping` (this needs access to the URL in order to download the list of valid targets). Intersphinx also works for some other -:ref:`domains' <domains>` roles including ``:ref:``, however it doesn't work for -``:doc:`` as that is non-domain role. +:ref:`domains' <domains>` roles including ``:ref:``, however it doesn't work +for ``:doc:`` as that is non-domain role. |more| See :mod:`sphinx.ext.intersphinx` for the complete description of the features of intersphinx. @@ -300,15 +318,15 @@ features of intersphinx. More topics to be covered ------------------------- -- :doc:`Other extensions <extensions>`: +- :doc:`Other extensions </extensions>`: - * :doc:`ext/math`, - * :doc:`ext/viewcode`, - * :doc:`ext/doctest`, + * :doc:`/ext/math`, + * :doc:`/ext/viewcode`, + * :doc:`/ext/doctest`, * ... - Static files -- :doc:`Selecting a theme <theming>` -- :doc:`setuptools` +- :doc:`Selecting a theme </theming>` +- :doc:`/setuptools` - :ref:`Templating <templating>` - Using extensions - :ref:`Writing extensions <dev-extensions>` @@ -320,6 +338,6 @@ More topics to be covered another directory, the :term:`configuration directory`. Refer to the :program:`sphinx-build man page <sphinx-build>` for more information. -.. |more| image:: more.png +.. |more| image:: /_static/more.png :align: middle :alt: more info @@ -2,6 +2,7 @@ import os import sys from distutils import log +from io import StringIO from setuptools import find_packages, setup @@ -63,6 +64,20 @@ extras_require = { cmdclass = {} + +class Tee(object): + def __init__(self, stream): + self.stream = stream + self.buffer = StringIO() + + def write(self, s): + self.stream.write(s) + self.buffer.write(s) + + def flush(self): + self.stream.flush() + + try: from babel.messages.pofile import read_po from babel.messages.frontend import compile_catalog @@ -80,7 +95,13 @@ else: """ def run(self): - compile_catalog.run(self) + try: + sys.stderr = Tee(sys.stderr) + compile_catalog.run(self) + finally: + if sys.stderr.buffer.getvalue(): + print("Compiling failed.") + sys.exit(1) if isinstance(self.domain, list): for domain in self.domain: diff --git a/sphinx/__init__.py b/sphinx/__init__.py index b819b1d5f..8ffeb4d93 100644 --- a/sphinx/__init__.py +++ b/sphinx/__init__.py @@ -22,6 +22,12 @@ from os import path from .deprecation import RemovedInNextVersionWarning from .deprecation import RemovedInSphinx20Warning +if False: + # For type annotation + # note: Don't use typing.TYPE_CHECK here (for py27 and py34). + from typing import Any # NOQA + + # by default, all DeprecationWarning under sphinx package will be emit. # Users can avoid this by using environment variable: PYTHONWARNINGS= if 'PYTHONWARNINGS' not in os.environ: @@ -31,13 +37,18 @@ if 'PYTHONWARNINGS' not in os.environ: warnings.filterwarnings('ignore', "'U' mode is deprecated", DeprecationWarning, module='docutils.io') -__version__ = '1.7.2+' -__released__ = '1.7.2' # used when Sphinx builds its own docs +__version__ = '1.8.0+' +__released__ = '1.8.0' # used when Sphinx builds its own docs -# version info for better programmatic use -# possible values for 3rd element: 'alpha', 'beta', 'rc', 'final' -# 'final' has 0 as the last element -version_info = (1, 7, 2, 'beta', 0) +#: Version info for better programmatic use. +#: +#: A tuple of five elements; for Sphinx version 1.2.1 beta 3 this would be +#: ``(1, 2, 1, 'beta', 3)``. The fourth element can be one of: ``alpha``, +#: ``beta``, ``rc``, ``final``. ``final`` always has 0 as the last element. +#: +#: .. versionadded:: 1.2 +#: Before version 1.2, check the string ``sphinx.__version__``. +version_info = (1, 8, 0, 'beta', 0) package_dir = path.abspath(path.dirname(__file__)) @@ -61,6 +72,7 @@ if __version__.endswith('+'): def main(*args, **kwargs): + # type: (Any, Any) -> int from .cmd import build warnings.warn( '`sphinx.main()` has moved to `sphinx.cmd.build.main()`.', diff --git a/sphinx/__main__.py b/sphinx/__main__.py index 47a183d08..02bc806e7 100644 --- a/sphinx/__main__.py +++ b/sphinx/__main__.py @@ -13,4 +13,4 @@ import sys from sphinx.cmd.build import main -sys.exit(main(sys.argv[1:])) +sys.exit(main(sys.argv[1:])) # type: ignore diff --git a/sphinx/apidoc.py b/sphinx/apidoc.py index be5e5c5ab..f6616e497 100644 --- a/sphinx/apidoc.py +++ b/sphinx/apidoc.py @@ -10,12 +10,18 @@ """ import warnings +from typing import TYPE_CHECKING from sphinx.deprecation import RemovedInSphinx20Warning from sphinx.ext.apidoc import main as _main +if TYPE_CHECKING: + from typing import Any # NOQA + from sphinx.application import Sphinx # NOQA + def main(*args, **kwargs): + # type: (Any, Any) -> None warnings.warn( '`sphinx.apidoc.main()` has moved to `sphinx.ext.apidoc.main()`.', RemovedInSphinx20Warning, diff --git a/sphinx/application.py b/sphinx/application.py index 8b8d02952..8c726b091 100644 --- a/sphinx/application.py +++ b/sphinx/application.py @@ -3,7 +3,7 @@ sphinx.application ~~~~~~~~~~~~~~~~~~ - Sphinx application object. + Sphinx application class and extensibility interface. Gracefully adapted from the TextPress system by Armin. @@ -17,35 +17,39 @@ import posixpath import sys import warnings from collections import deque +from inspect import isclass from os import path -from docutils import nodes -from docutils.parsers.rst import directives, roles -from six import iteritems, itervalues +from docutils.parsers.rst import Directive, directives, roles +from six import itervalues from six.moves import cStringIO import sphinx from sphinx import package_dir, locale from sphinx.config import Config -from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx.deprecation import RemovedInSphinx20Warning, RemovedInSphinx30Warning from sphinx.environment import BuildEnvironment -from sphinx.errors import ConfigError, ExtensionError, VersionRequirementError +from sphinx.errors import ( + ApplicationError, ConfigError, ExtensionError, VersionRequirementError +) from sphinx.events import EventManager -from sphinx.extension import verify_required_extensions from sphinx.locale import __ from sphinx.registry import SphinxComponentRegistry +from sphinx.util import docutils from sphinx.util import import_object from sphinx.util import logging from sphinx.util import pycompat # noqa: F401 +from sphinx.util.build_phase import BuildPhase from sphinx.util.console import bold # type: ignore -from sphinx.util.docutils import is_html5_writer_available, directive_helper +from sphinx.util.docutils import directive_helper from sphinx.util.i18n import find_catalog_source_files -from sphinx.util.osutil import ENOENT, ensuredir +from sphinx.util.osutil import abspath, ensuredir from sphinx.util.tags import Tags if False: # For type annotation from typing import Any, Callable, Dict, IO, Iterable, Iterator, List, Tuple, Type, Union # NOQA + from docutils import nodes # NOQA from docutils.parsers import Parser # NOQA from docutils.transforms import Transform # NOQA from sphinx.builders import Builder # NOQA @@ -54,7 +58,7 @@ if False: from sphinx.extension import Extension # NOQA from sphinx.roles import XRefRole # NOQA from sphinx.theming import Theme # NOQA - from sphinx.util.typing import RoleFunction # NOQA + from sphinx.util.typing import RoleFunction, TitleGetter # NOQA builtin_extensions = ( 'sphinx.builders.applehelp', @@ -73,6 +77,7 @@ builtin_extensions = ( 'sphinx.builders.text', 'sphinx.builders.websupport', 'sphinx.builders.xml', + 'sphinx.config', 'sphinx.domains.c', 'sphinx.domains.cpp', 'sphinx.domains.javascript', @@ -83,8 +88,10 @@ builtin_extensions = ( 'sphinx.directives.code', 'sphinx.directives.other', 'sphinx.directives.patches', + 'sphinx.extension', 'sphinx.io', 'sphinx.parsers', + 'sphinx.registry', 'sphinx.roles', 'sphinx.transforms.post_transforms', 'sphinx.transforms.post_transforms.images', @@ -108,25 +115,46 @@ logger = logging.getLogger(__name__) class Sphinx(object): + """The main application class and extensibility interface. + + :ivar srcdir: Directory containing source. + :ivar confdir: Directory containing ``conf.py``. + :ivar doctreedir: Directory for storing pickled doctrees. + :ivar outdir: Directory for storing build documents. + """ def __init__(self, srcdir, confdir, outdir, doctreedir, buildername, confoverrides=None, status=sys.stdout, warning=sys.stderr, freshenv=False, warningiserror=False, tags=None, verbosity=0, parallel=0): # type: (unicode, unicode, unicode, unicode, unicode, Dict, IO, IO, bool, bool, List[unicode], int, int) -> None # NOQA + self.phase = BuildPhase.INITIALIZATION self.verbosity = verbosity self.extensions = {} # type: Dict[unicode, Extension] self._setting_up_extension = ['?'] # type: List[unicode] self.builder = None # type: Builder self.env = None # type: BuildEnvironment self.registry = SphinxComponentRegistry() - self.enumerable_nodes = {} # type: Dict[nodes.Node, Tuple[unicode, Callable]] # NOQA self.html_themes = {} # type: Dict[unicode, unicode] - self.srcdir = srcdir # type: unicode + # validate provided directories + self.srcdir = abspath(srcdir) # type: unicode + self.outdir = abspath(outdir) # type: unicode + self.doctreedir = abspath(doctreedir) # type: unicode self.confdir = confdir - self.outdir = outdir - self.doctreedir = doctreedir + if self.confdir: # confdir is optional + self.confdir = abspath(self.confdir) + if not path.isfile(path.join(self.confdir, 'conf.py')): + raise ApplicationError(__("config directory doesn't contain a " + "conf.py file (%s)") % confdir) + + if not path.isdir(self.srcdir): + raise ApplicationError(__('Cannot find source directory (%s)') % + self.srcdir) + + if self.srcdir == self.outdir: + raise ApplicationError(__('Source directory and destination ' + 'directory cannot be identical')) self.parallel = parallel @@ -152,7 +180,7 @@ class Sphinx(object): self.messagelog = deque(maxlen=10) # type: deque # say hello to the world - logger.info(bold('Running Sphinx v%s' % sphinx.__display_version__)) + logger.info(bold(__('Running Sphinx v%s') % sphinx.__display_version__)) # status code for command-line application self.statuscode = 0 @@ -194,14 +222,13 @@ class Sphinx(object): self.preload_builder(buildername) if not path.isdir(outdir): - logger.info('making output directory...') + logger.info(__('making output directory...')) ensuredir(outdir) # the config file itself can be an extension if self.config.setup: self._setting_up_extension = ['conf.py'] - # py31 doesn't have 'callable' function for below check - if hasattr(self.config.setup, '__call__'): + if callable(self.config.setup): self.config.setup(self) else: raise ConfigError( @@ -212,9 +239,7 @@ class Sphinx(object): # now that we know all config values, collect them from conf.py self.config.init_values() - - # check extension versions if requested - verify_required_extensions(self, self.config.needs_extensions) + self.emit('config-inited', self.config) # check primary_domain if requested primary_domain = self.config.primary_domain @@ -225,14 +250,10 @@ class Sphinx(object): self.builder = self.create_builder(buildername) # check all configuration values for permissible types self.config.check_types() - # set up source_parsers - self._init_source_parsers() # set up the build environment self._init_env(freshenv) # set up the builder self._init_builder() - # set up the enumerable nodes - self._init_enumerable_nodes() def _init_i18n(self): # type: () -> None @@ -240,7 +261,7 @@ class Sphinx(object): the configuration. """ if self.config.language is not None: - logger.info(bold('loading translations [%s]... ' % self.config.language), + logger.info(bold(__('loading translations [%s]... ') % self.config.language), nonl=True) user_locale_dirs = [ path.join(self.srcdir, x) for x in self.config.locale_dirs] @@ -252,25 +273,18 @@ class Sphinx(object): locale_dirs = [None, path.join(package_dir, 'locale')] + user_locale_dirs # type: ignore # NOQA else: locale_dirs = [] - self.translator, has_translation = locale.init(locale_dirs, self.config.language) + self.translator, has_translation = locale.init(locale_dirs, self.config.language) # type: ignore # NOQA if self.config.language is not None: if has_translation or self.config.language == 'en': # "en" never needs to be translated logger.info(__('done')) else: - logger.info('not available for built-in messages') - - def _init_source_parsers(self): - # type: () -> None - for suffix, parser in iteritems(self.config.source_parsers): - self.add_source_parser(suffix, parser) - for suffix, parser in iteritems(self.registry.get_source_parsers()): - if suffix not in self.config.source_suffix and suffix != '*': - self.config.source_suffix.append(suffix) + logger.info(__('not available for built-in messages')) def _init_env(self, freshenv): # type: (bool) -> None - if freshenv: + filename = path.join(self.doctreedir, ENV_PICKLE_FILENAME) + if freshenv or not os.path.exists(filename): self.env = BuildEnvironment(self) self.env.find_files(self.config, self.builder) for domain in self.registry.create_domains(self.env): @@ -278,18 +292,17 @@ class Sphinx(object): else: try: logger.info(bold(__('loading pickled environment... ')), nonl=True) - filename = path.join(self.doctreedir, ENV_PICKLE_FILENAME) self.env = BuildEnvironment.frompickle(filename, self) + needed, reason = self.env.need_refresh(self) + if needed: + raise IOError(reason) self.env.domains = {} for domain in self.registry.create_domains(self.env): # this can raise if the data version doesn't fit self.env.domains[domain.name] = domain logger.info(__('done')) except Exception as err: - if isinstance(err, IOError) and err.errno == ENOENT: - logger.info(__('not yet created')) - else: - logger.info(__('failed: %s'), err) + logger.info(__('failed: %s'), err) self._init_env(freshenv=True) def preload_builder(self, name): @@ -310,15 +323,11 @@ class Sphinx(object): self.builder.init() self.emit('builder-inited') - def _init_enumerable_nodes(self): - # type: () -> None - for node, settings in iteritems(self.enumerable_nodes): - self.env.get_domain('std').enumerable_nodes[node] = settings # type: ignore - # ---- main "build" method ------------------------------------------------- def build(self, force_all=False, filenames=None): # type: (bool, List[unicode]) -> None + self.phase = BuildPhase.READING try: if force_all: self.builder.compile_all_catalogs() @@ -334,7 +343,7 @@ class Sphinx(object): __('succeeded') or __('finished with problems')) if self._warncount: logger.info(bold(__('build %s, %s warning.', - 'build %s, %s warnings.', self._warncount) % + 'build %s, %s warnings.', self._warncount) % (status, self._warncount))) else: logger.info(bold(__('build %s.') % status)) @@ -361,10 +370,15 @@ class Sphinx(object): # type: (unicode, unicode, unicode, unicode) -> None """Emit a warning. - If *location* is given, it should either be a tuple of (docname, lineno) - or a string describing the location of the warning as well as possible. + If *location* is given, it should either be a tuple of (*docname*, + *lineno*) or a string describing the location of the warning as well as + possible. + + *type* and *subtype* are used to suppress warnings with + :confval:`suppress_warnings`. - *type* and *subtype* are used to suppress warnings with :confval:`suppress_warnings`. + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. """ warnings.warn('app.warning() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) @@ -376,6 +390,9 @@ class Sphinx(object): If *nonl* is true, don't emit a newline at the end (which implies that more info output will follow soon.) + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. """ warnings.warn('app.info() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) @@ -383,21 +400,33 @@ class Sphinx(object): def verbose(self, message, *args, **kwargs): # type: (unicode, Any, Any) -> None - """Emit a verbose informational message.""" + """Emit a verbose informational message. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. + """ warnings.warn('app.verbose() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) logger.verbose(message, *args, **kwargs) def debug(self, message, *args, **kwargs): # type: (unicode, Any, Any) -> None - """Emit a debug-level informational message.""" + """Emit a debug-level informational message. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. + """ warnings.warn('app.debug() is now deprecated. Use sphinx.util.logging instead.', RemovedInSphinx20Warning) logger.debug(message, *args, **kwargs) def debug2(self, message, *args, **kwargs): # type: (unicode, Any, Any) -> None - """Emit a lowlevel debug-level informational message.""" + """Emit a lowlevel debug-level informational message. + + .. deprecated:: 1.6 + Use :mod:`sphinx.util.logging` instead. + """ warnings.warn('app.debug2() is now deprecated. Use debug() instead.', RemovedInSphinx20Warning) logger.debug(message, *args, **kwargs) @@ -406,35 +435,68 @@ class Sphinx(object): def setup_extension(self, extname): # type: (unicode) -> None - """Import and setup a Sphinx extension module. No-op if called twice.""" + """Import and setup a Sphinx extension module. + + Load the extension given by the module *name*. Use this if your + extension needs the features provided by another extension. No-op if + called twice. + """ logger.debug('[app] setting up extension: %r', extname) self.registry.load_extension(self, extname) def require_sphinx(self, version): # type: (unicode) -> None - # check the Sphinx version if requested + """Check the Sphinx version if requested. + + Compare *version* (which must be a ``major.minor`` version string, e.g. + ``'1.1'``) with the version of the running Sphinx, and abort the build + when it is too old. + + .. versionadded:: 1.0 + """ if version > sphinx.__display_version__[:3]: raise VersionRequirementError(version) def import_object(self, objname, source=None): # type: (str, unicode) -> Any - """Import an object from a 'module.name' string.""" + """Import an object from a ``module.name`` string. + + .. deprecated:: 1.8 + Use ``sphinx.util.import_object()`` instead. + """ + warnings.warn('app.import_object() is deprecated. ' + 'Use sphinx.util.add_object_type() instead.', + RemovedInSphinx30Warning) return import_object(objname, source=None) # event interface def connect(self, event, callback): # type: (unicode, Callable) -> int + """Register *callback* to be called when *event* is emitted. + + For details on available core events and the arguments of callback + functions, please see :ref:`events`. + + The method returns a "listener ID" that can be used as an argument to + :meth:`disconnect`. + """ listener_id = self.events.connect(event, callback) logger.debug('[app] connecting event %r: %r [id=%s]', event, callback, listener_id) return listener_id def disconnect(self, listener_id): # type: (int) -> None + """Unregister callback by *listener_id*.""" logger.debug('[app] disconnecting event: [id=%s]', listener_id) self.events.disconnect(listener_id) def emit(self, event, *args): # type: (unicode, Any) -> List + """Emit *event* and pass *arguments* to the callback functions. + + Return the return values of all callbacks as a list. Do not emit core + Sphinx events in extensions! + """ try: logger.debug('[app] emitting event: %r%s', event, repr(args)[:100]) except Exception: @@ -445,16 +507,53 @@ class Sphinx(object): def emit_firstresult(self, event, *args): # type: (unicode, Any) -> Any + """Emit *event* and pass *arguments* to the callback functions. + + Return the result of the first callback that doesn't return ``None``. + + .. versionadded:: 0.5 + """ return self.events.emit_firstresult(event, self, *args) # registering addon parts def add_builder(self, builder): # type: (Type[Builder]) -> None + """Register a new builder. + + *builder* must be a class that inherits from + :class:`~sphinx.builders.Builder`. + """ self.registry.add_builder(builder) + # TODO(stephenfin): Describe 'types' parameter def add_config_value(self, name, default, rebuild, types=()): # type: (unicode, Any, Union[bool, unicode], Any) -> None + """Register a configuration value. + + This is necessary for Sphinx to recognize new values and set default + values accordingly. The *name* should be prefixed with the extension + name, to avoid clashes. The *default* value can be any Python object. + The string value *rebuild* must be one of those values: + + * ``'env'`` if a change in the setting only takes effect when a + document is parsed -- this means that the whole environment must be + rebuilt. + * ``'html'`` if a change in the setting needs a full rebuild of HTML + documents. + * ``''`` if a change in the setting will not need any special rebuild. + + .. versionchanged:: 0.6 + Changed *rebuild* from a simple boolean (equivalent to ``''`` or + ``'env'``) to a string. However, booleans are still accepted and + converted internally. + + .. versionchanged:: 0.4 + If the *default* value is a callable, it will be called with the + config object as its argument in order to get the default value. + This can be used to implement config values whose default depends on + other values. + """ logger.debug('[app] adding config value: %r', (name, default, rebuild) + ((types,) if types else ())) # type: ignore if name in self.config: @@ -465,64 +564,141 @@ class Sphinx(object): def add_event(self, name): # type: (unicode) -> None + """Register an event called *name*. + + This is needed to be able to emit it. + """ logger.debug('[app] adding event: %r', name) self.events.add(name) def set_translator(self, name, translator_class): # type: (unicode, Type[nodes.NodeVisitor]) -> None + """Register or override a Docutils translator class. + + This is used to register a custom output translator or to replace a + builtin translator. This allows extensions to use custom translator + and define custom nodes for the translator (see :meth:`add_node`). + + .. versionadded:: 1.3 + """ self.registry.add_translator(name, translator_class) def add_node(self, node, **kwds): # type: (nodes.Node, Any) -> None + """Register a Docutils node class. + + This is necessary for Docutils internals. It may also be used in the + future to validate nodes in the parsed documents. + + Node visitor functions for the Sphinx HTML, LaTeX, text and manpage + writers can be given as keyword arguments: the keyword should be one or + more of ``'html'``, ``'latex'``, ``'text'``, ``'man'``, ``'texinfo'`` + or any other supported translators, the value a 2-tuple of ``(visit, + depart)`` methods. ``depart`` can be ``None`` if the ``visit`` + function raises :exc:`docutils.nodes.SkipNode`. Example: + + .. code-block:: python + + class math(docutils.nodes.Element): pass + + def visit_math_html(self, node): + self.body.append(self.starttag(node, 'math')) + def depart_math_html(self, node): + self.body.append('</math>') + + app.add_node(math, html=(visit_math_html, depart_math_html)) + + Obviously, translators for which you don't specify visitor methods will + choke on the node when encountered in a document to translate. + + .. versionchanged:: 0.5 + Added the support for keyword arguments giving visit functions. + """ logger.debug('[app] adding node: %r', (node, kwds)) - if not kwds.pop('override', False) and \ - hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__): + if not kwds.pop('override', False) and docutils.is_node_registered(node): logger.warning(__('while setting up extension %s: node class %r is ' 'already registered, its visitors will be overridden'), self._setting_up_extension, node.__name__, type='app', subtype='add_node') - nodes._add_node_class_names([node.__name__]) - for key, val in iteritems(kwds): - try: - visit, depart = val - except ValueError: - raise ExtensionError(__('Value for key %r must be a ' - '(visit, depart) function tuple') % key) - translator = self.registry.translators.get(key) - translators = [] - if translator is not None: - translators.append(translator) - elif key == 'html': - from sphinx.writers.html import HTMLTranslator - translators.append(HTMLTranslator) - if is_html5_writer_available(): - from sphinx.writers.html5 import HTML5Translator - translators.append(HTML5Translator) - elif key == 'latex': - from sphinx.writers.latex import LaTeXTranslator - translators.append(LaTeXTranslator) - elif key == 'text': - from sphinx.writers.text import TextTranslator - translators.append(TextTranslator) - elif key == 'man': - from sphinx.writers.manpage import ManualPageTranslator - translators.append(ManualPageTranslator) - elif key == 'texinfo': - from sphinx.writers.texinfo import TexinfoTranslator - translators.append(TexinfoTranslator) - - for translator in translators: - setattr(translator, 'visit_' + node.__name__, visit) - if depart: - setattr(translator, 'depart_' + node.__name__, depart) + docutils.register_node(node) + self.registry.add_translation_handlers(node, **kwds) def add_enumerable_node(self, node, figtype, title_getter=None, **kwds): - # type: (nodes.Node, unicode, Callable, Any) -> None - self.enumerable_nodes[node] = (figtype, title_getter) + # type: (nodes.Node, unicode, TitleGetter, Any) -> None + """Register a Docutils node class as a numfig target. + + Sphinx numbers the node automatically. And then the users can refer it + using :rst:role:`numref`. + + *figtype* is a type of enumerable nodes. Each figtypes have individual + numbering sequences. As a system figtypes, ``figure``, ``table`` and + ``code-block`` are defined. It is able to add custom nodes to these + default figtypes. It is also able to define new custom figtype if new + figtype is given. + + *title_getter* is a getter function to obtain the title of node. It + takes an instance of the enumerable node, and it must return its title + as string. The title is used to the default title of references for + :rst:role:`ref`. By default, Sphinx searches + ``docutils.nodes.caption`` or ``docutils.nodes.title`` from the node as + a title. + + Other keyword arguments are used for node visitor functions. See the + :meth:`Sphinx.add_node` for details. + + .. versionadded:: 1.4 + """ + self.registry.add_enumerable_node(node, figtype, title_getter) self.add_node(node, **kwds) + @property + def enumerable_nodes(self): + # type: () -> Dict[nodes.Node, Tuple[unicode, TitleGetter]] + warnings.warn('app.enumerable_nodes() is deprecated. ' + 'Use app.get_domain("std").enumerable_nodes instead.', + RemovedInSphinx30Warning) + return self.registry.enumerable_nodes + def add_directive(self, name, obj, content=None, arguments=None, **options): # type: (unicode, Any, bool, Tuple[int, int, bool], Any) -> None + """Register a Docutils directive. + + *name* must be the prospective directive name. There are two possible + ways to write a directive: + + - In the docutils 0.4 style, *obj* is the directive function. + *content*, *arguments* and *options* are set as attributes on the + function and determine whether the directive has content, arguments + and options, respectively. **This style is deprecated.** + + - In the docutils 0.5 style, *directiveclass* is the directive class. + It must already have attributes named *has_content*, + *required_arguments*, *optional_arguments*, + *final_argument_whitespace* and *option_spec* that correspond to the + options for the function way. See `the Docutils docs + <http://docutils.sourceforge.net/docs/howto/rst-directives.html>`_ + for details. + + The directive class must inherit from the class + ``docutils.parsers.rst.Directive``. + + For example, the (already existing) :rst:dir:`literalinclude` directive + would be added like this: + + .. code-block:: python + + from docutils.parsers.rst import directives + add_directive('literalinclude', literalinclude_directive, + content = 0, arguments = (1, 0, 0), + linenos = directives.flag, + language = directives.unchanged, + encoding = directives.encoding) + + .. versionchanged:: 0.6 + Docutils 0.5-style directive classes are now supported. + .. deprecated:: 1.8 + Docutils 0.4-style (function based) directives support is deprecated. + """ logger.debug('[app] adding directive: %r', (name, obj, content, arguments, options)) if name in directives._directives: @@ -530,11 +706,22 @@ class Sphinx(object): 'already registered, it will be overridden'), self._setting_up_extension[-1], name, type='app', subtype='add_directive') - directive = directive_helper(obj, content, arguments, **options) - directives.register_directive(name, directive) + + if not isclass(obj) or not issubclass(obj, Directive): + directive = directive_helper(obj, content, arguments, **options) + directives.register_directive(name, directive) + else: + directives.register_directive(name, obj) def add_role(self, name, role): # type: (unicode, Any) -> None + """Register a Docutils role. + + *name* must be the role name that occurs in the source, *role* the role + function. Refer to the `Docutils documentation + <http://docutils.sourceforge.net/docs/howto/rst-roles.html>`_ for + more information. + """ logger.debug('[app] adding role: %r', (name, role)) if name in roles._roles: logger.warning(__('while setting up extension %s: role %r is ' @@ -545,8 +732,15 @@ class Sphinx(object): def add_generic_role(self, name, nodeclass): # type: (unicode, Any) -> None - # don't use roles.register_generic_role because it uses - # register_canonical_role + """Register a generic Docutils role. + + Register a Docutils role that does nothing but wrap its contents in the + node given by *nodeclass*. + + .. versionadded:: 0.6 + """ + # Don't use ``roles.register_generic_role`` because it uses + # ``register_canonical_role``. logger.debug('[app] adding generic role: %r', (name, nodeclass)) if name in roles._roles: logger.warning(__('while setting up extension %s: role %r is ' @@ -558,30 +752,121 @@ class Sphinx(object): def add_domain(self, domain): # type: (Type[Domain]) -> None + """Register a domain. + + Make the given *domain* (which must be a class; more precisely, a + subclass of :class:`~sphinx.domains.Domain`) known to Sphinx. + + .. versionadded:: 1.0 + """ self.registry.add_domain(domain) def override_domain(self, domain): # type: (Type[Domain]) -> None + """Override a registered domain. + + Make the given *domain* class known to Sphinx, assuming that there is + already a domain with its ``.name``. The new domain must be a subclass + of the existing one. + + .. versionadded:: 1.0 + """ self.registry.override_domain(domain) def add_directive_to_domain(self, domain, name, obj, has_content=None, argument_spec=None, **option_spec): # type: (unicode, unicode, Any, bool, Any, Any) -> None + """Register a Docutils directive in a domain. + + Like :meth:`add_directive`, but the directive is added to the domain + named *domain*. + + .. versionadded:: 1.0 + """ self.registry.add_directive_to_domain(domain, name, obj, has_content, argument_spec, **option_spec) def add_role_to_domain(self, domain, name, role): # type: (unicode, unicode, Union[RoleFunction, XRefRole]) -> None + """Register a Docutils role in a domain. + + Like :meth:`add_role`, but the role is added to the domain named + *domain*. + + .. versionadded:: 1.0 + """ self.registry.add_role_to_domain(domain, name, role) def add_index_to_domain(self, domain, index): # type: (unicode, Type[Index]) -> None + """Register a custom index for a domain. + + Add a custom *index* class to the domain named *domain*. *index* must + be a subclass of :class:`~sphinx.domains.Index`. + + .. versionadded:: 1.0 + """ self.registry.add_index_to_domain(domain, index) def add_object_type(self, directivename, rolename, indextemplate='', parse_node=None, ref_nodeclass=None, objname='', doc_field_types=[]): # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List) -> None + """Register a new object type. + + This method is a very convenient way to add a new :term:`object` type + that can be cross-referenced. It will do this: + + - Create a new directive (called *directivename*) for documenting an + object. It will automatically add index entries if *indextemplate* + is nonempty; if given, it must contain exactly one instance of + ``%s``. See the example below for how the template will be + interpreted. * Create a new role (called *rolename*) to + cross-reference to these object descriptions. + - If you provide *parse_node*, it must be a function that takes a + string and a docutils node, and it must populate the node with + children parsed from the string. It must then return the name of the + item to be used in cross-referencing and index entries. See the + :file:`conf.py` file in the source for this documentation for an + example. + - The *objname* (if not given, will default to *directivename*) names + the type of object. It is used when listing objects, e.g. in search + results. + + For example, if you have this call in a custom Sphinx extension:: + + app.add_object_type('directive', 'dir', 'pair: %s; directive') + + you can use this markup in your documents:: + + .. rst:directive:: function + + Document a function. + + <...> + + See also the :rst:dir:`function` directive. + + For the directive, an index entry will be generated as if you had prepended :: + + .. index:: pair: function; directive + + The reference node will be of class ``literal`` (so it will be rendered + in a proportional font, as appropriate for code) unless you give the + *ref_nodeclass* argument, which must be a docutils node class. Most + useful are ``docutils.nodes.emphasis`` or ``docutils.nodes.strong`` -- + you can also use ``docutils.nodes.generated`` if you want no further + text decoration. If the text should be treated as literal (e.g. no + smart quote replacement), but not have typewriter styling, use + ``sphinx.addnodes.literal_emphasis`` or + ``sphinx.addnodes.literal_strong``. + + For the role content, you have the same syntactical possibilities as + for standard Sphinx roles (see :ref:`xref-syntax`). + + This method is also available under the deprecated alias + :meth:`add_description_unit`. + """ self.registry.add_object_type(directivename, rolename, indextemplate, parse_node, ref_nodeclass, objname, doc_field_types) @@ -589,6 +874,11 @@ class Sphinx(object): parse_node=None, ref_nodeclass=None, objname='', doc_field_types=[]): # type: (unicode, unicode, unicode, Callable, nodes.Node, unicode, List) -> None + """Deprecated alias for :meth:`add_object_type`. + + .. deprecated:: 1.6 + Use :meth:`add_object_type` instead. + """ warnings.warn('app.add_description_unit() is now deprecated. ' 'Use app.add_object_type() instead.', RemovedInSphinx20Warning) @@ -598,19 +888,67 @@ class Sphinx(object): def add_crossref_type(self, directivename, rolename, indextemplate='', ref_nodeclass=None, objname=''): # type: (unicode, unicode, unicode, nodes.Node, unicode) -> None + """Register a new crossref object type. + + This method is very similar to :meth:`add_object_type` except that the + directive it generates must be empty, and will produce no output. + + That means that you can add semantic targets to your sources, and refer + to them using custom roles instead of generic ones (like + :rst:role:`ref`). Example call:: + + app.add_crossref_type('topic', 'topic', 'single: %s', + docutils.nodes.emphasis) + + Example usage:: + + .. topic:: application API + + The application API + ------------------- + + Some random text here. + + See also :topic:`this section <application API>`. + + (Of course, the element following the ``topic`` directive needn't be a + section.) + """ self.registry.add_crossref_type(directivename, rolename, indextemplate, ref_nodeclass, objname) def add_transform(self, transform): # type: (Type[Transform]) -> None + """Register a Docutils transform to be applied after parsing. + + Add the standard docutils :class:`Transform` subclass *transform* to + the list of transforms that are applied after Sphinx parses a reST + document. + """ self.registry.add_transform(transform) def add_post_transform(self, transform): # type: (Type[Transform]) -> None + """Register a Docutils transform to be applied before writing. + + Add the standard docutils :class:`Transform` subclass *transform* to + the list of transforms that are applied before Sphinx writes a + document. + """ self.registry.add_post_transform(transform) def add_javascript(self, filename): # type: (unicode) -> None + """Register a JavaScript file to include in the HTML output. + + Add *filename* to the list of JavaScript files that the default HTML + template will include. The filename must be relative to the HTML + static path, see :confval:`the docs for the config value + <html_static_path>`. A full URI with scheme, like + ``http://example.org/foo.js``, is also supported. + + .. versionadded:: 0.5 + """ logger.debug('[app] adding javascript: %r', filename) from sphinx.builders.html import StandaloneHTMLBuilder if '://' in filename: @@ -621,6 +959,21 @@ class Sphinx(object): def add_stylesheet(self, filename, alternate=False, title=None): # type: (unicode, bool, unicode) -> None + """Register a stylesheet to include in the HTML output. + + Add *filename* to the list of CSS files that the default HTML template + will include. Like for :meth:`add_javascript`, the filename must be + relative to the HTML static path, or a full URI with scheme. + + .. versionadded:: 1.0 + + .. versionchanged:: 1.6 + Optional ``alternate`` and/or ``title`` attributes can be supplied + with the *alternate* (of boolean type) and *title* (a string) + arguments. The default is no title and *alternate* = ``False``. For + more information, refer to the `documentation + <https://mdn.io/Web/CSS/Alternative_style_sheets>`__. + """ logger.debug('[app] adding stylesheet: %r', filename) from sphinx.builders.html import StandaloneHTMLBuilder, Stylesheet if '://' not in filename: @@ -634,12 +987,32 @@ class Sphinx(object): def add_latex_package(self, packagename, options=None): # type: (unicode, unicode) -> None - logger.debug('[app] adding latex package: %r', packagename) - if hasattr(self.builder, 'usepackages'): # only for LaTeX builder - self.builder.usepackages.append((packagename, options)) # type: ignore + r"""Register a package to include in the LaTeX source code. + + Add *packagename* to the list of packages that LaTeX source code will + include. If you provide *options*, it will be taken to `\usepackage` + declaration. + + .. code-block:: python + + app.add_latex_package('mypackage') + # => \usepackage{mypackage} + app.add_latex_package('mypackage', 'foo,bar') + # => \usepackage[foo,bar]{mypackage} + + .. versionadded:: 1.3 + """ + self.registry.add_latex_package(packagename, options) def add_lexer(self, alias, lexer): # type: (unicode, Any) -> None + """Register a new lexer for source code. + + Use *lexer*, which must be an instance of a Pygments lexer class, to + highlight code blocks with the given language *alias*. + + .. versionadded:: 0.6 + """ logger.debug('[app] adding lexer: %r', (alias, lexer)) from sphinx.highlighting import lexers if lexers is None: @@ -648,6 +1021,18 @@ class Sphinx(object): def add_autodocumenter(self, cls): # type: (Any) -> None + """Register a new documenter class for the autodoc extension. + + Add *cls* as a new documenter class for the :mod:`sphinx.ext.autodoc` + extension. It must be a subclass of + :class:`sphinx.ext.autodoc.Documenter`. This allows to auto-document + new types of objects. See the source of the autodoc module for + examples on how to subclass :class:`Documenter`. + + .. todo:: Add real docs for Documenter and subclassing + + .. versionadded:: 0.6 + """ logger.debug('[app] adding autodocumenter: %r', cls) from sphinx.ext.autodoc.directive import AutodocDirective self.registry.add_documenter(cls.objtype, cls) @@ -655,30 +1040,92 @@ class Sphinx(object): def add_autodoc_attrgetter(self, typ, getter): # type: (Type, Callable[[Any, unicode, Any], Any]) -> None + """Register a new ``getattr``-like function for the autodoc extension. + + Add *getter*, which must be a function with an interface compatible to + the :func:`getattr` builtin, as the autodoc attribute getter for + objects that are instances of *typ*. All cases where autodoc needs to + get an attribute of a type are then handled by this function instead of + :func:`getattr`. + + .. versionadded:: 0.6 + """ logger.debug('[app] adding autodoc attrgetter: %r', (typ, getter)) self.registry.add_autodoc_attrgetter(typ, getter) def add_search_language(self, cls): # type: (Any) -> None + """Register a new language for the HTML search index. + + Add *cls*, which must be a subclass of + :class:`sphinx.search.SearchLanguage`, as a support language for + building the HTML full-text search index. The class must have a *lang* + attribute that indicates the language it should be used for. See + :confval:`html_search_language`. + + .. versionadded:: 1.1 + """ logger.debug('[app] adding search language: %r', cls) from sphinx.search import languages, SearchLanguage assert issubclass(cls, SearchLanguage) languages[cls.lang] = cls - def add_source_parser(self, suffix, parser): - # type: (unicode, Parser) -> None - self.registry.add_source_parser(suffix, parser) + def add_source_suffix(self, suffix, filetype): + # type: (unicode, unicode) -> None + """Register a suffix of source files. + + Same as :confval:`source_suffix`. The users can override this + using the setting. + """ + self.registry.add_source_suffix(suffix, filetype) + + def add_source_parser(self, *args): + # type: (Any) -> None + """Register a parser class. + + .. versionadded:: 1.4 + .. versionchanged:: 1.8 + *suffix* argument is deprecated. It only accepts *parser* argument. + Use :meth:`add_source_suffix` API to register suffix instead. + """ + self.registry.add_source_parser(*args) def add_env_collector(self, collector): # type: (Type[EnvironmentCollector]) -> None + """Register an environment collector class. + + Refer to :ref:`collector-api`. + + .. versionadded:: 1.6 + """ logger.debug('[app] adding environment collector: %r', collector) collector().enable(self) def add_html_theme(self, name, theme_path): # type: (unicode, unicode) -> None + """Register a HTML Theme. + + The *name* is a name of theme, and *path* is a full path to the theme + (refs: :ref:`distribute-your-theme`). + + .. versionadded:: 1.6 + """ logger.debug('[app] adding HTML theme: %r, %r', name, theme_path) self.html_themes[name] = theme_path + def add_message_catalog(self, catalog, locale_dir): + # type: (unicode, unicode) -> None + """Register a message catalog. + + The *catalog* is a name of catalog, and *locale_dir* is a base path + of message catalog. For more details, see + :func:`sphinx.locale.get_translation()`. + + .. versionadded:: 1.8 + """ + locale.init([locale_dir], self.config.language, catalog) + locale.init_console(locale_dir, catalog) + # ---- other methods ------------------------------------------------- def is_parallel_allowed(self, typ): # type: (unicode) -> bool @@ -705,7 +1152,7 @@ class Sphinx(object): allowed = getattr(ext, attrname, None) if allowed is None: logger.warning(message, ext.name) - logger.warning('doing serial %s', typ) + logger.warning(__('doing serial %s'), typ) return False elif not allowed: return False diff --git a/sphinx/builders/__init__.py b/sphinx/builders/__init__.py index 5c4714e8f..a3c8e776d 100644 --- a/sphinx/builders/__init__.py +++ b/sphinx/builders/__init__.py @@ -15,8 +15,12 @@ from os import path from docutils import nodes from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx.environment import BuildEnvironment from sphinx.environment.adapters.asset import ImageAdapter -from sphinx.util import i18n, logging, status_iterator +from sphinx.errors import SphinxError +from sphinx.locale import __ +from sphinx.util import i18n, import_object, logging, status_iterator +from sphinx.util.build_phase import BuildPhase from sphinx.util.console import bold # type: ignore from sphinx.util.i18n import find_catalog from sphinx.util.osutil import SEP, ensuredir, relative_uri @@ -59,8 +63,8 @@ class Builder(object): #: ``project`` epilog = '' # type: unicode - # default translator class for the builder. This will be overrided by - # ``app.set_translator()``. + #: default translator class for the builder. This can be overrided by + #: :py:meth:`app.set_translator()`. default_translator_class = None # type: nodes.NodeVisitor # doctree versioning method versioning_method = 'none' # type: unicode @@ -73,7 +77,9 @@ class Builder(object): #: The list of MIME types of image formats supported by the builder. #: Image files are searched in the order in which they appear here. supported_image_types = [] # type: List[unicode] + #: The builder supports remote images or not. supported_remote_images = False + #: The builder supports data URIs or not. supported_data_uri_images = False def __init__(self, app): @@ -125,9 +131,7 @@ class Builder(object): This method returns an instance of ``default_translator_class`` by default. Users can replace the translator class with ``app.set_translator()`` API. """ - translator_class = self.app.registry.get_translator_class(self) - assert translator_class, "translator not found for %s" % self.__class__.__name__ - return translator_class(*args) + return self.app.registry.create_translator(self, *args) @property def translator_class(self): @@ -157,8 +161,8 @@ class Builder(object): # type: () -> None """Return the template bridge configured.""" if self.config.template_bridge: - self.templates = self.app.import_object( - self.config.template_bridge, 'template_bridge setting')() + self.templates = import_object(self.config.template_bridge, + 'template_bridge setting')() else: from sphinx.jinja2glue import BuiltinTemplateLoader self.templates = BuiltinTemplateLoader() @@ -211,7 +215,7 @@ class Builder(object): if candidate: break else: - logger.warning('no matching candidate for image URI %r', + logger.warning(__('no matching candidate for image URI %r'), images.get_original_image_uri(node['uri']), location=node) continue @@ -234,8 +238,8 @@ class Builder(object): # type: (CatalogInfo) -> unicode return path.relpath(cat.mo_path, self.env.srcdir).replace(path.sep, SEP) - logger.info(bold('building [mo]: ') + message) - for catalog in status_iterator(catalogs, 'writing output... ', "darkgreen", + logger.info(bold(__('building [mo]: ')) + message) + for catalog in status_iterator(catalogs, __('writing output... '), "darkgreen", len(catalogs), self.app.verbosity, stringify_func=cat2relpath): catalog.write_mo(self.config.language) @@ -248,7 +252,7 @@ class Builder(object): charset=self.config.source_encoding, gettext_compact=self.config.gettext_compact, force_all=True) - message = 'all of %d po files' % len(catalogs) + message = __('all of %d po files') % len(catalogs) self.compile_catalogs(catalogs, message) def compile_specific_catalogs(self, specified_files): @@ -269,7 +273,7 @@ class Builder(object): domains=list(specified_domains), charset=self.config.source_encoding, gettext_compact=self.config.gettext_compact) - message = 'targets for %d po files that are specified' % len(catalogs) + message = __('targets for %d po files that are specified') % len(catalogs) self.compile_catalogs(catalogs, message) def compile_update_catalogs(self): @@ -279,7 +283,7 @@ class Builder(object): self.config.language, charset=self.config.source_encoding, gettext_compact=self.config.gettext_compact) - message = 'targets for %d po files that are out of date' % len(catalogs) + message = __('targets for %d po files that are out of date') % len(catalogs) self.compile_catalogs(catalogs, message) # build methods @@ -287,7 +291,7 @@ class Builder(object): def build_all(self): # type: () -> None """Build all source files.""" - self.build(None, summary='all source files', method='all') + self.build(None, summary=__('all source files'), method='all') def build_specific(self, filenames): # type: (List[unicode]) -> None @@ -301,13 +305,13 @@ class Builder(object): for filename in filenames: filename = path.normpath(path.abspath(filename)) if not filename.startswith(self.srcdir): - logger.warning('file %r given on command line is not under the ' - 'source directory, ignoring', filename) + logger.warning(__('file %r given on command line is not under the ' + 'source directory, ignoring'), filename) continue if not (path.isfile(filename) or any(path.isfile(filename + suffix) for suffix in suffixes)): - logger.warning('file %r given on command line does not exist, ' - 'ignoring', filename) + logger.warning(__('file %r given on command line does not exist, ' + 'ignoring'), filename) continue filename = filename[dirlen:] for suffix in suffixes: @@ -317,8 +321,7 @@ class Builder(object): filename = filename.replace(path.sep, SEP) to_write.append(filename) self.build(to_write, method='specific', - summary='%d source files given on command ' - 'line' % len(to_write)) + summary=__('%d source files given on command line') % len(to_write)) def build_update(self): # type: () -> None @@ -329,8 +332,8 @@ class Builder(object): else: to_build = list(to_build) self.build(to_build, - summary='targets for %d source files that are ' - 'out of date' % len(to_build)) + summary=__('targets for %d source files that are out of date') % + len(to_build)) def build(self, docnames, summary=None, method='update'): # type: (Iterable[unicode], unicode, unicode) -> None @@ -339,38 +342,41 @@ class Builder(object): First updates the environment, and then calls :meth:`write`. """ if summary: - logger.info(bold('building [%s]' % self.name) + ': ' + summary) + logger.info(bold(__('building [%s]') % self.name) + ': ' + summary) # while reading, collect all warnings from docutils with logging.pending_warnings(): - updated_docnames = set(self.env.update(self.config, self.srcdir, self.doctreedir)) + updated_docnames = set(self.read()) doccount = len(updated_docnames) - logger.info(bold('looking for now-outdated files... '), nonl=1) + logger.info(bold(__('looking for now-outdated files... ')), nonl=1) for docname in self.env.check_dependents(self.app, updated_docnames): updated_docnames.add(docname) outdated = len(updated_docnames) - doccount if outdated: - logger.info('%d found', outdated) + logger.info(__('%d found'), outdated) else: - logger.info('none found') + logger.info(__('none found')) if updated_docnames: # save the environment from sphinx.application import ENV_PICKLE_FILENAME - logger.info(bold('pickling environment... '), nonl=True) + logger.info(bold(__('pickling environment... ')), nonl=True) self.env.topickle(path.join(self.doctreedir, ENV_PICKLE_FILENAME)) - logger.info('done') + logger.info(__('done')) # global actions - logger.info(bold('checking consistency... '), nonl=True) + self.app.phase = BuildPhase.CONSISTENCY_CHECK + logger.info(bold(__('checking consistency... ')), nonl=True) self.env.check_consistency() - logger.info('done') + logger.info(__('done')) else: if method == 'update' and not docnames: - logger.info(bold('no targets are out of date.')) + logger.info(bold(__('no targets are out of date.'))) return + self.app.phase = BuildPhase.RESOLVING + # filter "docnames" (list of outdated files) by the updated # found_docs of the environment; this will remove docs that # have since been removed @@ -399,6 +405,106 @@ class Builder(object): # wait for all tasks self.finish_tasks.join() + def read(self): + # type: () -> List[unicode] + """(Re-)read all files new or changed since last update. + + Store all environment docnames in the canonical format (ie using SEP as + a separator in place of os.path.sep). + """ + updated, reason = self.env.update_config(self.config, self.srcdir, self.doctreedir) + + logger.info(bold('updating environment: '), nonl=True) + + self.env.find_files(self.config, self) + added, changed, removed = self.env.get_outdated_files(updated) + + # allow user intervention as well + for docs in self.app.emit('env-get-outdated', self, added, changed, removed): + changed.update(set(docs) & self.env.found_docs) + + # if files were added or removed, all documents with globbed toctrees + # must be reread + if added or removed: + # ... but not those that already were removed + changed.update(self.env.glob_toctrees & self.env.found_docs) + + if changed: + logger.info('[%s] ', reason, nonl=True) + logger.info('%s added, %s changed, %s removed', + len(added), len(changed), len(removed)) + + # clear all files no longer present + for docname in removed: + self.app.emit('env-purge-doc', self.env, docname) + self.env.clear_doc(docname) + + # read all new and changed files + docnames = sorted(added | changed) + # allow changing and reordering the list of docs to read + self.app.emit('env-before-read-docs', self.env, docnames) + + # check if we should do parallel or serial read + if parallel_available and len(docnames) > 5 and self.app.parallel > 1: + par_ok = self.app.is_parallel_allowed('read') + else: + par_ok = False + + if par_ok: + self._read_parallel(docnames, nproc=self.app.parallel) + else: + self._read_serial(docnames) + + if self.config.master_doc not in self.env.all_docs: + raise SphinxError('master file %s not found' % + self.env.doc2path(self.config.master_doc)) + + for retval in self.app.emit('env-updated', self.env): + if retval is not None: + docnames.extend(retval) + + return sorted(docnames) + + def _read_serial(self, docnames): + # type: (List[unicode]) -> None + for docname in status_iterator(docnames, 'reading sources... ', "purple", + len(docnames), self.app.verbosity): + # remove all inventory entries for that file + self.app.emit('env-purge-doc', self.env, docname) + self.env.clear_doc(docname) + self.env.read_doc(docname, self.app) + + def _read_parallel(self, docnames, nproc): + # type: (List[unicode], int) -> None + # clear all outdated docs at once + for docname in docnames: + self.app.emit('env-purge-doc', self.env, docname) + self.env.clear_doc(docname) + + def read_process(docs): + # type: (List[unicode]) -> unicode + self.env.app = self.app + for docname in docs: + self.env.read_doc(docname, self.app) + # allow pickling self to send it back + return BuildEnvironment.dumps(self.env) + + def merge(docs, otherenv): + # type: (List[unicode], unicode) -> None + env = BuildEnvironment.loads(otherenv) + self.env.merge_info_from(docs, env, self.app) + + tasks = ParallelTasks(nproc) + chunks = make_chunks(docnames, nproc) + + for chunk in status_iterator(chunks, 'reading sources... ', "purple", + len(chunks), self.app.verbosity): + tasks.add_task(read_process, chunk, merge) + + # make sure all threads have finished + logger.info(bold('waiting for workers...')) + tasks.join() + def write(self, build_docnames, updated_docnames, method='update'): # type: (Iterable[unicode], Sequence[unicode], unicode) -> None if build_docnames is None or build_docnames == ['__all__']: @@ -409,7 +515,7 @@ class Builder(object): docnames = set(build_docnames) | set(updated_docnames) else: docnames = set(build_docnames) - logger.debug('docnames to write: %s', ', '.join(sorted(docnames))) + logger.debug(__('docnames to write: %s'), ', '.join(sorted(docnames))) # add all toctree-containing files that may have changed for docname in list(docnames): @@ -418,9 +524,9 @@ class Builder(object): docnames.add(tocdocname) docnames.add(self.config.master_doc) - logger.info(bold('preparing documents... '), nonl=True) + logger.info(bold(__('preparing documents... ')), nonl=True) self.prepare_writing(docnames) - logger.info('done') + logger.info(__('done')) if self.parallel_ok: # number of subprocesses is parallel-1 because the main process @@ -433,9 +539,11 @@ class Builder(object): def _write_serial(self, docnames): # type: (Sequence[unicode]) -> None with logging.pending_warnings(): - for docname in status_iterator(docnames, 'writing output... ', "darkgreen", + for docname in status_iterator(docnames, __('writing output... '), "darkgreen", len(docnames), self.app.verbosity): + self.app.phase = BuildPhase.RESOLVING doctree = self.env.get_and_resolve_doctree(docname, self) + self.app.phase = BuildPhase.WRITING self.write_doc_serialized(docname, doctree) self.write_doc(docname, doctree) @@ -443,19 +551,23 @@ class Builder(object): # type: (Sequence[unicode], int) -> None def write_process(docs): # type: (List[Tuple[unicode, nodes.Node]]) -> None + self.app.phase = BuildPhase.WRITING for docname, doctree in docs: self.write_doc(docname, doctree) # warm up caches/compile templates using the first document firstname, docnames = docnames[0], docnames[1:] + self.app.phase = BuildPhase.RESOLVING doctree = self.env.get_and_resolve_doctree(firstname, self) + self.app.phase = BuildPhase.WRITING self.write_doc_serialized(firstname, doctree) self.write_doc(firstname, doctree) tasks = ParallelTasks(nproc) chunks = make_chunks(docnames, nproc) - for chunk in status_iterator(chunks, 'writing output... ', "darkgreen", + self.app.phase = BuildPhase.RESOLVING + for chunk in status_iterator(chunks, __('writing output... '), "darkgreen", len(chunks), self.app.verbosity): arg = [] for i, docname in enumerate(chunk): @@ -465,7 +577,7 @@ class Builder(object): tasks.add_task(write_process, arg) # make sure all threads have finished - logger.info(bold('waiting for workers...')) + logger.info(bold(__('waiting for workers...'))) tasks.join() def prepare_writing(self, docnames): diff --git a/sphinx/builders/_epub_base.py b/sphinx/builders/_epub_base.py index d44e7f2be..a2fa6d576 100644 --- a/sphinx/builders/_epub_base.py +++ b/sphinx/builders/_epub_base.py @@ -20,6 +20,7 @@ from docutils.utils import smartquotes from sphinx import addnodes from sphinx.builders.html import BuildInfo, StandaloneHTMLBuilder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util import status_iterator from sphinx.util.fileutil import copy_asset_file @@ -401,13 +402,13 @@ class EpubBuilder(StandaloneHTMLBuilder): img = Image.open(path.join(self.srcdir, src)) except IOError: if not self.is_vector_graphics(src): - logger.warning('cannot read image file %r: copying it instead', + logger.warning(__('cannot read image file %r: copying it instead'), path.join(self.srcdir, src)) try: copyfile(path.join(self.srcdir, src), path.join(self.outdir, self.imagedir, dest)) except (IOError, OSError) as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) continue if self.config.epub_fix_images: @@ -423,7 +424,7 @@ class EpubBuilder(StandaloneHTMLBuilder): try: img.save(path.join(self.outdir, self.imagedir, dest)) except (IOError, OSError) as err: - logger.warning('cannot write image file %r: %s', + logger.warning(__('cannot write image file %r: %s'), path.join(self.srcdir, src), err) def copy_image_files(self): @@ -434,7 +435,7 @@ class EpubBuilder(StandaloneHTMLBuilder): if self.images: if self.config.epub_fix_images or self.config.epub_max_image_width: if not Image: - logger.warning('PIL not found - copying image files') + logger.warning(__('PIL not found - copying image files')) super(EpubBuilder, self).copy_image_files() else: self.copy_image_files_pil() @@ -464,14 +465,14 @@ class EpubBuilder(StandaloneHTMLBuilder): def build_mimetype(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file mimetype.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) copy_asset_file(path.join(self.template_dir, 'mimetype'), path.join(outdir, outname)) def build_container(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file META-INF/container.xml.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) filename = path.join(outdir, outname) ensuredir(path.dirname(filename)) copy_asset_file(path.join(self.template_dir, 'container.xml'), filename) @@ -501,7 +502,7 @@ class EpubBuilder(StandaloneHTMLBuilder): """Write the metainfo file content.opf It contains bibliographic data, a file list and the spine (the reading order). """ - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) metadata = self.content_metadata() # files @@ -527,7 +528,7 @@ class EpubBuilder(StandaloneHTMLBuilder): # we always have JS and potentially OpenSearch files, don't # always warn about them if ext not in ('.js', '.xml'): - logger.warning('unknown mimetype for %s, ignoring', filename, + logger.warning(__('unknown mimetype for %s, ignoring'), filename, type='epub', subtype='unknown_project_files') continue filename = filename.replace(os.sep, '/') @@ -680,7 +681,7 @@ class EpubBuilder(StandaloneHTMLBuilder): def build_toc(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file toc.ncx.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) if self.config.epub_tocscope == 'default': doctree = self.env.get_and_resolve_doctree(self.config.master_doc, @@ -705,7 +706,7 @@ class EpubBuilder(StandaloneHTMLBuilder): It is a zip file with the mimetype file stored uncompressed as the first entry. """ - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) epub_filename = path.join(outdir, outname) with ZipFile(epub_filename, 'w', ZIP_DEFLATED) as epub: # type: ignore epub.write(path.join(outdir, 'mimetype'), 'mimetype', ZIP_STORED) # type: ignore diff --git a/sphinx/builders/applehelp.py b/sphinx/builders/applehelp.py index d7974696c..79d57210c 100644 --- a/sphinx/builders/applehelp.py +++ b/sphinx/builders/applehelp.py @@ -20,6 +20,7 @@ from os import path, environ from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.config import string_classes from sphinx.errors import SphinxError +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.console import bold # type: ignore from sphinx.util.fileutil import copy_asset @@ -60,11 +61,11 @@ access_page_template = '''\ class AppleHelpIndexerFailed(SphinxError): - category = 'Help indexer failed' + category = __('Help indexer failed') class AppleHelpCodeSigningFailed(SphinxError): - category = 'Code signing failed' + category = __('Code signing failed') class AppleHelpBuilder(StandaloneHTMLBuilder): @@ -73,10 +74,10 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): on the ``hiutil`` command line tool. """ name = 'applehelp' - epilog = ('The help book is in %(outdir)s.\n' - 'Note that won\'t be able to view it unless you put it in ' - '~/Library/Documentation/Help or install it in your application ' - 'bundle.') + epilog = __('The help book is in %(outdir)s.\n' + 'Note that won\'t be able to view it unless you put it in ' + '~/Library/Documentation/Help or install it in your application ' + 'bundle.') # don't copy the reST source copysource = False @@ -100,8 +101,8 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): self.link_suffix = '.html' if self.config.applehelp_bundle_id is None: - raise SphinxError('You must set applehelp_bundle_id before ' - 'building Apple Help output') + raise SphinxError(__('You must set applehelp_bundle_id before ' + 'building Apple Help output')) self.bundle_path = path.join(self.outdir, self.config.applehelp_bundle_name + @@ -124,13 +125,13 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): target_dir = self.outdir if path.isdir(source_dir): - logger.info(bold('copying localized files... '), nonl=True) + logger.info(bold(__('copying localized files... ')), nonl=True) excluded = Matcher(self.config.exclude_patterns + ['**/.*']) copy_asset(source_dir, target_dir, excluded, context=self.globalcontext, renderer=self.templates) - logger.info('done') + logger.info(__('done')) def build_helpbook(self): # type: () -> None @@ -171,36 +172,36 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): if self.config.applehelp_remote_url is not None: info_plist['HPDBookRemoteURL'] = self.config.applehelp_remote_url - logger.info(bold('writing Info.plist... '), nonl=True) + logger.info(bold(__('writing Info.plist... ')), nonl=True) with open(path.join(contents_dir, 'Info.plist'), 'wb') as f: write_plist(info_plist, f) - logger.info('done') + logger.info(__('done')) # Copy the icon, if one is supplied if self.config.applehelp_icon: - logger.info(bold('copying icon... '), nonl=True) + logger.info(bold(__('copying icon... ')), nonl=True) try: copyfile(path.join(self.srcdir, self.config.applehelp_icon), path.join(resources_dir, info_plist['HPDBookIconPath'])) - logger.info('done') + logger.info(__('done')) except Exception as err: - logger.warning('cannot copy icon file %r: %s', + logger.warning(__('cannot copy icon file %r: %s'), path.join(self.srcdir, self.config.applehelp_icon), err) del info_plist['HPDBookIconPath'] # Build the access page - logger.info(bold('building access page...'), nonl=True) + logger.info(bold(__('building access page...')), nonl=True) with codecs.open(path.join(language_dir, '_access.html'), 'w') as f: # type: ignore f.write(access_page_template % { 'toc': htmlescape(toc, quote=True), 'title': htmlescape(self.config.applehelp_title) }) - logger.info('done') + logger.info(__('done')) # Generate the help index - logger.info(bold('generating help index... '), nonl=True) + logger.info(bold(__('generating help index... ')), nonl=True) args = [ self.config.applehelp_indexer_path, @@ -222,9 +223,9 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): args += ['-l', self.config.applehelp_locale] if self.config.applehelp_disable_external_tools: - logger.info('skipping') + logger.info(__('skipping')) - logger.warning('you will need to index this help book with:\n %s', + logger.warning(__('you will need to index this help book with:\n %s'), ' '.join([pipes.quote(arg) for arg in args])) else: try: @@ -237,13 +238,13 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): if p.returncode != 0: raise AppleHelpIndexerFailed(output) else: - logger.info('done') + logger.info(__('done')) except OSError: - raise AppleHelpIndexerFailed('Command not found: %s' % args[0]) + raise AppleHelpIndexerFailed(__('Command not found: %s') % args[0]) # If we've been asked to, sign the bundle if self.config.applehelp_codesign_identity: - logger.info(bold('signing help book... '), nonl=True) + logger.info(bold(__('signing help book... ')), nonl=True) args = [ self.config.applehelp_codesign_path, @@ -256,8 +257,8 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): args.append(self.bundle_path) if self.config.applehelp_disable_external_tools: - logger.info('skipping') - logger.warning('you will need to sign this help book with:\n %s', + logger.info(__('skipping')) + logger.warning(__('you will need to sign this help book with:\n %s'), ' '.join([pipes.quote(arg) for arg in args])) else: try: @@ -270,9 +271,9 @@ class AppleHelpBuilder(StandaloneHTMLBuilder): if p.returncode != 0: raise AppleHelpCodeSigningFailed(output) else: - logger.info('done') + logger.info(__('done')) except OSError: - raise AppleHelpCodeSigningFailed('Command not found: %s' % args[0]) + raise AppleHelpCodeSigningFailed(__('Command not found: %s') % args[0]) def setup(app): diff --git a/sphinx/builders/changes.py b/sphinx/builders/changes.py index e8b6650cb..011aa6ebe 100644 --- a/sphinx/builders/changes.py +++ b/sphinx/builders/changes.py @@ -16,7 +16,7 @@ from six import iteritems from sphinx import package_dir from sphinx.builders import Builder -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.theming import HTMLThemeFactory from sphinx.util import logging from sphinx.util.console import bold # type: ignore @@ -38,7 +38,7 @@ class ChangesBuilder(Builder): Write a summary with all versionadded/changed directives. """ name = 'changes' - epilog = 'The overview file is in %(outdir)s.' + epilog = __('The overview file is in %(outdir)s.') def init(self): # type: () -> None @@ -64,7 +64,7 @@ class ChangesBuilder(Builder): apichanges = [] # type: List[Tuple[unicode, unicode, int]] otherchanges = {} # type: Dict[Tuple[unicode, unicode], List[Tuple[unicode, unicode, int]]] # NOQA if version not in self.env.versionchanges: - logger.info(bold('no changes in version %s.' % version)) + logger.info(bold(__('no changes in version %s.') % version)) return logger.info(bold('writing summary file...')) for type, docname, lineno, module, descname, content in \ @@ -129,14 +129,14 @@ class ChangesBuilder(Builder): break return line - logger.info(bold('copying source files...')) + logger.info(bold(__('copying source files...'))) for docname in self.env.all_docs: with codecs.open(self.env.doc2path(docname), 'r', # type: ignore self.env.config.source_encoding) as f: try: lines = f.readlines() except UnicodeDecodeError: - logger.warning('could not read %r for changelog creation', docname) + logger.warning(__('could not read %r for changelog creation'), docname) continue targetfn = path.join(self.outdir, 'rst', os_path(docname)) + '.html' ensuredir(path.dirname(targetfn)) diff --git a/sphinx/builders/devhelp.py b/sphinx/builders/devhelp.py index 96e06afdc..3d35b7496 100644 --- a/sphinx/builders/devhelp.py +++ b/sphinx/builders/devhelp.py @@ -5,7 +5,7 @@ Build HTML documentation and Devhelp_ support files. - .. _Devhelp: http://live.gnome.org/devhelp + .. _Devhelp: https://wiki.gnome.org/Apps/Devhelp :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. @@ -21,6 +21,7 @@ from docutils import nodes from sphinx import addnodes from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.environment.adapters.indexentries import IndexEntries +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import make_filename @@ -43,10 +44,10 @@ class DevhelpBuilder(StandaloneHTMLBuilder): Builder that also outputs GNOME Devhelp file. """ name = 'devhelp' - epilog = ('To view the help file:\n' - '$ mkdir -p $HOME/.local/share/devhelp/%(project)s\n' - '$ ln -s %(outdir)s $HOME/.local/share/devhelp/%(project)s\n' - '$ devhelp') + epilog = __('To view the help file:\n' + '$ mkdir -p $HOME/.local/share/devhelp/%(project)s\n' + '$ ln -s %(outdir)s $HOME/.local/share/devhelp/%(project)s\n' + '$ devhelp') # don't copy the reST source copysource = False @@ -69,7 +70,7 @@ class DevhelpBuilder(StandaloneHTMLBuilder): def build_devhelp(self, outdir, outname): # type: (unicode, unicode) -> None - logger.info('dumping devhelp index...') + logger.info(__('dumping devhelp index...')) # Basic info root = etree.Element('book', diff --git a/sphinx/builders/dummy.py b/sphinx/builders/dummy.py index 08d99a584..805924290 100644 --- a/sphinx/builders/dummy.py +++ b/sphinx/builders/dummy.py @@ -11,6 +11,7 @@ from sphinx.builders import Builder +from sphinx.locale import __ if False: # For type annotation @@ -21,7 +22,7 @@ if False: class DummyBuilder(Builder): name = 'dummy' - epilog = 'The dummy builder generates no files.' + epilog = __('The dummy builder generates no files.') allow_parallel = True diff --git a/sphinx/builders/epub3.py b/sphinx/builders/epub3.py index 008942a6a..752a3990d 100644 --- a/sphinx/builders/epub3.py +++ b/sphinx/builders/epub3.py @@ -16,6 +16,7 @@ from os import path from sphinx import package_dir from sphinx.builders import _epub_base from sphinx.config import string_classes, ENUM +from sphinx.locale import __ from sphinx.util import logging, xmlname_checker from sphinx.util.fileutil import copy_asset_file from sphinx.util.i18n import format_date @@ -63,7 +64,7 @@ class Epub3Builder(_epub_base.EpubBuilder): an epub file. """ name = 'epub' - epilog = 'The ePub file is in %(outdir)s.' + epilog = __('The ePub file is in %(outdir)s.') supported_remote_images = False template_dir = path.join(package_dir, 'templates', 'epub3') @@ -85,39 +86,40 @@ class Epub3Builder(_epub_base.EpubBuilder): self.build_epub(self.outdir, self.config.epub_basename + '.epub') def validate_config_value(self): + # type: () -> None # <package> lang attribute, dc:language if not self.app.config.epub_language: - logger.warning('conf value "epub_language" (or "language") ' - 'should not be empty for EPUB3') + logger.warning(__('conf value "epub_language" (or "language") ' + 'should not be empty for EPUB3')) # <package> unique-identifier attribute if not xmlname_checker().match(self.app.config.epub_uid): - logger.warning('conf value "epub_uid" should be XML NAME for EPUB3') + logger.warning(__('conf value "epub_uid" should be XML NAME for EPUB3')) # dc:title if not self.app.config.epub_title: - logger.warning('conf value "epub_title" (or "html_title") ' - 'should not be empty for EPUB3') + logger.warning(__('conf value "epub_title" (or "html_title") ' + 'should not be empty for EPUB3')) # dc:creator if not self.app.config.epub_author: - logger.warning('conf value "epub_author" should not be empty for EPUB3') + logger.warning(__('conf value "epub_author" should not be empty for EPUB3')) # dc:contributor if not self.app.config.epub_contributor: - logger.warning('conf value "epub_contributor" should not be empty for EPUB3') + logger.warning(__('conf value "epub_contributor" should not be empty for EPUB3')) # dc:description if not self.app.config.epub_description: - logger.warning('conf value "epub_description" should not be empty for EPUB3') + logger.warning(__('conf value "epub_description" should not be empty for EPUB3')) # dc:publisher if not self.app.config.epub_publisher: - logger.warning('conf value "epub_publisher" should not be empty for EPUB3') + logger.warning(__('conf value "epub_publisher" should not be empty for EPUB3')) # dc:rights if not self.app.config.epub_copyright: - logger.warning('conf value "epub_copyright" (or "copyright")' - 'should not be empty for EPUB3') + logger.warning(__('conf value "epub_copyright" (or "copyright")' + 'should not be empty for EPUB3')) # dc:identifier if not self.app.config.epub_identifier: - logger.warning('conf value "epub_identifier" should not be empty for EPUB3') + logger.warning(__('conf value "epub_identifier" should not be empty for EPUB3')) # meta ibooks:version if not self.app.config.version: - logger.warning('conf value "version" should not be empty for EPUB3') + logger.warning(__('conf value "version" should not be empty for EPUB3')) def content_metadata(self): # type: () -> Dict @@ -203,7 +205,7 @@ class Epub3Builder(_epub_base.EpubBuilder): def build_navigation_doc(self, outdir, outname): # type: (unicode, unicode) -> None """Write the metainfo file nav.xhtml.""" - logger.info('writing %s file...', outname) + logger.info(__('writing %s file...'), outname) if self.config.epub_tocscope == 'default': doctree = self.env.get_and_resolve_doctree( @@ -234,9 +236,9 @@ def setup(app): app.add_config_value('epub_theme', 'epub', 'epub') app.add_config_value('epub_theme_options', {}, 'epub') app.add_config_value('epub_title', lambda self: self.html_title, 'epub') - app.add_config_value('epub_author', 'unknown', 'epub') + app.add_config_value('epub_author', lambda self: self.author, 'epub') app.add_config_value('epub_language', lambda self: self.language or 'en', 'epub') - app.add_config_value('epub_publisher', 'unknown', 'epub') + app.add_config_value('epub_publisher', lambda self: self.author, 'epub') app.add_config_value('epub_copyright', lambda self: self.copyright, 'epub') app.add_config_value('epub_identifier', 'unknown', 'epub') app.add_config_value('epub_scheme', 'unknown', 'epub') diff --git a/sphinx/builders/gettext.py b/sphinx/builders/gettext.py index 579e05534..c8fbb9c32 100644 --- a/sphinx/builders/gettext.py +++ b/sphinx/builders/gettext.py @@ -21,7 +21,8 @@ from uuid import uuid4 from six import iteritems, StringIO from sphinx.builders import Builder -from sphinx.locale import pairindextypes +from sphinx.domains.python import pairindextypes +from sphinx.locale import __ from sphinx.util import split_index_msg, logging, status_iterator from sphinx.util.console import bold # type: ignore from sphinx.util.i18n import find_catalog @@ -192,6 +193,7 @@ ltz = LocalTimeZone() def should_write(filepath, new_content): + # type: (unicode, unicode) -> bool if not path.exists(filepath): return True try: @@ -214,7 +216,7 @@ class MessageCatalogBuilder(I18nBuilder): Builds gettext-style message catalogs (.pot files). """ name = 'gettext' - epilog = 'The message catalogs are in %(outdir)s.' + epilog = __('The message catalogs are in %(outdir)s.') def init(self): # type: () -> None @@ -238,12 +240,12 @@ class MessageCatalogBuilder(I18nBuilder): # type: () -> None files = list(self._collect_templates()) files.sort() - logger.info(bold('building [%s]: ' % self.name), nonl=1) - logger.info('targets for %d template files', len(files)) + logger.info(bold(__('building [%s]: ') % self.name), nonl=1) + logger.info(__('targets for %d template files'), len(files)) extract_translations = self.templates.environment.extract_translations - for template in status_iterator(files, 'reading templates... ', "purple", # type: ignore # NOQA + for template in status_iterator(files, __('reading templates... '), "purple", # type: ignore # NOQA len(files), self.app.verbosity): with open(template, 'r', encoding='utf-8') as f: # type: ignore context = f.read() @@ -267,7 +269,7 @@ class MessageCatalogBuilder(I18nBuilder): timestamp, ltz).strftime('%Y-%m-%d %H:%M%z'), ) for textdomain, catalog in status_iterator(iteritems(self.catalogs), # type: ignore - "writing message catalogs... ", + __("writing message catalogs... "), "darkgreen", len(self.catalogs), self.app.verbosity, lambda textdomain__: textdomain__[0]): diff --git a/sphinx/builders/html.py b/sphinx/builders/html.py index 183ffc017..fc295329a 100644 --- a/sphinx/builders/html.py +++ b/sphinx/builders/html.py @@ -36,7 +36,7 @@ from sphinx.environment.adapters.asset import ImageAdapter from sphinx.environment.adapters.indexentries import IndexEntries from sphinx.environment.adapters.toctree import TocTree from sphinx.highlighting import PygmentsBridge -from sphinx.locale import _, l_ +from sphinx.locale import _, __ from sphinx.search import js_index from sphinx.theming import HTMLThemeFactory from sphinx.util import jsonimpl, logging, status_iterator @@ -96,28 +96,32 @@ class CSSContainer(list): the entry with Stylesheet class. """ def append(self, obj): + # type: (Union[unicode, Stylesheet]) -> None if isinstance(obj, Stylesheet): super(CSSContainer, self).append(obj) else: - super(CSSContainer, self).append(Stylesheet(obj, None, 'stylesheet')) + super(CSSContainer, self).append(Stylesheet(obj, None, 'stylesheet')) # type: ignore # NOQA def insert(self, index, obj): + # type: (int, Union[unicode, Stylesheet]) -> None warnings.warn('builder.css_files is deprecated. ' 'Please use app.add_stylesheet() instead.', RemovedInSphinx20Warning) if isinstance(obj, Stylesheet): super(CSSContainer, self).insert(index, obj) else: - super(CSSContainer, self).insert(index, Stylesheet(obj, None, 'stylesheet')) + super(CSSContainer, self).insert(index, Stylesheet(obj, None, 'stylesheet')) # type: ignore # NOQA - def extend(self, other): + def extend(self, other): # type: ignore + # type: (List[Union[unicode, Stylesheet]]) -> None warnings.warn('builder.css_files is deprecated. ' 'Please use app.add_stylesheet() instead.', RemovedInSphinx20Warning) for item in other: self.append(item) - def __iadd__(self, other): + def __iadd__(self, other): # type: ignore + # type: (List[Union[unicode, Stylesheet]]) -> CSSContainer warnings.warn('builder.css_files is deprecated. ' 'Please use app.add_stylesheet() instead.', RemovedInSphinx20Warning) @@ -126,6 +130,7 @@ class CSSContainer(list): return self def __add__(self, other): + # type: (List[Union[unicode, Stylesheet]]) -> CSSContainer ret = CSSContainer(self) ret += other return ret @@ -169,7 +174,7 @@ class BuildInfo(object): build_info.tags_hash = lines[3].split()[1].strip() return build_info except Exception as exc: - raise ValueError('build info file is broken: %r' % exc) + raise ValueError(__('build info file is broken: %r') % exc) def __init__(self, config=None, tags=None, config_categories=[]): # type: (Config, Tags, List[unicode]) -> None @@ -208,7 +213,7 @@ class StandaloneHTMLBuilder(Builder): """ name = 'html' format = 'html' - epilog = 'The HTML pages are in %(outdir)s.' + epilog = __('The HTML pages are in %(outdir)s.') copysource = True allow_parallel = True @@ -320,6 +325,7 @@ class StandaloneHTMLBuilder(Builder): @property def default_translator_class(self): + # type: () -> nodes.NodeVisitor use_html5_writer = self.config.html_experimental_html5_writer if use_html5_writer is None: use_html5_writer = self.default_html5_translator @@ -340,7 +346,7 @@ class StandaloneHTMLBuilder(Builder): yield docname return except ValueError as exc: - logger.warning('Failed to read build info file: %r', exc) + logger.warning(__('Failed to read build info file: %r'), exc) except IOError: # ignore errors on reading pass @@ -440,7 +446,7 @@ class StandaloneHTMLBuilder(Builder): # typically doesn't include the time of day lufmt = self.config.html_last_updated_fmt if lufmt is not None: - self.last_updated = format_date(lufmt or _('%b %d, %Y'), # type: ignore + self.last_updated = format_date(lufmt or _('%b %d, %Y'), language=self.config.language) else: self.last_updated = None @@ -452,7 +458,7 @@ class StandaloneHTMLBuilder(Builder): path.basename(self.config.html_favicon) or '' if not isinstance(self.config.html_use_opensearch, string_types): - logger.warning('html_use_opensearch config value must now be a string') + logger.warning(__('html_use_opensearch config value must now be a string')) self.relations = self.env.collect_relations() @@ -629,7 +635,7 @@ class StandaloneHTMLBuilder(Builder): def gen_indices(self): # type: () -> None - logger.info(bold('generating indices...'), nonl=1) + logger.info(bold(__('generating indices...')), nonl=1) # the global general index if self.use_index: @@ -647,7 +653,7 @@ class StandaloneHTMLBuilder(Builder): for pagename, context, template in pagelist: self.handle_page(pagename, context, template) - logger.info(bold('writing additional pages...'), nonl=1) + logger.info(bold(__('writing additional pages...')), nonl=1) # additional pages from conf.py for pagename, template in self.config.html_additional_pages.items(): @@ -713,7 +719,7 @@ class StandaloneHTMLBuilder(Builder): if self.images: stringify_func = ImageAdapter(self.app.env).get_original_image_uri ensuredir(path.join(self.outdir, self.imagedir)) - for src in status_iterator(self.images, 'copying images... ', "brown", + for src in status_iterator(self.images, __('copying images... '), "brown", len(self.images), self.app.verbosity, stringify_func=stringify_func): dest = self.images[src] @@ -721,7 +727,7 @@ class StandaloneHTMLBuilder(Builder): copyfile(path.join(self.srcdir, src), path.join(self.outdir, self.imagedir, dest)) except Exception as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) def copy_download_files(self): @@ -732,7 +738,7 @@ class StandaloneHTMLBuilder(Builder): # copy downloadable files if self.env.dlfiles: ensuredir(path.join(self.outdir, '_downloads')) - for src in status_iterator(self.env.dlfiles, 'copying downloadable files... ', + for src in status_iterator(self.env.dlfiles, __('copying downloadable files... '), "brown", len(self.env.dlfiles), self.app.verbosity, stringify_func=to_relpath): dest = self.env.dlfiles[src][1] @@ -740,13 +746,13 @@ class StandaloneHTMLBuilder(Builder): copyfile(path.join(self.srcdir, src), path.join(self.outdir, '_downloads', dest)) except Exception as err: - logger.warning('cannot copy downloadable file %r: %s', + logger.warning(__('cannot copy downloadable file %r: %s'), path.join(self.srcdir, src), err) def copy_static_files(self): # type: () -> None # copy static files - logger.info(bold('copying static files... '), nonl=True) + logger.info(bold(__('copying static files... ')), nonl=True) ensuredir(path.join(self.outdir, '_static')) # first, create pygments style file with open(path.join(self.outdir, '_static', 'pygments.css'), 'w') as f: @@ -781,7 +787,7 @@ class StandaloneHTMLBuilder(Builder): for static_path in self.config.html_static_path: entry = path.join(self.confdir, static_path) if not path.exists(entry): - logger.warning('html_static_path entry %r does not exist', entry) + logger.warning(__('html_static_path entry %r does not exist'), entry) continue copy_asset(entry, path.join(self.outdir, '_static'), excluded, context=ctx, renderer=self.templates) @@ -790,7 +796,7 @@ class StandaloneHTMLBuilder(Builder): logobase = path.basename(self.config.html_logo) logotarget = path.join(self.outdir, '_static', logobase) if not path.isfile(path.join(self.confdir, self.config.html_logo)): - logger.warning('logo file %r does not exist', self.config.html_logo) + logger.warning(__('logo file %r does not exist'), self.config.html_logo) elif not path.isfile(logotarget): copyfile(path.join(self.confdir, self.config.html_logo), logotarget) @@ -798,7 +804,7 @@ class StandaloneHTMLBuilder(Builder): iconbase = path.basename(self.config.html_favicon) icontarget = path.join(self.outdir, '_static', iconbase) if not path.isfile(path.join(self.confdir, self.config.html_favicon)): - logger.warning('favicon file %r does not exist', self.config.html_favicon) + logger.warning(__('favicon file %r does not exist'), self.config.html_favicon) elif not path.isfile(icontarget): copyfile(path.join(self.confdir, self.config.html_favicon), icontarget) @@ -807,17 +813,17 @@ class StandaloneHTMLBuilder(Builder): def copy_extra_files(self): # type: () -> None # copy html_extra_path files - logger.info(bold('copying extra files... '), nonl=True) + logger.info(bold(__('copying extra files... ')), nonl=True) excluded = Matcher(self.config.exclude_patterns) for extra_path in self.config.html_extra_path: entry = path.join(self.confdir, extra_path) if not path.exists(entry): - logger.warning('html_extra_path entry %r does not exist', entry) + logger.warning(__('html_extra_path entry %r does not exist'), entry) continue copy_asset(entry, self.outdir, excluded) - logger.info('done') + logger.info(__('done')) def write_buildinfo(self): # type: () -> None @@ -825,7 +831,7 @@ class StandaloneHTMLBuilder(Builder): with open(path.join(self.outdir, '.buildinfo'), 'w') as fp: self.build_info.dump(fp) except IOError as exc: - logger.warning('Failed to write build info file: %r', exc) + logger.warning(__('Failed to write build info file: %r'), exc) def cleanup(self): # type: () -> None @@ -872,9 +878,9 @@ class StandaloneHTMLBuilder(Builder): self.indexer.load(f, self.indexer_format) except (IOError, OSError, ValueError): if keep: - logger.warning('search index couldn\'t be loaded, but not all ' - 'documents will be built: the index will be ' - 'incomplete.') + logger.warning(__('search index couldn\'t be loaded, but not all ' + 'documents will be built: the index will be ' + 'incomplete.')) # delete all entries for files that will be rebuilt self.indexer.prune(keep) @@ -926,8 +932,8 @@ class StandaloneHTMLBuilder(Builder): if has_wildcard(pattern): # warn if both patterns contain wildcards if has_wildcard(matched): - logger.warning('page %s matches two patterns in ' - 'html_sidebars: %r and %r', + logger.warning(__('page %s matches two patterns in ' + 'html_sidebars: %r and %r'), pagename, matched, pattern) # else the already matched pattern is more specific # than the present one, because it contains no wildcard @@ -1012,9 +1018,9 @@ class StandaloneHTMLBuilder(Builder): try: output = self.templates.render(templatename, ctx) except UnicodeError: - logger.warning("a Unicode error occurred when rendering the page %s. " - "Please make sure all config values that contain " - "non-ASCII content are Unicode strings.", pagename) + logger.warning(__("a Unicode error occurred when rendering the page %s. " + "Please make sure all config values that contain " + "non-ASCII content are Unicode strings."), pagename) return if not outfilename: @@ -1025,7 +1031,7 @@ class StandaloneHTMLBuilder(Builder): with codecs.open(outfilename, 'w', ctx['encoding'], 'xmlcharrefreplace') as f: # type: ignore # NOQA f.write(output) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", outfilename, err) + logger.warning(__("error writing file %s: %s"), outfilename, err) if self.copysource and ctx.get('sourcename'): # copy the source file for the "show source" link source_name = path.join(self.outdir, '_sources', @@ -1045,14 +1051,14 @@ class StandaloneHTMLBuilder(Builder): def dump_inventory(self): # type: () -> None - logger.info(bold('dumping object inventory... '), nonl=True) + logger.info(bold(__('dumping object inventory... ')), nonl=True) InventoryFile.dump(path.join(self.outdir, INVENTORY_FILENAME), self.env, self) - logger.info('done') + logger.info(__('done')) def dump_search_index(self): # type: () -> None logger.info( - bold('dumping search index in %s ... ' % self.indexer.label()), + bold(__('dumping search index in %s ... ') % self.indexer.label()), nonl=True) self.indexer.prune(self.env.all_docs) searchindexfn = path.join(self.outdir, self.searchindex_filename) @@ -1065,7 +1071,7 @@ class StandaloneHTMLBuilder(Builder): with f: self.indexer.dump(f, self.indexer_format) movefile(searchindexfn + '.tmp', searchindexfn) - logger.info('done') + logger.info(__('done')) class DirectoryHTMLBuilder(StandaloneHTMLBuilder): @@ -1107,7 +1113,7 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder): HTML page. """ name = 'singlehtml' - epilog = 'The HTML page is in %(outdir)s.' + epilog = __('The HTML page is in %(outdir)s.') copysource = False @@ -1237,24 +1243,24 @@ class SingleFileHTMLBuilder(StandaloneHTMLBuilder): # type: (Any) -> None docnames = self.env.all_docs - logger.info(bold('preparing documents... '), nonl=True) + logger.info(bold(__('preparing documents... ')), nonl=True) self.prepare_writing(docnames) - logger.info('done') + logger.info(__('done')) - logger.info(bold('assembling single document... '), nonl=True) + logger.info(bold(__('assembling single document... ')), nonl=True) doctree = self.assemble_doctree() self.env.toc_secnumbers = self.assemble_toc_secnumbers() self.env.toc_fignumbers = self.assemble_toc_fignumbers() logger.info('') - logger.info(bold('writing... '), nonl=True) + logger.info(bold(__('writing... ')), nonl=True) self.write_doc_serialized(self.config.master_doc, doctree) self.write_doc(self.config.master_doc, doctree) - logger.info('done') + logger.info(__('done')) def finish(self): # type: () -> None # no indices or search pages are supported - logger.info(bold('writing additional files...'), nonl=1) + logger.info(bold(__('writing additional files...')), nonl=1) # additional pages from conf.py for pagename, template in self.config.html_additional_pages.items(): @@ -1371,7 +1377,7 @@ class PickleHTMLBuilder(SerializingHTMLBuilder): A Builder that dumps the generated HTML into pickle files. """ name = 'pickle' - epilog = 'You can now process the pickle files in %(outdir)s.' + epilog = __('You can now process the pickle files in %(outdir)s.') implementation = pickle implementation_dumps_unicode = False @@ -1392,7 +1398,7 @@ class JSONHTMLBuilder(SerializingHTMLBuilder): A builder that dumps the generated HTML into JSON files. """ name = 'json' - epilog = 'You can now process the JSON files in %(outdir)s.' + epilog = __('You can now process the JSON files in %(outdir)s.') implementation = jsonimpl implementation_dumps_unicode = True @@ -1421,7 +1427,7 @@ def setup(app): app.add_config_value('html_theme_path', [], 'html') app.add_config_value('html_theme_options', {}, 'html') app.add_config_value('html_title', - lambda self: l_('%s %s documentation') % (self.project, self.release), + lambda self: _('%s %s documentation') % (self.project, self.release), 'html', string_classes) app.add_config_value('html_short_title', lambda self: self.html_title, 'html') app.add_config_value('html_style', None, 'html', string_classes) diff --git a/sphinx/builders/htmlhelp.py b/sphinx/builders/htmlhelp.py index ac32a42db..8d4ded246 100644 --- a/sphinx/builders/htmlhelp.py +++ b/sphinx/builders/htmlhelp.py @@ -20,6 +20,7 @@ from docutils import nodes from sphinx import addnodes from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.environment.adapters.indexentries import IndexEntries +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import make_filename from sphinx.util.pycompat import htmlescape @@ -174,8 +175,8 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): index files. Adapted from the original Doc/tools/prechm.py. """ name = 'htmlhelp' - epilog = ('You can now run HTML Help Workshop with the .htp file in ' - '%(outdir)s.') + epilog = __('You can now run HTML Help Workshop with the .htp file in ' + '%(outdir)s.') # don't copy the reST source copysource = False @@ -228,12 +229,12 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): def build_hhx(self, outdir, outname): # type: (unicode, unicode) -> None - logger.info('dumping stopword list...') + logger.info(__('dumping stopword list...')) with self.open_file(outdir, outname + '.stp') as f: for word in sorted(stopwords): print(word, file=f) - logger.info('writing project file...') + logger.info(__('writing project file...')) with self.open_file(outdir, outname + '.hhp') as f: f.write(project_template % { 'outname': outname, @@ -254,7 +255,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): print(path.join(root, fn)[olen:].replace(os.sep, '\\'), file=f) - logger.info('writing TOC file...') + logger.info(__('writing TOC file...')) with self.open_file(outdir, outname + '.hhc') as f: f.write(contents_header) # special books @@ -296,7 +297,7 @@ class HTMLHelpBuilder(StandaloneHTMLBuilder): write_toc(node) f.write(contents_footer) - logger.info('writing index file...') + logger.info(__('writing index file...')) index = IndexEntries(self.env).create_index(self) with self.open_file(outdir, outname + '.hhk') as f: f.write('<UL>\n') diff --git a/sphinx/builders/latex.py b/sphinx/builders/latex.py index c66d00d4f..e0ce9f235 100644 --- a/sphinx/builders/latex.py +++ b/sphinx/builders/latex.py @@ -23,7 +23,7 @@ from sphinx.config import string_classes, ENUM from sphinx.environment import NoUri from sphinx.environment.adapters.asset import ImageAdapter from sphinx.errors import SphinxError, ConfigError -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import texescape, logging, status_iterator from sphinx.util.console import bold, darkgreen # type: ignore from sphinx.util.docutils import new_document @@ -48,11 +48,11 @@ class LaTeXBuilder(Builder): """ name = 'latex' format = 'latex' - epilog = 'The LaTeX files are in %(outdir)s.' + epilog = __('The LaTeX files are in %(outdir)s.') if os.name == 'posix': - epilog += ("\nRun 'make' in that directory to run these through " - "(pdf)latex\n" - "(use `make latexpdf' here to do that automatically).") + epilog += __("\nRun 'make' in that directory to run these through " + "(pdf)latex\n" + "(use `make latexpdf' here to do that automatically).") supported_image_types = ['application/pdf', 'image/png', 'image/jpeg'] supported_remote_images = False @@ -62,7 +62,7 @@ class LaTeXBuilder(Builder): # type: () -> None self.docnames = [] # type: Iterable[unicode] self.document_data = [] # type: List[Tuple[unicode, unicode, unicode, unicode, unicode, bool]] # NOQA - self.usepackages = [] # type: List[unicode] + self.usepackages = self.app.registry.latex_packages texescape.init() def get_outdated_docs(self): @@ -85,16 +85,16 @@ class LaTeXBuilder(Builder): # type: () -> None preliminary_document_data = [list(x) for x in self.config.latex_documents] if not preliminary_document_data: - logger.warning('no "latex_documents" config value found; no documents ' - 'will be written') + logger.warning(__('no "latex_documents" config value found; no documents ' + 'will be written')) return # assign subdirs to titles self.titles = [] # type: List[Tuple[unicode, unicode]] for entry in preliminary_document_data: docname = entry[0] if docname not in self.env.all_docs: - logger.warning('"latex_documents" config value references unknown ' - 'document %s', docname) + logger.warning(__('"latex_documents" config value references unknown ' + 'document %s'), docname) continue self.document_data.append(entry) # type: ignore if docname.endswith(SEP + 'index'): @@ -131,7 +131,7 @@ class LaTeXBuilder(Builder): destination = FileOutput( destination_path=path.join(self.outdir, targetname), encoding='utf-8') - logger.info("processing %s...", targetname, nonl=1) + logger.info(__("processing %s..."), targetname, nonl=1) toctrees = self.env.get_doctree(docname).traverse(addnodes.toctree) if toctrees: if toctrees[0].get('maxdepth') > 0: @@ -145,7 +145,7 @@ class LaTeXBuilder(Builder): appendices=((docclass != 'howto') and self.config.latex_appendices or [])) doctree['tocdepth'] = tocdepth self.post_process_images(doctree) - logger.info("writing... ", nonl=1) + logger.info(__("writing... "), nonl=1) doctree.settings = docsettings doctree.settings.author = author doctree.settings.title = title @@ -191,7 +191,7 @@ class LaTeXBuilder(Builder): appendix['docname'] = docname largetree.append(appendix) logger.info('') - logger.info("resolving references...") + logger.info(__("resolving references...")) self.env.resolve_references(largetree, indexfile, self) # resolve :ref:s to distant tex files -- we can't add a cross-reference, # but append the document name @@ -216,7 +216,7 @@ class LaTeXBuilder(Builder): # copy TeX support files from texinputs context = {'latex_engine': self.config.latex_engine} - logger.info(bold('copying TeX support files...')) + logger.info(bold(__('copying TeX support files...'))) staticdirname = path.join(package_dir, 'texinputs') for filename in os.listdir(staticdirname): if not filename.startswith('.'): @@ -231,7 +231,7 @@ class LaTeXBuilder(Builder): # copy additional files if self.config.latex_additional_files: - logger.info(bold('copying additional files...'), nonl=1) + logger.info(bold(__('copying additional files...')), nonl=1) for filename in self.config.latex_additional_files: logger.info(' ' + filename, nonl=1) copy_asset_file(path.join(self.confdir, filename), self.outdir) @@ -240,16 +240,16 @@ class LaTeXBuilder(Builder): # the logo is handled differently if self.config.latex_logo: if not path.isfile(path.join(self.confdir, self.config.latex_logo)): - raise SphinxError('logo file %r does not exist' % self.config.latex_logo) + raise SphinxError(__('logo file %r does not exist') % self.config.latex_logo) else: copy_asset_file(path.join(self.confdir, self.config.latex_logo), self.outdir) - logger.info('done') + logger.info(__('done')) def copy_image_files(self): # type: () -> None if self.images: stringify_func = ImageAdapter(self.app.env).get_original_image_uri - for src in status_iterator(self.images, 'copying images... ', "brown", + for src in status_iterator(self.images, __('copying images... '), "brown", len(self.images), self.app.verbosity, stringify_func=stringify_func): dest = self.images[src] @@ -257,27 +257,27 @@ class LaTeXBuilder(Builder): copy_asset_file(path.join(self.srcdir, src), path.join(self.outdir, dest)) except Exception as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) -def validate_config_values(app): - # type: (Sphinx) -> None - for document in app.config.latex_documents: +def validate_config_values(app, config): + # type: (Sphinx, Config) -> None + for document in config.latex_documents: try: text_type(document[2]) except UnicodeDecodeError: raise ConfigError( - 'Invalid latex_documents.title found (might contain non-ASCII chars. ' - 'Please use u"..." notation instead): %r' % (document,) + __('Invalid latex_documents.title found (might contain non-ASCII chars. ' + 'Please use u"..." notation instead): %r') % (document,) ) try: text_type(document[3]) except UnicodeDecodeError: raise ConfigError( - 'Invalid latex_documents.author found (might contain non-ASCII chars. ' - 'Please use u"..." notation instead): %r' % (document,) + __('Invalid latex_documents.author found (might contain non-ASCII chars. ' + 'Please use u"..." notation instead): %r') % (document,) ) @@ -303,7 +303,7 @@ def default_latex_docclass(config): def setup(app): # type: (Sphinx) -> Dict[unicode, Any] app.add_builder(LaTeXBuilder) - app.connect('builder-inited', validate_config_values) + app.connect('config-inited', validate_config_values) app.add_config_value('latex_engine', default_latex_engine, None, ENUM('pdflatex', 'xelatex', 'lualatex', 'platex')) diff --git a/sphinx/builders/linkcheck.py b/sphinx/builders/linkcheck.py index 8a10a4f2c..22e7f01ad 100644 --- a/sphinx/builders/linkcheck.py +++ b/sphinx/builders/linkcheck.py @@ -31,6 +31,7 @@ except ImportError: pass from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import encode_uri, requests, logging from sphinx.util.console import ( # type: ignore purple, red, darkgreen, darkgray, darkred, turquoise @@ -58,6 +59,7 @@ class AnchorCheckParser(html_parser.HTMLParser): self.found = False def handle_starttag(self, tag, attrs): + # type: (Any, Any) -> None for key, value in attrs: if key in ('id', 'name') and value == self.search_anchor: self.found = True @@ -90,8 +92,8 @@ class CheckExternalLinksBuilder(Builder): Checks for broken external links. """ name = 'linkcheck' - epilog = ('Look for any errors in the above output or in ' - '%(outdir)s/output.txt') + epilog = __('Look for any errors in the above output or in ' + '%(outdir)s/output.txt') def init(self): # type: () -> None @@ -151,7 +153,7 @@ class CheckExternalLinksBuilder(Builder): found = check_anchor(response, unquote(anchor)) if not found: - raise Exception("Anchor '%s' not found" % anchor) + raise Exception(__("Anchor '%s' not found") % anchor) else: try: # try a HEAD request first, which should be easier on @@ -249,7 +251,7 @@ class CheckExternalLinksBuilder(Builder): elif status == 'broken': self.write_entry('broken', docname, lineno, uri + ': ' + info) if self.app.quiet or self.app.warningiserror: - logger.warning('broken link: %s (%s)', uri, info, + logger.warning(__('broken link: %s (%s)'), uri, info, location=(self.env.doc2path(docname), lineno)) else: logger.info(red('broken ') + uri + red(' - ' + info)) diff --git a/sphinx/builders/manpage.py b/sphinx/builders/manpage.py index 7a691ccf0..32bd9950b 100644 --- a/sphinx/builders/manpage.py +++ b/sphinx/builders/manpage.py @@ -18,6 +18,7 @@ from six import string_types from sphinx import addnodes from sphinx.builders import Builder from sphinx.environment import NoUri +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.console import bold, darkgreen # type: ignore from sphinx.util.nodes import inline_all_toctrees @@ -39,7 +40,7 @@ class ManualPageBuilder(Builder): """ name = 'man' format = 'man' - epilog = 'The manual pages are in %(outdir)s.' + epilog = __('The manual pages are in %(outdir)s.') default_translator_class = ManualPageTranslator supported_image_types = [] # type: List[unicode] @@ -47,8 +48,8 @@ class ManualPageBuilder(Builder): def init(self): # type: () -> None if not self.config.man_pages: - logger.warning('no "man_pages" config value found; no manual pages ' - 'will be written') + logger.warning(__('no "man_pages" config value found; no manual pages ' + 'will be written')) def get_outdated_docs(self): # type: () -> Union[unicode, List[unicode]] @@ -68,7 +69,7 @@ class ManualPageBuilder(Builder): components=(docwriter,), read_config_files=True).get_default_values() - logger.info(bold('writing... '), nonl=True) + logger.info(bold(__('writing... ')), nonl=True) for info in self.config.man_pages: docname, name, description, authors, section = info diff --git a/sphinx/builders/qthelp.py b/sphinx/builders/qthelp.py index 776a2d142..0539e18de 100644 --- a/sphinx/builders/qthelp.py +++ b/sphinx/builders/qthelp.py @@ -22,6 +22,7 @@ from sphinx import addnodes from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.config import string_classes from sphinx.environment.adapters.indexentries import IndexEntries +from sphinx.locale import __ from sphinx.util import force_decode, logging from sphinx.util.osutil import make_filename from sphinx.util.pycompat import htmlescape @@ -107,11 +108,11 @@ class QtHelpBuilder(StandaloneHTMLBuilder): Builder that also outputs Qt help project, contents and index files. """ name = 'qthelp' - epilog = ('You can now run "qcollectiongenerator" with the .qhcp ' - 'project file in %(outdir)s, like this:\n' - '$ qcollectiongenerator %(outdir)s/%(project)s.qhcp\n' - 'To view the help file:\n' - '$ assistant -collectionFile %(outdir)s/%(project)s.qhc') + epilog = __('You can now run "qcollectiongenerator" with the .qhcp ' + 'project file in %(outdir)s, like this:\n' + '$ qcollectiongenerator %(outdir)s/%(project)s.qhcp\n' + 'To view the help file:\n' + '$ assistant -collectionFile %(outdir)s/%(project)s.qhc') # don't copy the reST source copysource = False @@ -147,7 +148,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder): def build_qhp(self, outdir, outname): # type: (unicode, unicode) -> None - logger.info('writing project file...') + logger.info(__('writing project file...')) # sections tocdoc = self.env.get_and_resolve_doctree(self.config.master_doc, self, @@ -230,7 +231,7 @@ class QtHelpBuilder(StandaloneHTMLBuilder): nspace, 'doc', self.get_target_uri(self.config.master_doc)) startpage = 'qthelp://' + posixpath.join(nspace, 'doc', 'index.html') - logger.info('writing collection project file...') + logger.info(__('writing collection project file...')) with codecs.open(path.join(outdir, outname + '.qhcp'), 'w', 'utf-8') as f: # type: ignore # NOQA f.write(collection_template % { 'outname': htmlescape(outname), diff --git a/sphinx/builders/texinfo.py b/sphinx/builders/texinfo.py index c9a95a5fc..2a022158f 100644 --- a/sphinx/builders/texinfo.py +++ b/sphinx/builders/texinfo.py @@ -20,7 +20,7 @@ from sphinx import addnodes from sphinx.builders import Builder from sphinx.environment import NoUri from sphinx.environment.adapters.asset import ImageAdapter -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util import status_iterator from sphinx.util.console import bold, darkgreen # type: ignore @@ -98,11 +98,11 @@ class TexinfoBuilder(Builder): """ name = 'texinfo' format = 'texinfo' - epilog = 'The Texinfo files are in %(outdir)s.' + epilog = __('The Texinfo files are in %(outdir)s.') if os.name == 'posix': - epilog += ("\nRun 'make' in that directory to run these through " - "makeinfo\n" - "(use 'make info' here to do that automatically).") + epilog += __("\nRun 'make' in that directory to run these through " + "makeinfo\n" + "(use 'make info' here to do that automatically).") supported_image_types = ['image/png', 'image/jpeg', 'image/gif'] @@ -133,16 +133,16 @@ class TexinfoBuilder(Builder): # type: () -> None preliminary_document_data = [list(x) for x in self.config.texinfo_documents] if not preliminary_document_data: - logger.warning('no "texinfo_documents" config value found; no documents ' - 'will be written') + logger.warning(__('no "texinfo_documents" config value found; no documents ' + 'will be written')) return # assign subdirs to titles self.titles = [] # type: List[Tuple[unicode, unicode]] for entry in preliminary_document_data: docname = entry[0] if docname not in self.env.all_docs: - logger.warning('"texinfo_documents" config value references unknown ' - 'document %s', docname) + logger.warning(__('"texinfo_documents" config value references unknown ' + 'document %s'), docname) continue self.document_data.append(entry) # type: ignore if docname.endswith(SEP + 'index'): @@ -164,11 +164,11 @@ class TexinfoBuilder(Builder): destination = FileOutput( destination_path=path.join(self.outdir, targetname), encoding='utf-8') - logger.info("processing " + targetname + "... ", nonl=1) + logger.info(__("processing %s..."), targetname, nonl=1) doctree = self.assemble_doctree( docname, toctree_only, appendices=(self.config.texinfo_appendices or [])) - logger.info("writing... ", nonl=1) + logger.info(__("writing... "), nonl=1) self.post_process_images(doctree) docwriter = TexinfoWriter(self) settings = OptionParser( @@ -185,7 +185,7 @@ class TexinfoBuilder(Builder): settings.docname = docname doctree.settings = settings docwriter.write(doctree, destination) - logger.info("done") + logger.info(__("done")) def assemble_doctree(self, indexfile, toctree_only, appendices): # type: (unicode, bool, List[unicode]) -> nodes.Node @@ -212,7 +212,7 @@ class TexinfoBuilder(Builder): appendix['docname'] = docname largetree.append(appendix) logger.info('') - logger.info("resolving references...") + logger.info(__("resolving references...")) self.env.resolve_references(largetree, indexfile, self) # TODO: add support for external :ref:s for pendingnode in largetree.traverse(addnodes.pending_xref): @@ -234,7 +234,7 @@ class TexinfoBuilder(Builder): # type: () -> None self.copy_image_files() - logger.info(bold('copying Texinfo support files... '), nonl=True) + logger.info(bold(__('copying Texinfo support files... ')), nonl=True) # copy Makefile fn = path.join(self.outdir, 'Makefile') logger.info(fn, nonl=1) @@ -242,14 +242,14 @@ class TexinfoBuilder(Builder): with open(fn, 'w') as mkfile: mkfile.write(TEXINFO_MAKEFILE) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", fn, err) - logger.info(' done') + logger.warning(__("error writing file %s: %s"), fn, err) + logger.info(__(' done')) def copy_image_files(self): # type: () -> None if self.images: stringify_func = ImageAdapter(self.app.env).get_original_image_uri - for src in status_iterator(self.images, 'copying images... ', "brown", + for src in status_iterator(self.images, __('copying images... '), "brown", len(self.images), self.app.verbosity, stringify_func=stringify_func): dest = self.images[src] @@ -257,7 +257,7 @@ class TexinfoBuilder(Builder): copy_asset_file(path.join(self.srcdir, src), path.join(self.outdir, dest)) except Exception as err: - logger.warning('cannot copy image file %r: %s', + logger.warning(__('cannot copy image file %r: %s'), path.join(self.srcdir, src), err) diff --git a/sphinx/builders/text.py b/sphinx/builders/text.py index babb6ccee..81209d165 100644 --- a/sphinx/builders/text.py +++ b/sphinx/builders/text.py @@ -15,6 +15,7 @@ from os import path from docutils.io import StringOutput from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import ensuredir, os_path from sphinx.writers.text import TextWriter, TextTranslator @@ -31,7 +32,7 @@ logger = logging.getLogger(__name__) class TextBuilder(Builder): name = 'text' format = 'text' - epilog = 'The text files are in %(outdir)s.' + epilog = __('The text files are in %(outdir)s.') out_suffix = '.txt' allow_parallel = True @@ -84,7 +85,7 @@ class TextBuilder(Builder): with codecs.open(outfilename, 'w', 'utf-8') as f: # type: ignore f.write(self.writer.output) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", outfilename, err) + logger.warning(__("error writing file %s: %s"), outfilename, err) def finish(self): # type: () -> None diff --git a/sphinx/builders/xml.py b/sphinx/builders/xml.py index 80d7723aa..6198532c9 100644 --- a/sphinx/builders/xml.py +++ b/sphinx/builders/xml.py @@ -17,6 +17,7 @@ from docutils.io import StringOutput from docutils.writers.docutils_xml import XMLTranslator from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import ensuredir, os_path from sphinx.writers.xml import XMLWriter, PseudoXMLWriter @@ -35,7 +36,7 @@ class XMLBuilder(Builder): """ name = 'xml' format = 'xml' - epilog = 'The XML files are in %(outdir)s.' + epilog = __('The XML files are in %(outdir)s.') out_suffix = '.xml' allow_parallel = True @@ -97,7 +98,7 @@ class XMLBuilder(Builder): with codecs.open(outfilename, 'w', 'utf-8') as f: # type: ignore f.write(self.writer.output) except (IOError, OSError) as err: - logger.warning("error writing file %s: %s", outfilename, err) + logger.warning(__("error writing file %s: %s"), outfilename, err) def finish(self): # type: () -> None @@ -110,7 +111,7 @@ class PseudoXMLBuilder(XMLBuilder): """ name = 'pseudoxml' format = 'pseudoxml' - epilog = 'The pseudo-XML files are in %(outdir)s.' + epilog = __('The pseudo-XML files are in %(outdir)s.') out_suffix = '.pseudoxml' diff --git a/sphinx/cmd/build.py b/sphinx/cmd/build.py index c0c31ae67..865e272cf 100644 --- a/sphinx/cmd/build.py +++ b/sphinx/cmd/build.py @@ -8,30 +8,308 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ +from __future__ import print_function +import argparse +import locale +import multiprocessing +import os import sys +import traceback + +from docutils.utils import SystemMessage +from six import text_type, binary_type + +import sphinx.locale +from sphinx import __display_version__, package_dir +from sphinx.application import Sphinx +from sphinx.errors import SphinxError +from sphinx.locale import __ +from sphinx.util import Tee, format_exception_cut_frames, save_traceback +from sphinx.util.console import red, nocolor, color_terminal # type: ignore +from sphinx.util.docutils import docutils_namespace, patch_docutils +from sphinx.util.pycompat import terminal_safe if False: # For type annotation - from typing import List # NOQA + from typing import Any, IO, List, Union # NOQA -def build_main(argv=sys.argv[1:]): - # type: (List[str]) -> int - """Sphinx build "main" command-line entry.""" - from sphinx import cmdline - return cmdline.main(argv) # type: ignore +def handle_exception(app, args, exception, stderr=sys.stderr): + # type: (Sphinx, Any, Union[Exception, KeyboardInterrupt], IO) -> None + if args.pdb: + import pdb + print(red(__('Exception occurred while building, starting debugger:')), + file=stderr) + traceback.print_exc() + pdb.post_mortem(sys.exc_info()[2]) + else: + print(file=stderr) + if args.verbosity or args.traceback: + traceback.print_exc(None, stderr) + print(file=stderr) + if isinstance(exception, KeyboardInterrupt): + print(__('interrupted!'), file=stderr) + elif isinstance(exception, SystemMessage): + print(red(__('reST markup error:')), file=stderr) + print(terminal_safe(exception.args[0]), file=stderr) + elif isinstance(exception, SphinxError): + print(red('%s:' % exception.category), file=stderr) + print(terminal_safe(text_type(exception)), file=stderr) + elif isinstance(exception, UnicodeError): + print(red(__('Encoding error:')), file=stderr) + print(terminal_safe(text_type(exception)), file=stderr) + tbpath = save_traceback(app) + print(red(__('The full traceback has been saved in %s, if you want ' + 'to report the issue to the developers.') % tbpath), + file=stderr) + elif isinstance(exception, RuntimeError) and 'recursion depth' in str(exception): + print(red(__('Recursion error:')), file=stderr) + print(terminal_safe(text_type(exception)), file=stderr) + print(file=stderr) + print(__('This can happen with very large or deeply nested source ' + 'files. You can carefully increase the default Python ' + 'recursion limit of 1000 in conf.py with e.g.:'), file=stderr) + print(__(' import sys; sys.setrecursionlimit(1500)'), file=stderr) + else: + print(red(__('Exception occurred:')), file=stderr) + print(format_exception_cut_frames().rstrip(), file=stderr) + tbpath = save_traceback(app) + print(red(__('The full traceback has been saved in %s, if you ' + 'want to report the issue to the developers.') % tbpath), + file=stderr) + print(__('Please also report this if it was a user error, so ' + 'that a better error message can be provided next time.'), + file=stderr) + print(__('A bug report can be filed in the tracker at ' + '<https://github.com/sphinx-doc/sphinx/issues>. Thanks!'), + file=stderr) + + +def jobs_argument(value): + # type: (str) -> int + """ + Special type to handle 'auto' flags passed to 'sphinx-build' via -j flag. Can + be expanded to handle other special scaling requests, such as setting job count + to cpu_count. + """ + if value == 'auto': + return multiprocessing.cpu_count() + else: + jobs = int(value) + if jobs <= 0: + raise argparse.ArgumentTypeError(__('job number should be a positive number')) + else: + return jobs + + +def get_parser(): + # type: () -> argparse.ArgumentParser + parser = argparse.ArgumentParser( + usage='usage: %(prog)s [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]', + epilog=__('For more information, visit <http://sphinx-doc.org/>.'), + description=__(""" +Generate documentation from source files. +sphinx-build generates documentation from the files in SOURCEDIR and places it +in OUTPUTDIR. It looks for 'conf.py' in SOURCEDIR for the configuration +settings. The 'sphinx-quickstart' tool may be used to generate template files, +including 'conf.py' -def make_main(argv=sys.argv[1:]): - # type: (List[str]) -> int +sphinx-build can create documentation in different formats. A format is +selected by specifying the builder name on the command line; it defaults to +HTML. Builders can also perform other tasks related to documentation +processing. + +By default, everything that is outdated is built. Output only for selected +files can be built by specifying individual filenames. +""")) + + parser.add_argument('--version', action='version', dest='show_version', + version='%%(prog)s %s' % __display_version__) + + parser.add_argument('sourcedir', + help=__('path to documentation source files')) + parser.add_argument('outputdir', + help=__('path to output directory')) + parser.add_argument('filenames', nargs='*', + help=__('a list of specific files to rebuild. Ignored ' + 'if -a is specified')) + + group = parser.add_argument_group(__('general options')) + group.add_argument('-b', metavar='BUILDER', dest='builder', + default='html', + help=__('builder to use (default: html)')) + group.add_argument('-a', action='store_true', dest='force_all', + help=__('write all files (default: only write new and ' + 'changed files)')) + group.add_argument('-E', action='store_true', dest='freshenv', + help=__('don\'t use a saved environment, always read ' + 'all files')) + group.add_argument('-d', metavar='PATH', dest='doctreedir', + help=__('path for the cached environment and doctree ' + 'files (default: OUTPUTDIR/.doctrees)')) + group.add_argument('-j', metavar='N', default=1, type=jobs_argument, dest='jobs', + help=__('build in parallel with N processes where ' + 'possible (special value "auto" will set N to cpu-count)')) + group = parser.add_argument_group('build configuration options') + group.add_argument('-c', metavar='PATH', dest='confdir', + help=__('path where configuration file (conf.py) is ' + 'located (default: same as SOURCEDIR)')) + group.add_argument('-C', action='store_true', dest='noconfig', + help=__('use no config file at all, only -D options')) + group.add_argument('-D', metavar='setting=value', action='append', + dest='define', default=[], + help=__('override a setting in configuration file')) + group.add_argument('-A', metavar='name=value', action='append', + dest='htmldefine', default=[], + help=__('pass a value into HTML templates')) + group.add_argument('-t', metavar='TAG', action='append', + dest='tags', default=[], + help=__('define tag: include "only" blocks with TAG')) + group.add_argument('-n', action='store_true', dest='nitpicky', + help=__('nit-picky mode, warn about all missing ' + 'references')) + + group = parser.add_argument_group(__('console output options')) + group.add_argument('-v', action='count', dest='verbosity', default=0, + help=__('increase verbosity (can be repeated)')) + group.add_argument('-q', action='store_true', dest='quiet', + help=__('no output on stdout, just warnings on stderr')) + group.add_argument('-Q', action='store_true', dest='really_quiet', + help=__('no output at all, not even warnings')) + group.add_argument('--color', action='store_const', const='yes', + default='auto', + help=__('do emit colored output (default: auto-detect)')) + group.add_argument('-N', '--no-color', dest='color', action='store_const', + const='no', + help=__('do not emit colored output (default: ' + 'auto-detect)')) + group.add_argument('-w', metavar='FILE', dest='warnfile', + help=__('write warnings (and errors) to given file')) + group.add_argument('-W', action='store_true', dest='warningiserror', + help=__('turn warnings into errors')) + group.add_argument('-T', action='store_true', dest='traceback', + help=__('show full traceback on exception')) + group.add_argument('-P', action='store_true', dest='pdb', + help=__('run Pdb on exception')) + + return parser + + +def make_main(argv=sys.argv[1:]): # type: ignore + # type: (List[unicode]) -> int """Sphinx build "make mode" entry.""" from sphinx import make_mode - return make_mode.run_make_mode(argv[1:]) # type: ignore + return make_mode.run_make_mode(argv[1:]) + + +def build_main(argv=sys.argv[1:]): # type: ignore + # type: (List[unicode]) -> int + """Sphinx build "main" command-line entry.""" + + parser = get_parser() + args = parser.parse_args(argv) + + if args.noconfig: + args.confdir = None + elif not args.confdir: + args.confdir = args.sourcedir + + if not args.doctreedir: + args.doctreedir = os.path.join(args.sourcedir, '.doctrees') + + # handle remaining filename arguments + filenames = args.filenames + missing_files = [] + for filename in filenames: + if not os.path.isfile(filename): + missing_files.append(filename) + if missing_files: + parser.error(__('cannot find files %r') % missing_files) + + # likely encoding used for command-line arguments + try: + locale = __import__('locale') # due to submodule of the same name + likely_encoding = locale.getpreferredencoding() + except Exception: + likely_encoding = None + + if args.force_all and filenames: + parser.error(__('cannot combine -a option and filenames')) + + if args.color == 'no' or (args.color == 'auto' and not color_terminal()): + nocolor() + + status = sys.stdout + warning = sys.stderr + error = sys.stderr + + if args.quiet: + status = None + + if args.really_quiet: + status = warning = None + + if warning and args.warnfile: + try: + warnfp = open(args.warnfile, 'w') + except Exception as exc: + parser.error(__('cannot open warning file %r: %s') % ( + args.warnfile, exc)) + warning = Tee(warning, warnfp) # type: ignore + error = warning + + confoverrides = {} + for val in args.define: + try: + key, val = val.split('=', 1) + except ValueError: + parser.error(__('-D option argument must be in the form name=value')) + if likely_encoding and isinstance(val, binary_type): + try: + val = val.decode(likely_encoding) + except UnicodeError: + pass + confoverrides[key] = val + + for val in args.htmldefine: + try: + key, val = val.split('=') + except ValueError: + parser.error(__('-A option argument must be in the form name=value')) + try: + val = int(val) + except ValueError: + if likely_encoding and isinstance(val, binary_type): + try: + val = val.decode(likely_encoding) + except UnicodeError: + pass + confoverrides['html_context.%s' % key] = val + + if args.nitpicky: + confoverrides['nitpicky'] = True + + app = None + try: + with patch_docutils(), docutils_namespace(): + app = Sphinx(args.sourcedir, args.confdir, args.outputdir, + args.doctreedir, args.builder, confoverrides, status, + warning, args.freshenv, args.warningiserror, + args.tags, args.verbosity, args.jobs) + app.build(args.force_all, filenames) + return app.statuscode + except (Exception, KeyboardInterrupt) as exc: + handle_exception(app, args, exc, error) + return 2 + +def main(argv=sys.argv[1:]): # type: ignore + # type: (List[unicode]) -> int + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') -def main(argv=sys.argv[1:]): - # type: (List[str]) -> int if sys.argv[1:2] == ['-M']: return make_main(argv) else: @@ -39,4 +317,4 @@ def main(argv=sys.argv[1:]): if __name__ == '__main__': - sys.exit(main(sys.argv[1:])) + sys.exit(main(sys.argv[1:])) # type: ignore diff --git a/sphinx/cmd/quickstart.py b/sphinx/cmd/quickstart.py index 68718eeaf..d6c7fa22a 100644 --- a/sphinx/cmd/quickstart.py +++ b/sphinx/cmd/quickstart.py @@ -12,6 +12,7 @@ from __future__ import absolute_import from __future__ import print_function import argparse +import locale import os import re import sys @@ -35,7 +36,9 @@ from six import PY2, PY3, text_type, binary_type from six.moves import input from six.moves.urllib.parse import quote as urlquote +import sphinx.locale from sphinx import __display_version__, package_dir +from sphinx.locale import __ from sphinx.util import texescape from sphinx.util.console import ( # type: ignore purple, bold, red, turquoise, nocolor, color_terminal @@ -50,18 +53,18 @@ if False: TERM_ENCODING = getattr(sys.stdin, 'encoding', None) EXTENSIONS = OrderedDict([ - ('autodoc', 'automatically insert docstrings from modules'), - ('doctest', 'automatically test code snippets in doctest blocks'), - ('intersphinx', 'link between Sphinx documentation of different projects'), - ('todo', 'write "todo" entries that can be shown or hidden on build'), - ('coverage', 'checks for documentation coverage'), - ('imgmath', 'include math, rendered as PNG or SVG images'), - ('mathjax', 'include math, rendered in the browser by MathJax'), - ('ifconfig', 'conditional inclusion of content based on config values'), + ('autodoc', __('automatically insert docstrings from modules')), + ('doctest', __('automatically test code snippets in doctest blocks')), + ('intersphinx', __('link between Sphinx documentation of different projects')), + ('todo', __('write "todo" entries that can be shown or hidden on build')), + ('coverage', __('checks for documentation coverage')), + ('imgmath', __('include math, rendered as PNG or SVG images')), + ('mathjax', __('include math, rendered in the browser by MathJax')), + ('ifconfig', __('conditional inclusion of content based on config values')), ('viewcode', - 'include links to the source code of documented Python objects'), + __('include links to the source code of documented Python objects')), ('githubpages', - 'create .nojekyll file to publish the document on GitHub pages'), + __('create .nojekyll file to publish the document on GitHub pages')), ]) DEFAULTS = { @@ -94,7 +97,7 @@ def is_path(x): # type: (unicode) -> unicode x = path.expanduser(x) if path.exists(x) and not path.isdir(x): - raise ValidationError("Please enter a valid path name.") + raise ValidationError(__("Please enter a valid path name.")) return x @@ -106,7 +109,7 @@ def allow_empty(x): def nonempty(x): # type: (unicode) -> unicode if not x: - raise ValidationError("Please enter some text.") + raise ValidationError(__("Please enter some text.")) return x @@ -115,7 +118,7 @@ def choice(*l): def val(x): # type: (unicode) -> unicode if x not in l: - raise ValidationError('Please enter one of %s.' % ', '.join(l)) + raise ValidationError(__('Please enter one of %s.') % ', '.join(l)) return x return val @@ -123,15 +126,15 @@ def choice(*l): def boolean(x): # type: (unicode) -> bool if x.upper() not in ('Y', 'YES', 'N', 'NO'): - raise ValidationError("Please enter either 'y' or 'n'.") + raise ValidationError(__("Please enter either 'y' or 'n'.")) return x.upper() in ('Y', 'YES') def suffix(x): # type: (unicode) -> unicode if not (x[0:1] == '.' and len(x) > 1): - raise ValidationError("Please enter a file suffix, " - "e.g. '.rst' or '.txt'.") + raise ValidationError(__("Please enter a file suffix, " + "e.g. '.rst' or '.txt'.")) return x @@ -153,9 +156,9 @@ def term_decode(text): if text.decode('ascii', 'replace').encode('ascii', 'replace') == text: return text.decode('ascii') - print(turquoise('* Note: non-ASCII characters entered ' - 'and terminal encoding unknown -- assuming ' - 'UTF-8 or Latin-1.')) + print(turquoise(__('* Note: non-ASCII characters entered ' + 'and terminal encoding unknown -- assuming ' + 'UTF-8 or Latin-1.'))) try: return text.decode('utf-8') except UnicodeDecodeError: @@ -176,9 +179,9 @@ def do_prompt(text, default=None, validator=nonempty): if TERM_ENCODING: prompt = prompt.encode(TERM_ENCODING) else: - print(turquoise('* Note: non-ASCII default value provided ' - 'and terminal encoding unknown -- assuming ' - 'UTF-8 or Latin-1.')) + print(turquoise(__('* Note: non-ASCII default value provided ' + 'and terminal encoding unknown -- assuming ' + 'UTF-8 or Latin-1.'))) try: prompt = prompt.encode('utf-8') except UnicodeEncodeError: @@ -243,110 +246,110 @@ def ask_user(d): * batchfile: make command file """ - print(bold('Welcome to the Sphinx %s quickstart utility.') % __display_version__) - print(''' + print(bold(__('Welcome to the Sphinx %s quickstart utility.')) % __display_version__) + print(__(''' Please enter values for the following settings (just press Enter to -accept a default value, if one is given in brackets).''') +accept a default value, if one is given in brackets).''')) if 'path' in d: - print(bold(''' -Selected root path: %s''' % d['path'])) + print(bold(__(''' +Selected root path: %s''') % d['path'])) else: - print(''' -Enter the root path for documentation.''') - d['path'] = do_prompt('Root path for the documentation', '.', is_path) + print(__(''' +Enter the root path for documentation.''')) + d['path'] = do_prompt(__('Root path for the documentation'), '.', is_path) while path.isfile(path.join(d['path'], 'conf.py')) or \ path.isfile(path.join(d['path'], 'source', 'conf.py')): print() - print(bold('Error: an existing conf.py has been found in the ' - 'selected root path.')) - print('sphinx-quickstart will not overwrite existing Sphinx projects.') + print(bold(__('Error: an existing conf.py has been found in the ' + 'selected root path.'))) + print(__('sphinx-quickstart will not overwrite existing Sphinx projects.')) print() - d['path'] = do_prompt('Please enter a new root path (or just Enter ' - 'to exit)', '', is_path) + d['path'] = do_prompt(__('Please enter a new root path (or just Enter ' + 'to exit)'), '', is_path) if not d['path']: sys.exit(1) if 'sep' not in d: - print(''' + print(__(''' You have two options for placing the build directory for Sphinx output. Either, you use a directory "_build" within the root path, or you separate -"source" and "build" directories within the root path.''') - d['sep'] = do_prompt('Separate source and build directories (y/n)', +"source" and "build" directories within the root path.''')) + d['sep'] = do_prompt(__('Separate source and build directories (y/n)'), 'n', boolean) if 'dot' not in d: - print(''' + print(__(''' Inside the root directory, two more directories will be created; "_templates" for custom HTML templates and "_static" for custom stylesheets and other static -files. You can enter another prefix (such as ".") to replace the underscore.''') - d['dot'] = do_prompt('Name prefix for templates and static dir', '_', ok) +files. You can enter another prefix (such as ".") to replace the underscore.''')) + d['dot'] = do_prompt(__('Name prefix for templates and static dir'), '_', ok) if 'project' not in d: - print(''' -The project name will occur in several places in the built documentation.''') - d['project'] = do_prompt('Project name') + print(__(''' +The project name will occur in several places in the built documentation.''')) + d['project'] = do_prompt(__('Project name')) if 'author' not in d: - d['author'] = do_prompt('Author name(s)') + d['author'] = do_prompt(__('Author name(s)')) if 'version' not in d: - print(''' + print(__(''' Sphinx has the notion of a "version" and a "release" for the software. Each version can have multiple releases. For example, for Python the version is something like 2.5 or 3.0, while the release is something like 2.5.1 or 3.0a1. If you don't need this dual structure, -just set both to the same value.''') - d['version'] = do_prompt('Project version', '', allow_empty) +just set both to the same value.''')) + d['version'] = do_prompt(__('Project version'), '', allow_empty) if 'release' not in d: - d['release'] = do_prompt('Project release', d['version'], allow_empty) + d['release'] = do_prompt(__('Project release'), d['version'], allow_empty) if 'language' not in d: - print(''' + print(__(''' If the documents are to be written in a language other than English, you can select a language here by its language code. Sphinx will then translate text that it generates into that language. For a list of supported codes, see -http://sphinx-doc.org/config.html#confval-language.''') - d['language'] = do_prompt('Project language', 'en') +http://sphinx-doc.org/config.html#confval-language.''')) + d['language'] = do_prompt(__('Project language'), 'en') if d['language'] == 'en': d['language'] = None if 'suffix' not in d: - print(''' + print(__(''' The file name suffix for source files. Commonly, this is either ".txt" -or ".rst". Only files with this suffix are considered documents.''') - d['suffix'] = do_prompt('Source file suffix', '.rst', suffix) +or ".rst". Only files with this suffix are considered documents.''')) + d['suffix'] = do_prompt(__('Source file suffix'), '.rst', suffix) if 'master' not in d: - print(''' + print(__(''' One document is special in that it is considered the top node of the "contents tree", that is, it is the root of the hierarchical structure of the documents. Normally, this is "index", but if your "index" -document is a custom template, you can also set this to another filename.''') - d['master'] = do_prompt('Name of your master document (without suffix)', +document is a custom template, you can also set this to another filename.''')) + d['master'] = do_prompt(__('Name of your master document (without suffix)'), 'index') while path.isfile(path.join(d['path'], d['master'] + d['suffix'])) or \ path.isfile(path.join(d['path'], 'source', d['master'] + d['suffix'])): print() - print(bold('Error: the master file %s has already been found in the ' - 'selected root path.' % (d['master'] + d['suffix']))) - print('sphinx-quickstart will not overwrite the existing file.') + print(bold(__('Error: the master file %s has already been found in the ' + 'selected root path.') % (d['master'] + d['suffix']))) + print(__('sphinx-quickstart will not overwrite the existing file.')) print() - d['master'] = do_prompt('Please enter a new file name, or rename the ' - 'existing file and press Enter', d['master']) + d['master'] = do_prompt(__('Please enter a new file name, or rename the ' + 'existing file and press Enter'), d['master']) if 'epub' not in d: - print(''' -Sphinx can also add configuration for epub output:''') - d['epub'] = do_prompt('Do you want to use the epub builder (y/n)', + print(__(''' +Sphinx can also add configuration for epub output:''')) + d['epub'] = do_prompt(__('Do you want to use the epub builder (y/n)'), 'n', boolean) if 'extensions' not in d: - print('Indicate which of the following Sphinx extensions should be ' - 'enabled:') + print(__('Indicate which of the following Sphinx extensions should be ' + 'enabled:')) d['extensions'] = [] for name, description in EXTENSIONS.items(): if do_prompt('%s: %s (y/n)' % (name, description), 'n', boolean): @@ -355,19 +358,19 @@ Sphinx can also add configuration for epub output:''') # Handle conflicting options if set(['sphinx.ext.imgmath', 'sphinx.ext.mathjax']).issubset( d['extensions']): - print('Note: imgmath and mathjax cannot be enabled at the same ' - 'time. imgmath has been deselected.') + print(__('Note: imgmath and mathjax cannot be enabled at the same ' + 'time. imgmath has been deselected.')) d['extensions'].remove('sphinx.ext.imgmath') if 'makefile' not in d: - print(''' + print(__(''' A Makefile and a Windows command file can be generated for you so that you only have to run e.g. `make html' instead of invoking sphinx-build -directly.''') - d['makefile'] = do_prompt('Create Makefile? (y/n)', 'y', boolean) +directly.''')) + d['makefile'] = do_prompt(__('Create Makefile? (y/n)'), 'y', boolean) if 'batchfile' not in d: - d['batchfile'] = do_prompt('Create Windows command file? (y/n)', + d['batchfile'] = do_prompt(__('Create Windows command file? (y/n)'), 'y', boolean) print() @@ -429,12 +432,12 @@ def generate(d, overwrite=True, silent=False, templatedir=None): # type: (unicode, unicode, unicode) -> None if overwrite or not path.isfile(fpath): if 'quiet' not in d: - print('Creating file %s.' % fpath) + print(__('Creating file %s.') % fpath) with open(fpath, 'wt', encoding='utf-8', newline=newline) as f: f.write(content) else: if 'quiet' not in d: - print('File %s already exists, skipping.' % fpath) + print(__('File %s already exists, skipping.') % fpath) conf_path = os.path.join(templatedir, 'conf.py_t') if templatedir else None if not conf_path or not path.isfile(conf_path): @@ -470,18 +473,18 @@ def generate(d, overwrite=True, silent=False, templatedir=None): if silent: return print() - print(bold('Finished: An initial directory structure has been created.')) - print(''' + print(bold(__('Finished: An initial directory structure has been created.'))) + print(__(''' You should now populate your master file %s and create other documentation -source files. ''' % masterfile + ((d['makefile'] or d['batchfile']) and '''\ +source files. ''') % masterfile + ((d['makefile'] or d['batchfile']) and __('''\ Use the Makefile to build the docs, like so: make builder -''' or '''\ +''') or __('''\ Use the sphinx-build command to build the docs, like so: sphinx-build -b builder %s %s -''' % (srcdir, builddir)) + '''\ +''') % (srcdir, builddir)) + __('''\ where "builder" is one of the supported builders, e.g. html, latex or linkcheck. -''') +''')) def valid_dir(d): @@ -518,86 +521,89 @@ def get_parser(): # type: () -> argparse.ArgumentParser parser = argparse.ArgumentParser( usage='%(prog)s [OPTIONS] <PROJECT_DIR>', - epilog="For more information, visit <http://sphinx-doc.org/>.", - description=""" + epilog=__("For more information, visit <http://sphinx-doc.org/>."), + description=__(""" Generate required files for a Sphinx project. sphinx-quickstart is an interactive tool that asks some questions about your project and then generates a complete documentation directory and sample Makefile to be used with sphinx-build. -""") +""")) parser.add_argument('-q', '--quiet', action='store_true', dest='quiet', default=False, - help='quiet mode') + help=__('quiet mode')) parser.add_argument('--version', action='version', dest='show_version', version='%%(prog)s %s' % __display_version__) parser.add_argument('path', metavar='PROJECT_DIR', default='.', nargs='?', - help='output path') + help=__('output path')) - group = parser.add_argument_group('Structure options') + group = parser.add_argument_group(__('Structure options')) group.add_argument('--sep', action='store_true', - help='if specified, separate source and build dirs') + help=__('if specified, separate source and build dirs')) group.add_argument('--dot', metavar='DOT', - help='replacement for dot in _templates etc.') + help=__('replacement for dot in _templates etc.')) - group = parser.add_argument_group('Project basic options') + group = parser.add_argument_group(__('Project basic options')) group.add_argument('-p', '--project', metavar='PROJECT', dest='project', - help='project name') + help=__('project name')) group.add_argument('-a', '--author', metavar='AUTHOR', dest='author', - help='author names') + help=__('author names')) group.add_argument('-v', metavar='VERSION', dest='version', default='', - help='version of project') + help=__('version of project')) group.add_argument('-r', '--release', metavar='RELEASE', dest='release', - help='release of project') + help=__('release of project')) group.add_argument('-l', '--language', metavar='LANGUAGE', dest='language', - help='document language') + help=__('document language')) group.add_argument('--suffix', metavar='SUFFIX', - help='source file suffix') + help=__('source file suffix')) group.add_argument('--master', metavar='MASTER', - help='master document name') + help=__('master document name')) group.add_argument('--epub', action='store_true', default=False, - help='use epub') + help=__('use epub')) - group = parser.add_argument_group('Extension options') + group = parser.add_argument_group(__('Extension options')) for ext in EXTENSIONS: group.add_argument('--ext-%s' % ext, action='append_const', const='sphinx.ext.%s' % ext, dest='extensions', - help='enable %s extension' % ext) + help=__('enable %s extension') % ext) group.add_argument('--extensions', metavar='EXTENSIONS', dest='extensions', - action='append', help='enable arbitrary extensions') + action='append', help=__('enable arbitrary extensions')) - group = parser.add_argument_group('Makefile and Batchfile creation') + group = parser.add_argument_group(__('Makefile and Batchfile creation')) group.add_argument('--makefile', action='store_true', dest='makefile', - help='create makefile') + help=__('create makefile')) group.add_argument('--no-makefile', action='store_false', dest='makefile', - help='do not create makefile') + help=__('do not create makefile')) group.add_argument('--batchfile', action='store_true', dest='batchfile', - help='create batchfile') + help=__('create batchfile')) group.add_argument('--no-batchfile', action='store_false', dest='batchfile', - help='do not create batchfile') + help=__('do not create batchfile')) group.add_argument('-m', '--use-make-mode', action='store_true', dest='make_mode', default=True, - help='use make-mode for Makefile/make.bat') + help=__('use make-mode for Makefile/make.bat')) group.add_argument('-M', '--no-use-make-mode', action='store_false', dest='make_mode', - help='do not use make-mode for Makefile/make.bat') + help=__('do not use make-mode for Makefile/make.bat')) - group = parser.add_argument_group('Project templating') + group = parser.add_argument_group(__('Project templating')) group.add_argument('-t', '--templatedir', metavar='TEMPLATEDIR', dest='templatedir', - help='template directory for template files') + help=__('template directory for template files')) group.add_argument('-d', metavar='NAME=VALUE', action='append', dest='variables', - help='define a template variable') + help=__('define a template variable')) return parser def main(argv=sys.argv[1:]): # type: (List[str]) -> int + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') + if not color_terminal(): nocolor() @@ -615,8 +621,8 @@ def main(argv=sys.argv[1:]): try: if 'quiet' in d: if not set(['project', 'author']).issubset(d): - print('''"quiet" is specified, but any of "project" or \ -"author" is not specified.''') + print(__('''"quiet" is specified, but any of "project" or \ +"author" is not specified.''')) return 1 if set(['quiet', 'project', 'author']).issubset(d): @@ -629,10 +635,10 @@ def main(argv=sys.argv[1:]): if not valid_dir(d): print() - print(bold('Error: specified path is not a directory, or sphinx' - ' files already exist.')) - print('sphinx-quickstart only generate into a empty directory.' - ' Please specify a new root path.') + print(bold(__('Error: specified path is not a directory, or sphinx' + ' files already exist.'))) + print(__('sphinx-quickstart only generate into a empty directory.' + ' Please specify a new root path.')) return 1 else: ask_user(d) @@ -658,7 +664,7 @@ def main(argv=sys.argv[1:]): name, value = variable.split('=') d[name] = value except ValueError: - print('Invalid template variable: %s' % variable) + print(__('Invalid template variable: %s') % variable) generate(d, templatedir=args.templatedir) return 0 diff --git a/sphinx/cmdline.py b/sphinx/cmdline.py index 908da68d8..10835d2e7 100644 --- a/sphinx/cmdline.py +++ b/sphinx/cmdline.py @@ -8,301 +8,45 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ +from __future__ import absolute_import from __future__ import print_function -import argparse -import multiprocessing import sys -import traceback -from os import path +import warnings -from docutils.utils import SystemMessage -from six import text_type, binary_type - -from sphinx import __display_version__ -from sphinx.application import Sphinx -from sphinx.errors import SphinxError -from sphinx.util import Tee, format_exception_cut_frames, save_traceback -from sphinx.util.console import red, nocolor, color_terminal # type: ignore -from sphinx.util.docutils import docutils_namespace, patch_docutils -from sphinx.util.osutil import abspath, fs_encoding -from sphinx.util.pycompat import terminal_safe +from sphinx.cmd import build +from sphinx.deprecation import RemovedInSphinx30Warning if False: # For type annotation + import argparse # NOQA from typing import Any, IO, List, Union # NOQA + from sphinx.application import Sphinx # NOQA def handle_exception(app, args, exception, stderr=sys.stderr): # type: (Sphinx, Any, Union[Exception, KeyboardInterrupt], IO) -> None - if args.pdb: - import pdb - print(red('Exception occurred while building, starting debugger:'), - file=stderr) - traceback.print_exc() - pdb.post_mortem(sys.exc_info()[2]) - else: - print(file=stderr) - if args.verbosity or args.traceback: - traceback.print_exc(None, stderr) - print(file=stderr) - if isinstance(exception, KeyboardInterrupt): - print('interrupted!', file=stderr) - elif isinstance(exception, SystemMessage): - print(red('reST markup error:'), file=stderr) - print(terminal_safe(exception.args[0]), file=stderr) - elif isinstance(exception, SphinxError): - print(red('%s:' % exception.category), file=stderr) - print(terminal_safe(text_type(exception)), file=stderr) - elif isinstance(exception, UnicodeError): - print(red('Encoding error:'), file=stderr) - print(terminal_safe(text_type(exception)), file=stderr) - tbpath = save_traceback(app) - print(red('The full traceback has been saved in %s, if you want ' - 'to report the issue to the developers.' % tbpath), - file=stderr) - elif isinstance(exception, RuntimeError) and 'recursion depth' in str(exception): - print(red('Recursion error:'), file=stderr) - print(terminal_safe(text_type(exception)), file=stderr) - print(file=stderr) - print('This can happen with very large or deeply nested source ' - 'files. You can carefully increase the default Python ' - 'recursion limit of 1000 in conf.py with e.g.:', file=stderr) - print(' import sys; sys.setrecursionlimit(1500)', file=stderr) - else: - print(red('Exception occurred:'), file=stderr) - print(format_exception_cut_frames().rstrip(), file=stderr) - tbpath = save_traceback(app) - print(red('The full traceback has been saved in %s, if you ' - 'want to report the issue to the developers.' % tbpath), - file=stderr) - print('Please also report this if it was a user error, so ' - 'that a better error message can be provided next time.', - file=stderr) - print('A bug report can be filed in the tracker at ' - '<https://github.com/sphinx-doc/sphinx/issues>. Thanks!', - file=stderr) + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + build.handle_exception(app, args, exception, stderr) def jobs_argument(value): # type: (str) -> int - """ - Special type to handle 'auto' flags passed to 'sphinx-build' via -j flag. Can - be expanded to handle other special scaling requests, such as setting job count - to cpu_count. - """ - if value == 'auto': - return multiprocessing.cpu_count() - else: - jobs = int(value) - if jobs <= 0: - raise argparse.ArgumentTypeError('job number should be a positive number') - else: - return jobs + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + return build.jobs_argument(value) def get_parser(): # type: () -> argparse.ArgumentParser - parser = argparse.ArgumentParser( - usage='usage: %(prog)s [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]', - epilog='For more information, visit <http://sphinx-doc.org/>.', - description=""" -Generate documentation from source files. - -sphinx-build generates documentation from the files in SOURCEDIR and places it -in OUTPUTDIR. It looks for 'conf.py' in SOURCEDIR for the configuration -settings. The 'sphinx-quickstart' tool may be used to generate template files, -including 'conf.py' - -sphinx-build can create documentation in different formats. A format is -selected by specifying the builder name on the command line; it defaults to -HTML. Builders can also perform other tasks related to documentation -processing. - -By default, everything that is outdated is built. Output only for selected -files can be built by specifying individual filenames. -""") - - parser.add_argument('--version', action='version', dest='show_version', - version='%%(prog)s %s' % __display_version__) - - parser.add_argument('sourcedir', - help='path to documentation source files') - parser.add_argument('outputdir', - help='path to output directory') - parser.add_argument('filenames', nargs='*', - help='a list of specific files to rebuild. Ignored ' - 'if -a is specified') - - group = parser.add_argument_group('general options') - group.add_argument('-b', metavar='BUILDER', dest='builder', - default='html', - help='builder to use (default: html)') - group.add_argument('-a', action='store_true', dest='force_all', - help='write all files (default: only write new and ' - 'changed files)') - group.add_argument('-E', action='store_true', dest='freshenv', - help='don\'t use a saved environment, always read ' - 'all files') - group.add_argument('-d', metavar='PATH', dest='doctreedir', - help='path for the cached environment and doctree ' - 'files (default: OUTPUTDIR/.doctrees)') - group.add_argument('-j', metavar='N', default=1, type=jobs_argument, dest='jobs', - help='build in parallel with N processes where ' - 'possible (special value "auto" will set N to cpu-count)') - group = parser.add_argument_group('build configuration options') - group.add_argument('-c', metavar='PATH', dest='confdir', - help='path where configuration file (conf.py) is ' - 'located (default: same as SOURCEDIR)') - group.add_argument('-C', action='store_true', dest='noconfig', - help='use no config file at all, only -D options') - group.add_argument('-D', metavar='setting=value', action='append', - dest='define', default=[], - help='override a setting in configuration file') - group.add_argument('-A', metavar='name=value', action='append', - dest='htmldefine', default=[], - help='pass a value into HTML templates') - group.add_argument('-t', metavar='TAG', action='append', - dest='tags', default=[], - help='define tag: include "only" blocks with TAG') - group.add_argument('-n', action='store_true', dest='nitpicky', - help='nit-picky mode, warn about all missing ' - 'references') - - group = parser.add_argument_group('console output options') - group.add_argument('-v', action='count', dest='verbosity', default=0, - help='increase verbosity (can be repeated)') - group.add_argument('-q', action='store_true', dest='quiet', - help='no output on stdout, just warnings on stderr') - group.add_argument('-Q', action='store_true', dest='really_quiet', - help='no output at all, not even warnings') - group.add_argument('--color', action='store_const', const='yes', - default='auto', - help='do emit colored output (default: auto-detect)') - group.add_argument('-N', '--no-color', dest='color', action='store_const', - const='no', - help='do not emit colored output (default: ' - 'auto-detect)') - group.add_argument('-w', metavar='FILE', dest='warnfile', - help='write warnings (and errors) to given file') - group.add_argument('-W', action='store_true', dest='warningiserror', - help='turn warnings into errors') - group.add_argument('-T', action='store_true', dest='traceback', - help='show full traceback on exception') - group.add_argument('-P', action='store_true', dest='pdb', - help='run Pdb on exception') - - return parser + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + return build.get_parser() def main(argv=sys.argv[1:]): # type: ignore # type: (List[unicode]) -> int - - parser = get_parser() - args = parser.parse_args(argv) - - # get paths (first and second positional argument) - try: - srcdir = abspath(args.sourcedir) - confdir = abspath(args.confdir or srcdir) - if args.noconfig: - confdir = None - - if not path.isdir(srcdir): - parser.error('cannot find source directory (%s)' % srcdir) - if not args.noconfig and not path.isfile(path.join(confdir, 'conf.py')): - parser.error("config directory doesn't contain a conf.py file " - "(%s)" % confdir) - - outdir = abspath(args.outputdir) - if srcdir == outdir: - parser.error('source directory and destination directory are same') - except UnicodeError: - parser.error('multibyte filename not supported on this filesystem ' - 'encoding (%r)' % fs_encoding) - - # handle remaining filename arguments - filenames = args.filenames - missing_files = [] - for filename in filenames: - if not path.isfile(filename): - missing_files.append(filename) - if missing_files: - parser.error('cannot find files %r' % missing_files) - - # likely encoding used for command-line arguments - try: - locale = __import__('locale') # due to submodule of the same name - likely_encoding = locale.getpreferredencoding() - except Exception: - likely_encoding = None - - if args.force_all and filenames: - parser.error('cannot combine -a option and filenames') - - if args.color == 'no' or (args.color == 'auto' and not color_terminal()): - nocolor() - - doctreedir = abspath(args.doctreedir or path.join(outdir, '.doctrees')) - - status = sys.stdout - warning = sys.stderr - error = sys.stderr - - if args.quiet: - status = None - - if args.really_quiet: - status = warning = None - - if warning and args.warnfile: - try: - warnfp = open(args.warnfile, 'w') - except Exception as exc: - parser.error('cannot open warning file %r: %s' % ( - args.warnfile, exc)) - warning = Tee(warning, warnfp) # type: ignore - error = warning - - confoverrides = {} - for val in args.define: - try: - key, val = val.split('=', 1) - except ValueError: - parser.error('-D option argument must be in the form name=value') - if likely_encoding and isinstance(val, binary_type): - try: - val = val.decode(likely_encoding) - except UnicodeError: - pass - confoverrides[key] = val - - for val in args.htmldefine: - try: - key, val = val.split('=') - except ValueError: - parser.error('-A option argument must be in the form name=value') - try: - val = int(val) - except ValueError: - if likely_encoding and isinstance(val, binary_type): - try: - val = val.decode(likely_encoding) - except UnicodeError: - pass - confoverrides['html_context.%s' % key] = val - - if args.nitpicky: - confoverrides['nitpicky'] = True - - app = None - try: - with patch_docutils(), docutils_namespace(): - app = Sphinx(srcdir, confdir, outdir, doctreedir, args.builder, - confoverrides, status, warning, args.freshenv, - args.warningiserror, args.tags, args.verbosity, args.jobs) - app.build(args.force_all, filenames) - return app.statuscode - except (Exception, KeyboardInterrupt) as exc: - handle_exception(app, args, exc, error) - return 2 + warnings.warn('sphinx.cmdline module is deprecated. Use sphinx.cmd.build instead.', + RemovedInSphinx30Warning) + return build.main(argv) diff --git a/sphinx/config.py b/sphinx/config.py index 1f80ab366..df4a8f184 100644 --- a/sphinx/config.py +++ b/sphinx/config.py @@ -11,13 +11,14 @@ import re import traceback +from collections import OrderedDict from os import path, getenv from typing import Any, NamedTuple, Union from six import PY2, PY3, iteritems, string_types, binary_type, text_type, integer_types from sphinx.errors import ConfigError -from sphinx.locale import l_, __ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.i18n import format_date from sphinx.util.osutil import cd @@ -26,6 +27,7 @@ from sphinx.util.pycompat import execfile_, NoneType if False: # For type annotation from typing import Any, Callable, Dict, Iterable, Iterator, List, Tuple, Union # NOQA + from sphinx.application import Sphinx # NOQA from sphinx.util.tags import Tags # NOQA logger = logging.getLogger(__name__) @@ -33,18 +35,18 @@ logger = logging.getLogger(__name__) nonascii_re = re.compile(br'[\x80-\xff]') copyright_year_re = re.compile(r'^((\d{4}-)?)(\d{4})(?=[ ,])') -CONFIG_SYNTAX_ERROR = "There is a syntax error in your configuration file: %s" +CONFIG_SYNTAX_ERROR = __("There is a syntax error in your configuration file: %s") if PY3: - CONFIG_SYNTAX_ERROR += "\nDid you change the syntax from 2.x to 3.x?" -CONFIG_ERROR = "There is a programable error in your configuration file:\n\n%s" -CONFIG_EXIT_ERROR = "The configuration file (or one of the modules it imports) " \ - "called sys.exit()" -CONFIG_ENUM_WARNING = "The config value `{name}` has to be a one of {candidates}, " \ - "but `{current}` is given." -CONFIG_PERMITTED_TYPE_WARNING = "The config value `{name}' has type `{current.__name__}', " \ - "expected to {permitted}." -CONFIG_TYPE_WARNING = "The config value `{name}' has type `{current.__name__}', " \ - "defaults to `{default.__name__}'." + CONFIG_SYNTAX_ERROR += __("\nDid you change the syntax from 2.x to 3.x?") +CONFIG_ERROR = __("There is a programable error in your configuration file:\n\n%s") +CONFIG_EXIT_ERROR = __("The configuration file (or one of the modules it imports) " + "called sys.exit()") +CONFIG_ENUM_WARNING = __("The config value `{name}` has to be a one of {candidates}, " + "but `{current}` is given.") +CONFIG_PERMITTED_TYPE_WARNING = __("The config value `{name}' has type `{current.__name__}', " + "expected to {permitted}.") +CONFIG_TYPE_WARNING = __("The config value `{name}' has type `{current.__name__}', " + "defaults to `{default.__name__}'.") if PY3: unicode = str # special alias for static typing... @@ -54,6 +56,10 @@ ConfigValue = NamedTuple('ConfigValue', [('name', str), ('rebuild', Union[bool, unicode])]) +#: represents the config value accepts any type of value. +Any = object() + + class ENUM(object): """represents the config value should be a one of candidates. @@ -78,8 +84,15 @@ if PY2: class Config(object): - """ - Configuration file abstraction. + """Configuration file abstraction. + + The config object makes the values of all config values available as + attributes. + + It is exposed via the :py:attr:`sphinx.application.Application.config` and + :py:attr:`sphinx.environment.Environment.config` attributes. For example, + to get the value of :confval:`language`, use either ``app.config.language`` + or ``env.config.language``. """ # the values are: (default, what needs to be rebuilt if changed) @@ -90,6 +103,7 @@ class Config(object): config_values = dict( # general options project = ('Python', 'env'), + author = ('unknown', 'env'), copyright = ('', 'html'), version = ('', 'env'), release = ('', 'env'), @@ -102,7 +116,7 @@ class Config(object): figure_language_filename = (u'{root}.{language}{ext}', 'env', [str]), master_doc = ('contents', 'env'), - source_suffix = (['.rst'], 'env'), + source_suffix = ({'.rst': 'restructuredtext'}, 'env', Any), source_encoding = ('utf-8-sig', 'env'), source_parsers = ({}, 'env'), exclude_patterns = ([], 'env'), @@ -130,10 +144,10 @@ class Config(object): nitpick_ignore = ([], None), numfig = (False, 'env'), numfig_secnum_depth = (1, 'env'), - numfig_format = ({'section': l_('Section %s'), - 'figure': l_('Fig. %s'), - 'table': l_('Table %s'), - 'code-block': l_('Listing %s')}, + numfig_format = ({'section': _('Section %s'), + 'figure': _('Fig. %s'), + 'table': _('Table %s'), + 'code-block': _('Listing %s')}, 'env'), tls_verify = (True, 'env'), @@ -191,7 +205,7 @@ class Config(object): # type: () -> None # check all values for deviation from the default value's type, since # that can result in TypeErrors all over the place - # NB. since config values might use l_() we have to wait with calling + # NB. since config values might use _() we have to wait with calling # this method until i18n is initialized for name in self._raw_config: if name not in self.values: @@ -201,11 +215,14 @@ class Config(object): permitted = settings[2] if len(settings) == 3 else () if hasattr(default, '__call__'): - default = default(self) # could invoke l_() + default = default(self) # could invoke _() if default is None and not permitted: continue # neither inferrable nor expliclitly permitted types current = self[name] - if isinstance(permitted, ENUM): + if permitted is Any: + # any type of value is accepted + pass + elif isinstance(permitted, ENUM): if not permitted.match(current): logger.warning(CONFIG_ENUM_WARNING.format( name=name, current=current, candidates=permitted.candidates)) @@ -235,9 +252,9 @@ class Config(object): # since that can result in UnicodeErrors all over the place for name, value in iteritems(self._raw_config): if isinstance(value, binary_type) and nonascii_re.search(value): - logger.warning('the config value %r is set to a string with non-ASCII ' - 'characters; this can lead to Unicode errors occurring. ' - 'Please use Unicode strings, e.g. %r.', name, u'Content') + logger.warning(__('the config value %r is set to a string with non-ASCII ' + 'characters; this can lead to Unicode errors occurring. ' + 'Please use Unicode strings, e.g. %r.'), name, u'Content') def convert_overrides(self, name, value): # type: (unicode, Any) -> Any @@ -245,7 +262,9 @@ class Config(object): return value else: defvalue = self.values[name][0] - if isinstance(defvalue, dict): + if self.values[name][-1] == Any: + return value + elif isinstance(defvalue, dict): raise ValueError(__('cannot override dictionary config setting %r, ' 'ignoring (use %r to set individual elements)') % (name, name + '.key=value')) @@ -302,8 +321,6 @@ class Config(object): for name in config: if name in self.values: self.__dict__[name] = config[name] # type: ignore - if isinstance(self.source_suffix, string_types): # type: ignore - self.source_suffix = [self.source_suffix] # type: ignore def __getattr__(self, name): # type: (unicode) -> Any @@ -346,3 +363,40 @@ class Config(object): if isinstance(rebuild, string_types): rebuild = [rebuild] return (value for value in self if value.rebuild in rebuild) # type: ignore + + +def convert_source_suffix(app, config): + # type: (Sphinx, Config) -> None + """This converts old styled source_suffix to new styled one. + + * old style: str or list + * new style: a dict which maps from fileext to filetype + """ + source_suffix = config.source_suffix + if isinstance(source_suffix, string_types): + # if str, considers as default filetype (None) + # + # The default filetype is determined on later step. + # By default, it is considered as restructuredtext. + config.source_suffix = OrderedDict({source_suffix: None}) # type: ignore + elif isinstance(source_suffix, (list, tuple)): + # if list, considers as all of them are default filetype + config.source_suffix = OrderedDict([(s, None) for s in source_suffix]) # type: ignore # NOQA + elif isinstance(source_suffix, dict): + # if dict, convert it to OrderedDict + config.source_suffix = OrderedDict(config.source_suffix) # type: ignore + else: + logger.warning(__("The config value `source_suffix' expected to " + "a string, list of strings or dictionary. " + "But `%r' is given." % source_suffix)) + + +def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] + app.connect('config-inited', convert_source_suffix) + + return { + 'version': 'builtin', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/deprecation.py b/sphinx/deprecation.py index e28e0f916..1b0e862f0 100644 --- a/sphinx/deprecation.py +++ b/sphinx/deprecation.py @@ -9,6 +9,13 @@ :license: BSD, see LICENSE for details. """ +import warnings + +if False: + # For type annotation + # note: Don't use typing.TYPE_CHECK here (for py27 and py34). + from typing import Any, Dict, Type # NOQA + class RemovedInSphinx18Warning(DeprecationWarning): pass @@ -22,4 +29,43 @@ class RemovedInSphinx20Warning(PendingDeprecationWarning): pass +class RemovedInSphinx30Warning(PendingDeprecationWarning): + pass + + RemovedInNextVersionWarning = RemovedInSphinx18Warning + + +class DeprecatedDict(dict): + """A deprecated dict which warns on each access.""" + + def __init__(self, data, message, warning): + # type: (Dict, str, Type[Warning]) -> None + self.message = message + self.warning = warning + super(DeprecatedDict, self).__init__(data) + + def __setitem__(self, key, value): + # type: (unicode, Any) -> None + warnings.warn(self.message, self.warning) + super(DeprecatedDict, self).__setitem__(key, value) + + def setdefault(self, key, default=None): + # type: (unicode, Any) -> None + warnings.warn(self.message, self.warning) + return super(DeprecatedDict, self).setdefault(key, default) + + def __getitem__(self, key): + # type: (unicode) -> None + warnings.warn(self.message, self.warning) + return super(DeprecatedDict, self).__getitem__(key) + + def get(self, key, default=None): + # type: (unicode, Any) -> None + warnings.warn(self.message, self.warning) + return super(DeprecatedDict, self).get(key, default) + + def update(self, other=None): # type: ignore + # type: (Dict) -> None + warnings.warn(self.message, self.warning) + super(DeprecatedDict, self).update(other) diff --git a/sphinx/directives/code.py b/sphinx/directives/code.py index ae67cf4c6..8690fca0f 100644 --- a/sphinx/directives/code.py +++ b/sphinx/directives/code.py @@ -14,6 +14,7 @@ from difflib import unified_diff from docutils import nodes from docutils.parsers.rst import Directive, directives from docutils.statemachine import ViewList +from six import text_type from sphinx import addnodes from sphinx.locale import __ @@ -126,7 +127,7 @@ class CodeBlock(Directive): nlines = len(self.content) hl_lines = parselinenos(linespec, nlines) if any(i >= nlines for i in hl_lines): - logger.warning('line number spec is out of range(1-%d): %r' % + logger.warning(__('line number spec is out of range(1-%d): %r') % (nlines, self.options['emphasize-lines']), location=location) @@ -159,7 +160,7 @@ class CodeBlock(Directive): try: literal = container_wrapper(self, literal, caption) except ValueError as exc: - return [document.reporter.warning(str(exc), line=self.lineno)] + return [document.reporter.warning(text_type(exc), line=self.lineno)] # literal will be note_implicit_target that is linked from caption and numref. # when options['name'] is provided, it should be primary ID. @@ -268,7 +269,7 @@ class LiteralIncludeReader(object): if linespec: linelist = parselinenos(linespec, len(lines)) if any(i >= len(lines) for i in linelist): - logger.warning('line number spec is out of range(1-%d): %r' % + logger.warning(__('line number spec is out of range(1-%d): %r') % (len(lines), linespec), location=location) if 'lineno-match' in self.options: @@ -364,6 +365,7 @@ class LiteralIncludeReader(object): return lines def dedent_filter(self, lines, location=None): + # type: (List[unicode], Any) -> List[unicode] if 'dedent' in self.options: return dedent_lines(lines, self.options.get('dedent'), location=location) else: @@ -439,7 +441,7 @@ class LiteralInclude(Directive): if 'emphasize-lines' in self.options: hl_lines = parselinenos(self.options['emphasize-lines'], lines) if any(i >= lines for i in hl_lines): - logger.warning('line number spec is out of range(1-%d): %r' % + logger.warning(__('line number spec is out of range(1-%d): %r') % (lines, self.options['emphasize-lines']), location=location) extra_args['hl_lines'] = [x + 1 for x in hl_lines if x < lines] @@ -455,7 +457,7 @@ class LiteralInclude(Directive): return [retnode] except Exception as exc: - return [document.reporter.warning(str(exc), line=self.lineno)] + return [document.reporter.warning(text_type(exc), line=self.lineno)] def setup(app): diff --git a/sphinx/directives/other.py b/sphinx/directives/other.py index e1ba1a279..252e6522a 100644 --- a/sphinx/directives/other.py +++ b/sphinx/directives/other.py @@ -14,8 +14,9 @@ from docutils.parsers.rst.directives.misc import Class from docutils.parsers.rst.directives.misc import Include as BaseInclude from six.moves import range -from sphinx import addnodes -from sphinx.locale import versionlabels, _ +from sphinx import addnodes, locale +from sphinx.deprecation import DeprecatedDict, RemovedInSphinx30Warning +from sphinx.locale import _ from sphinx.util import url_re, docname_join from sphinx.util.matching import patfilter from sphinx.util.nodes import explicit_title_re, set_source_info, \ @@ -27,6 +28,20 @@ if False: from sphinx.application import Sphinx # NOQA +versionlabels = { + 'versionadded': _('New in version %s'), + 'versionchanged': _('Changed in version %s'), + 'deprecated': _('Deprecated since version %s'), +} # type: Dict[unicode, unicode] + +locale.versionlabels = DeprecatedDict( + versionlabels, + 'sphinx.locale.versionlabels is deprecated. ' + 'Please use sphinx.directives.other.versionlabels instead.', + RemovedInSphinx30Warning +) + + def int_or_nothing(argument): # type: (unicode) -> int if not argument: diff --git a/sphinx/directives/patches.py b/sphinx/directives/patches.py index c97340a81..14f0ca17e 100644 --- a/sphinx/directives/patches.py +++ b/sphinx/directives/patches.py @@ -16,7 +16,7 @@ from sphinx.util.nodes import set_source_info if False: # For type annotation - from typing import Dict, List # NOQA + from typing import Dict, List, Tuple # NOQA from sphinx.application import Sphinx # NOQA @@ -69,6 +69,7 @@ class RSTTable(tables.RSTTable): Only for docutils-0.13 or older version.""" def make_title(self): + # type: () -> Tuple[nodes.Node, unicode] title, message = tables.RSTTable.make_title(self) if title: set_source_info(self, title) @@ -82,6 +83,7 @@ class CSVTable(tables.CSVTable): Only for docutils-0.13 or older version.""" def make_title(self): + # type: () -> Tuple[nodes.Node, unicode] title, message = tables.CSVTable.make_title(self) if title: set_source_info(self, title) @@ -95,6 +97,7 @@ class ListTable(tables.ListTable): Only for docutils-0.13 or older version.""" def make_title(self): + # type: () -> Tuple[nodes.Node, unicode] title, message = tables.ListTable.make_title(self) if title: set_source_info(self, title) diff --git a/sphinx/domains/__init__.py b/sphinx/domains/__init__.py index c68d37472..009d3bfdb 100644 --- a/sphinx/domains/__init__.py +++ b/sphinx/domains/__init__.py @@ -281,11 +281,11 @@ class Domain(object): cross-reference. If no resolution can be found, None can be returned; the xref node will - then given to the 'missing-reference' event, and if that yields no + then given to the :event:`missing-reference` event, and if that yields no resolution, replaced by *contnode*. The method can also raise :exc:`sphinx.environment.NoUri` to suppress - the 'missing-reference' event being emitted. + the :event:`missing-reference` event being emitted. """ pass diff --git a/sphinx/domains/c.py b/sphinx/domains/c.py index f0c81db7b..50e9b2e5e 100644 --- a/sphinx/domains/c.py +++ b/sphinx/domains/c.py @@ -17,7 +17,7 @@ from docutils import nodes from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import l_, _ +from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util.docfields import Field, TypedField from sphinx.util.nodes import make_refnode @@ -62,12 +62,12 @@ class CObject(ObjectDescription): """ doc_field_types = [ - TypedField('parameter', label=l_('Parameters'), + TypedField('parameter', label=_('Parameters'), names=('param', 'parameter', 'arg', 'argument'), typerolename='type', typenames=('type',)), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), - Field('returntype', label=l_('Return type'), has_arg=False, + Field('returntype', label=_('Return type'), has_arg=False, names=('rtype',)), ] @@ -254,11 +254,11 @@ class CDomain(Domain): name = 'c' label = 'C' object_types = { - 'function': ObjType(l_('function'), 'func'), - 'member': ObjType(l_('member'), 'member'), - 'macro': ObjType(l_('macro'), 'macro'), - 'type': ObjType(l_('type'), 'type'), - 'var': ObjType(l_('variable'), 'data'), + 'function': ObjType(_('function'), 'func'), + 'member': ObjType(_('member'), 'member'), + 'macro': ObjType(_('macro'), 'macro'), + 'type': ObjType(_('type'), 'type'), + 'var': ObjType(_('variable'), 'data'), } directives = { @@ -330,6 +330,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/cpp.py b/sphinx/domains/cpp.py index 34385f9e6..f4234b0db 100644 --- a/sphinx/domains/cpp.py +++ b/sphinx/domains/cpp.py @@ -20,7 +20,7 @@ from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType from sphinx.environment import NoUri -from sphinx.locale import l_, _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import logging from sphinx.util.docfields import Field, GroupedField @@ -48,7 +48,7 @@ logger = logging.getLogger(__name__) It is not the actual old code, but a replication of the behaviour. - v2: 1.3 <= version < now Standardised mangling scheme from - http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling + https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling though not completely implemented. All versions are generated and attached to elements. The newest is used for the index. All of the versions should work as permalinks. @@ -615,6 +615,7 @@ class ASTBase(UnicodeMixin): raise NotImplementedError(repr(self)) def __repr__(self): + # type: () -> str return '<%s %s>' % (self.__class__.__name__, self) @@ -3655,8 +3656,8 @@ class Symbol(object): ourChild._fill_empty(otherChild.declaration, otherChild.docname) elif ourChild.docname != otherChild.docname: name = text_type(ourChild.declaration) - msg = "Duplicate declaration, also defined in '%s'.\n" - msg += "Declaration is '%s'." + msg = __("Duplicate declaration, also defined in '%s'.\n" + "Declaration is '%s'.") msg = msg % (ourChild.docname, name) logger.warning(msg, location=otherChild.docname) else: @@ -5562,16 +5563,16 @@ class CPPObject(ObjectDescription): """Description of a C++ language object.""" doc_field_types = [ - GroupedField('parameter', label=l_('Parameters'), + GroupedField('parameter', label=_('Parameters'), names=('param', 'parameter', 'arg', 'argument'), can_collapse=True), - GroupedField('template parameter', label=l_('Template Parameters'), + GroupedField('template parameter', label=_('Template Parameters'), names=('tparam', 'template parameter'), can_collapse=True), - GroupedField('exceptions', label=l_('Throws'), rolename='cpp:class', + GroupedField('exceptions', label=_('Throws'), rolename='cpp:class', names=('throws', 'throw', 'exception'), can_collapse=True), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), ] @@ -5984,13 +5985,13 @@ class CPPDomain(Domain): name = 'cpp' label = 'C++' object_types = { - 'class': ObjType(l_('class'), 'class', 'type', 'identifier'), - 'function': ObjType(l_('function'), 'function', 'func', 'type', 'identifier'), - 'member': ObjType(l_('member'), 'member', 'var'), - 'type': ObjType(l_('type'), 'type', 'identifier'), - 'concept': ObjType(l_('concept'), 'concept', 'identifier'), - 'enum': ObjType(l_('enum'), 'enum', 'type', 'identifier'), - 'enumerator': ObjType(l_('enumerator'), 'enumerator') + 'class': ObjType(_('class'), 'class', 'type', 'identifier'), + 'function': ObjType(_('function'), 'function', 'func', 'type', 'identifier'), + 'member': ObjType(_('member'), 'member', 'var'), + 'type': ObjType(_('type'), 'type', 'identifier'), + 'concept': ObjType(_('concept'), 'concept', 'identifier'), + 'enum': ObjType(_('enum'), 'enum', 'type', 'identifier'), + 'enumerator': ObjType(_('enumerator'), 'enumerator') } directives = { @@ -6052,8 +6053,8 @@ class CPPDomain(Domain): for name, docname in otherdata['names'].items(): if docname in docnames: if name in ourNames: - msg = "Duplicate declaration, also defined in '%s'.\n" - msg += "Name of declaration is '%s'." + msg = __("Duplicate declaration, also defined in '%s'.\n" + "Name of declaration is '%s'.") msg = msg % (ourNames[name], name) logger.warning(msg, docname) else: @@ -6223,6 +6224,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/javascript.py b/sphinx/domains/javascript.py index 734969fc1..0ea6114d6 100644 --- a/sphinx/domains/javascript.py +++ b/sphinx/domains/javascript.py @@ -16,7 +16,7 @@ from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType from sphinx.domains.python import _pseudo_parse_arglist -from sphinx.locale import l_, _ +from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util.docfields import Field, GroupedField, TypedField from sphinx.util.nodes import make_refnode @@ -201,15 +201,15 @@ class JSCallable(JSObject): has_arguments = True doc_field_types = [ - TypedField('arguments', label=l_('Arguments'), + TypedField('arguments', label=_('Arguments'), names=('argument', 'arg', 'parameter', 'param'), typerolename='func', typenames=('paramtype', 'type')), - GroupedField('errors', label=l_('Throws'), rolename='err', + GroupedField('errors', label=_('Throws'), rolename='err', names=('throws', ), can_collapse=True), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), - Field('returntype', label=l_('Return type'), has_arg=False, + Field('returntype', label=_('Return type'), has_arg=False, names=('rtype',)), ] @@ -296,12 +296,12 @@ class JavaScriptDomain(Domain): label = 'JavaScript' # if you add a new object type make sure to edit JSObject.get_index_string object_types = { - 'function': ObjType(l_('function'), 'func'), - 'method': ObjType(l_('method'), 'meth'), - 'class': ObjType(l_('class'), 'class'), - 'data': ObjType(l_('data'), 'data'), - 'attribute': ObjType(l_('attribute'), 'attr'), - 'module': ObjType(l_('module'), 'mod'), + 'function': ObjType(_('function'), 'func'), + 'method': ObjType(_('method'), 'meth'), + 'class': ObjType(_('class'), 'class'), + 'data': ObjType(_('data'), 'data'), + 'attribute': ObjType(_('attribute'), 'attr'), + 'module': ObjType(_('module'), 'mod'), } directives = { 'function': JSCallable, @@ -415,6 +415,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/python.py b/sphinx/domains/python.py index b0900b385..312b509a0 100644 --- a/sphinx/domains/python.py +++ b/sphinx/domains/python.py @@ -15,10 +15,11 @@ from docutils import nodes from docutils.parsers.rst import Directive, directives from six import iteritems -from sphinx import addnodes +from sphinx import addnodes, locale +from sphinx.deprecation import DeprecatedDict, RemovedInSphinx30Warning from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType, Index -from sphinx.locale import l_, _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import logging from sphinx.util.docfields import Field, GroupedField, TypedField @@ -44,6 +45,24 @@ py_sig_re = re.compile( ''', re.VERBOSE) +pairindextypes = { + 'module': _('module'), + 'keyword': _('keyword'), + 'operator': _('operator'), + 'object': _('object'), + 'exception': _('exception'), + 'statement': _('statement'), + 'builtin': _('built-in function'), +} # Dict[unicode, unicode] + +locale.pairindextypes = DeprecatedDict( + pairindextypes, + 'sphinx.locale.pairindextypes is deprecated. ' + 'Please use sphinx.domains.python.pairindextypes instead.', + RemovedInSphinx30Warning +) + + def _pseudo_parse_arglist(signode, arglist): # type: (addnodes.desc_signature, unicode) -> None """"Parse" a list of arguments separated by commas. @@ -173,21 +192,21 @@ class PyObject(ObjectDescription): } doc_field_types = [ - PyTypedField('parameter', label=l_('Parameters'), + PyTypedField('parameter', label=_('Parameters'), names=('param', 'parameter', 'arg', 'argument', 'keyword', 'kwarg', 'kwparam'), typerolename='class', typenames=('paramtype', 'type'), can_collapse=True), - PyTypedField('variable', label=l_('Variables'), rolename='obj', + PyTypedField('variable', label=_('Variables'), rolename='obj', names=('var', 'ivar', 'cvar'), typerolename='class', typenames=('vartype',), can_collapse=True), - PyGroupedField('exceptions', label=l_('Raises'), rolename='exc', + PyGroupedField('exceptions', label=_('Raises'), rolename='exc', names=('raises', 'raise', 'exception', 'except'), can_collapse=True), - Field('returnvalue', label=l_('Returns'), has_arg=False, + Field('returnvalue', label=_('Returns'), has_arg=False, names=('returns', 'return')), - PyField('returntype', label=l_('Return type'), has_arg=False, + PyField('returntype', label=_('Return type'), has_arg=False, names=('rtype',), bodyrolename='class'), ] @@ -631,8 +650,8 @@ class PythonModuleIndex(Index): """ name = 'modindex' - localname = l_('Python Module Index') - shortname = l_('modules') + localname = _('Python Module Index') + shortname = _('modules') def generate(self, docnames=None): # type: (Iterable[unicode]) -> Tuple[List[Tuple[unicode, List[List[Union[unicode, int]]]]], bool] # NOQA @@ -702,15 +721,15 @@ class PythonDomain(Domain): name = 'py' label = 'Python' object_types = { - 'function': ObjType(l_('function'), 'func', 'obj'), - 'data': ObjType(l_('data'), 'data', 'obj'), - 'class': ObjType(l_('class'), 'class', 'exc', 'obj'), - 'exception': ObjType(l_('exception'), 'exc', 'class', 'obj'), - 'method': ObjType(l_('method'), 'meth', 'obj'), - 'classmethod': ObjType(l_('class method'), 'meth', 'obj'), - 'staticmethod': ObjType(l_('static method'), 'meth', 'obj'), - 'attribute': ObjType(l_('attribute'), 'attr', 'obj'), - 'module': ObjType(l_('module'), 'mod', 'obj'), + 'function': ObjType(_('function'), 'func', 'obj'), + 'data': ObjType(_('data'), 'data', 'obj'), + 'class': ObjType(_('class'), 'class', 'exc', 'obj'), + 'exception': ObjType(_('exception'), 'exc', 'class', 'obj'), + 'method': ObjType(_('method'), 'meth', 'obj'), + 'classmethod': ObjType(_('class method'), 'meth', 'obj'), + 'staticmethod': ObjType(_('static method'), 'meth', 'obj'), + 'attribute': ObjType(_('attribute'), 'attr', 'obj'), + 'module': ObjType(_('module'), 'mod', 'obj'), } # type: Dict[unicode, ObjType] directives = { @@ -840,7 +859,7 @@ class PythonDomain(Domain): if not matches: return None elif len(matches) > 1: - logger.warning('more than one target found for cross-reference %r: %s', + logger.warning(__('more than one target found for cross-reference %r: %s'), target, ', '.join(match[0] for match in matches), type='ref', subtype='python', location=node) name, obj = matches[0] @@ -911,6 +930,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/rst.py b/sphinx/domains/rst.py index c14eb180f..fcb461484 100644 --- a/sphinx/domains/rst.py +++ b/sphinx/domains/rst.py @@ -16,7 +16,7 @@ from six import iteritems from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import l_, _ +from sphinx.locale import _ from sphinx.roles import XRefRole from sphinx.util.nodes import make_refnode @@ -116,8 +116,8 @@ class ReSTDomain(Domain): label = 'reStructuredText' object_types = { - 'directive': ObjType(l_('directive'), 'dir'), - 'role': ObjType(l_('role'), 'role'), + 'directive': ObjType(_('directive'), 'dir'), + 'role': ObjType(_('role'), 'role'), } directives = { 'directive': ReSTDirective, @@ -182,6 +182,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/domains/std.py b/sphinx/domains/std.py index 937760c45..307108b00 100644 --- a/sphinx/domains/std.py +++ b/sphinx/domains/std.py @@ -11,6 +11,7 @@ import re import unicodedata +from copy import copy from docutils import nodes from docutils.parsers.rst import Directive, directives @@ -20,7 +21,7 @@ from six import iteritems from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import Domain, ObjType -from sphinx.locale import l_, _ +from sphinx.locale import _, __ from sphinx.roles import XRefRole from sphinx.util import ws_re, logging, docname_join from sphinx.util.nodes import clean_astext, make_refnode @@ -80,7 +81,7 @@ class GenericObject(ObjectDescription): class EnvVar(GenericObject): - indextemplate = l_('environment variable; %s') + indextemplate = _('environment variable; %s') class EnvVarXRefRole(XRefRole): @@ -157,9 +158,9 @@ class Cmdoption(ObjectDescription): potential_option = potential_option.strip() m = option_desc_re.match(potential_option) # type: ignore if not m: - logger.warning('Malformed option description %r, should ' - 'look like "opt", "-opt args", "--opt args", ' - '"/opt args" or "+opt args"', potential_option, + logger.warning(__('Malformed option description %r, should ' + 'look like "opt", "-opt args", "--opt args", ' + '"/opt args" or "+opt args"'), potential_option, location=(self.env.docname, self.lineno)) continue optname, args = m.groups() @@ -451,13 +452,13 @@ class StandardDomain(Domain): label = 'Default' object_types = { - 'term': ObjType(l_('glossary term'), 'term', searchprio=-1), - 'token': ObjType(l_('grammar token'), 'token', searchprio=-1), - 'label': ObjType(l_('reference label'), 'ref', 'keyword', + 'term': ObjType(_('glossary term'), 'term', searchprio=-1), + 'token': ObjType(_('grammar token'), 'token', searchprio=-1), + 'label': ObjType(_('reference label'), 'ref', 'keyword', searchprio=-1), - 'envvar': ObjType(l_('environment variable'), 'envvar'), - 'cmdoption': ObjType(l_('program option'), 'option'), - 'doc': ObjType(l_('document'), 'doc', searchprio=-1) + 'envvar': ObjType(_('environment variable'), 'envvar'), + 'cmdoption': ObjType(_('program option'), 'option'), + 'doc': ObjType(_('document'), 'doc', searchprio=-1) } # type: Dict[unicode, ObjType] directives = { @@ -494,9 +495,9 @@ class StandardDomain(Domain): 'citations': {}, # citation_name -> docname, labelid, lineno 'citation_refs': {}, # citation_name -> list of docnames 'labels': { # labelname -> docname, labelid, sectionname - 'genindex': ('genindex', '', l_('Index')), - 'modindex': ('py-modindex', '', l_('Module Index')), - 'search': ('search', '', l_('Search Page')), + 'genindex': ('genindex', '', _('Index')), + 'modindex': ('py-modindex', '', _('Module Index')), + 'search': ('search', '', _('Search Page')), }, 'anonlabels': { # labelname -> docname, labelid 'genindex': ('genindex', ''), @@ -522,6 +523,15 @@ class StandardDomain(Domain): nodes.container: ('code-block', None), } # type: Dict[nodes.Node, Tuple[unicode, Callable]] + def __init__(self, env): + # type: (BuildEnvironment) -> None + super(StandardDomain, self).__init__(env) + + # set up enumerable nodes + self.enumerable_nodes = copy(self.enumerable_nodes) # create a copy for this instance + for node, settings in iteritems(env.app.registry.enumerable_nodes): + self.enumerable_nodes[node] = settings + def clear_doc(self, docname): # type: (unicode) -> None for key, (fn, _l) in list(self.data['progoptions'].items()): @@ -581,7 +591,7 @@ class StandardDomain(Domain): label = node[0].astext() if label in self.data['citations']: path = env.doc2path(self.data['citations'][label][0]) - logger.warning('duplicate citation %s, other instance in %s', label, path, + logger.warning(__('duplicate citation %s, other instance in %s'), label, path, location=node, type='ref', subtype='citation') self.data['citations'][label] = (docname, node['ids'][0], node.line) @@ -613,8 +623,8 @@ class StandardDomain(Domain): # link and object descriptions continue if name in labels: - logger.warning('duplicate label %s, ' % name + 'other instance ' - 'in ' + env.doc2path(labels[name][0]), + logger.warning(__('duplicate label %s, other instance in %s'), + name, env.doc2path(labels[name][0]), location=node) anonlabels[name] = docname, labelid if node.tagname in ('section', 'rubric'): @@ -638,7 +648,7 @@ class StandardDomain(Domain): # type: () -> None for name, (docname, labelid, lineno) in iteritems(self.data['citations']): if name not in self.data['citation_refs']: - logger.warning('Citation [%s] is not referenced.', name, + logger.warning(__('Citation [%s] is not referenced.'), name, type='ref', subtype='citation', location=(docname, lineno)) @@ -721,7 +731,7 @@ class StandardDomain(Domain): return None if figtype != 'section' and env.config.numfig is False: - logger.warning('numfig is disabled. :numref: is ignored.', location=node) + logger.warning(__('numfig is disabled. :numref: is ignored.'), location=node) return contnode try: @@ -729,7 +739,7 @@ class StandardDomain(Domain): if fignumber is None: return contnode except ValueError: - logger.warning("no number is assigned for %s: %s", figtype, labelid, + logger.warning(__("no number is assigned for %s: %s"), figtype, labelid, location=node) return contnode @@ -740,7 +750,7 @@ class StandardDomain(Domain): title = env.config.numfig_format.get(figtype, '') if figname is None and '{name}' in title: - logger.warning('the link has no caption: %s', title, location=node) + logger.warning(__('the link has no caption: %s'), title, location=node) return contnode else: fignum = '.'.join(map(str, fignumber)) @@ -754,10 +764,10 @@ class StandardDomain(Domain): # old style format (cf. "Fig.%s") newtitle = title % fignum except KeyError as exc: - logger.warning('invalid numfig_format: %s (%r)', title, exc, location=node) + logger.warning(__('invalid numfig_format: %s (%r)'), title, exc, location=node) return contnode except TypeError: - logger.warning('invalid numfig_format: %s', title, location=node) + logger.warning(__('invalid numfig_format: %s'), title, location=node) return contnode return self.build_reference_node(fromdocname, builder, @@ -979,6 +989,7 @@ def setup(app): return { 'version': 'builtin', + 'env_version': 1, 'parallel_read_safe': True, 'parallel_write_safe': True, } diff --git a/sphinx/environment/__init__.py b/sphinx/environment/__init__.py index b31021caa..461fc1227 100644 --- a/sphinx/environment/__init__.py +++ b/sphinx/environment/__init__.py @@ -25,21 +25,20 @@ from six import BytesIO, itervalues, class_types, next from six.moves import cPickle as pickle from sphinx import addnodes, versioning -from sphinx.deprecation import RemovedInSphinx20Warning +from sphinx.deprecation import RemovedInSphinx20Warning, RemovedInSphinx30Warning from sphinx.environment.adapters.indexentries import IndexEntries from sphinx.environment.adapters.toctree import TocTree from sphinx.errors import SphinxError, ExtensionError from sphinx.io import read_doc +from sphinx.locale import __ from sphinx.transforms import SphinxTransformer -from sphinx.util import get_matching_docs, FilenameUniqDict, status_iterator +from sphinx.util import get_matching_docs, FilenameUniqDict from sphinx.util import logging, rst -from sphinx.util.console import bold # type: ignore from sphinx.util.docutils import sphinx_domains, WarningStream from sphinx.util.i18n import find_catalog_files from sphinx.util.matching import compile_matchers from sphinx.util.nodes import is_translatable from sphinx.util.osutil import SEP, ensuredir -from sphinx.util.parallel import ParallelTasks, parallel_available, make_chunks from sphinx.util.websupport import is_commentable if False: @@ -107,13 +106,9 @@ class BuildEnvironment(object): # This can happen for example when the pickle is from a # different version of Sphinx. raise IOError(exc) - if env.version != ENV_VERSION: - raise IOError('build environment version not current') if app: env.app = app env.config.values = app.config.values - if env.srcdir != app.srcdir: - raise IOError('source directory has changed') return env @classmethod @@ -187,7 +182,7 @@ class BuildEnvironment(object): self._warnfunc = None # type: Callable # this is to invalidate old pickles - self.version = ENV_VERSION # type: int + self.version = app.registry.get_envversion(app) # type: Dict[unicode, unicode] # All "docnames" here are /-separated and relative and exclude # the source suffix. @@ -283,9 +278,9 @@ class BuildEnvironment(object): raise ValueError('invalid versioning method: %r' % method) condition = versioning_conditions[method] if self.versioning_condition not in (None, condition): - raise SphinxError('This environment is incompatible with the ' - 'selected builder, please choose another ' - 'doctree directory.') + raise SphinxError(__('This environment is incompatible with the ' + 'selected builder, please choose another ' + 'doctree directory.')) self.versioning_condition = condition self.versioning_compare = compare @@ -304,6 +299,19 @@ class BuildEnvironment(object): """Like :meth:`warn`, but with source information taken from *node*.""" self._warnfunc(msg, '%s:%s' % get_source_line(node), **kwargs) + def need_refresh(self, app): + # type: (Sphinx) -> Tuple[bool, unicode] + """Check refresh environment is needed. + + If needed, this method returns the reason for refresh. + """ + if self.version != app.registry.get_envversion(app): + return True, __('build environment version not current') + elif self.srcdir != app.srcdir: + return True, __('source directory has changed') + else: + return False, None + def clear_doc(self, docname): # type: (unicode) -> None """Remove all traces of a source file in the inventory.""" @@ -419,7 +427,7 @@ class BuildEnvironment(object): if os.access(self.doc2path(docname), os.R_OK): self.found_docs.add(docname) else: - logger.warning("document not readable. Ignored.", location=docname) + logger.warning(__("document not readable. Ignored."), location=docname) # Current implementation is applying translated messages in the reading # phase.Therefore, in order to apply the updated message catalog, it is @@ -490,138 +498,56 @@ class BuildEnvironment(object): return added, changed, removed - def update(self, config, srcdir, doctreedir): - # type: (Config, unicode, unicode) -> List[unicode] - """(Re-)read all files new or changed since last update. + def check_dependents(self, app, already): + # type: (Sphinx, Set[unicode]) -> Iterator[unicode] + to_rewrite = [] # type: List[unicode] + for docnames in app.emit('env-get-updated', self): + to_rewrite.extend(docnames) + for docname in set(to_rewrite): + if docname not in already: + yield docname - Store all environment docnames in the canonical format (ie using SEP as - a separator in place of os.path.sep). - """ - config_changed = False + def update_config(self, config, srcdir, doctreedir): + # type: (Config, unicode, unicode) -> Tuple[bool, unicode] + """Update configurations by new one.""" + changed_reason = '' if self.config is None: - msg = '[new config] ' - config_changed = True + changed_reason = __('new config') else: # check if a config value was changed that affects how # doctrees are read for confval in config.filter('env'): if self.config[confval.name] != confval.value: - msg = '[config changed] ' - config_changed = True + changed_reason = __('config changed') break - else: - msg = '' + # this value is not covered by the above loop because it is handled # specially by the config class if self.config.extensions != config.extensions: - msg = '[extensions changed] ' - config_changed = True + changed_reason = __('extensions changed') + # the source and doctree directories may have been relocated self.srcdir = srcdir self.doctreedir = doctreedir - self.find_files(config, self.app.builder) self.config = config + self._update_settings(config) # this cache also needs to be updated every time self._nitpick_ignore = set(self.config.nitpick_ignore) - logger.info(bold('updating environment: '), nonl=True) - - added, changed, removed = self.get_outdated_files(config_changed) - - # allow user intervention as well - for docs in self.app.emit('env-get-outdated', self, added, changed, removed): - changed.update(set(docs) & self.found_docs) - - # if files were added or removed, all documents with globbed toctrees - # must be reread - if added or removed: - # ... but not those that already were removed - changed.update(self.glob_toctrees & self.found_docs) - - msg += '%s added, %s changed, %s removed' % (len(added), len(changed), - len(removed)) - logger.info(msg) - - # clear all files no longer present - for docname in removed: - self.app.emit('env-purge-doc', self, docname) - self.clear_doc(docname) - - # read all new and changed files - docnames = sorted(added | changed) - # allow changing and reordering the list of docs to read - self.app.emit('env-before-read-docs', self, docnames) - - # check if we should do parallel or serial read - if parallel_available and len(docnames) > 5 and self.app.parallel > 1: - par_ok = self.app.is_parallel_allowed('read') - else: - par_ok = False - - if par_ok: - self._read_parallel(docnames, self.app, nproc=self.app.parallel) - else: - self._read_serial(docnames, self.app) - - if config.master_doc not in self.all_docs: - raise SphinxError('master file %s not found' % - self.doc2path(config.master_doc)) - - for retval in self.app.emit('env-updated', self): - if retval is not None: - docnames.extend(retval) - - return sorted(docnames) - - def _read_serial(self, docnames, app): - # type: (List[unicode], Sphinx) -> None - for docname in status_iterator(docnames, 'reading sources... ', "purple", - len(docnames), self.app.verbosity): - # remove all inventory entries for that file - app.emit('env-purge-doc', self, docname) - self.clear_doc(docname) - self.read_doc(docname, app) - - def _read_parallel(self, docnames, app, nproc): - # type: (List[unicode], Sphinx, int) -> None - # clear all outdated docs at once - for docname in docnames: - app.emit('env-purge-doc', self, docname) - self.clear_doc(docname) - - def read_process(docs): - # type: (List[unicode]) -> unicode - self.app = app - for docname in docs: - self.read_doc(docname, app) - # allow pickling self to send it back - return BuildEnvironment.dumps(self) + # return tuple of (changed, reason) + return bool(changed_reason), changed_reason - def merge(docs, otherenv): - # type: (List[unicode], unicode) -> None - env = BuildEnvironment.loads(otherenv) - self.merge_info_from(docs, env, app) + def _update_settings(self, config): + # type: (Config) -> None + """Update settings by new config.""" + self.settings['input_encoding'] = config.source_encoding + self.settings['trim_footnote_reference_space'] = config.trim_footnote_reference_space + self.settings['gettext_compact'] = config.gettext_compact + self.settings['language_code'] = config.language or 'en' - tasks = ParallelTasks(nproc) - chunks = make_chunks(docnames, nproc) - - for chunk in status_iterator(chunks, 'reading sources... ', "purple", - len(chunks), self.app.verbosity): - tasks.add_task(read_process, chunk, merge) - - # make sure all threads have finished - logger.info(bold('waiting for workers...')) - tasks.join() - - def check_dependents(self, app, already): - # type: (Sphinx, Set[unicode]) -> Iterator[unicode] - to_rewrite = [] # type: List[unicode] - for docnames in app.emit('env-get-updated', self): - to_rewrite.extend(docnames) - for docname in set(to_rewrite): - if docname not in already: - yield docname + # Allow to disable by 3rd party extension (workaround) + self.settings.setdefault('smart_quotes', True) # --------- SINGLE FILE READING -------------------------------------------- @@ -634,16 +560,6 @@ class BuildEnvironment(object): self.temp_data['default_domain'] = \ self.domains.get(self.config.primary_domain) - self.settings['input_encoding'] = self.config.source_encoding - self.settings['trim_footnote_reference_space'] = \ - self.config.trim_footnote_reference_space - self.settings['gettext_compact'] = self.config.gettext_compact - - self.settings['language_code'] = self.config.language or 'en' - - # Allow to disable by 3rd party extension (workaround) - self.settings.setdefault('smart_quotes', True) - def read_doc(self, docname, app=None): # type: (unicode, Sphinx) -> None """Parse a file and add/update inventory entries for the doctree.""" @@ -771,7 +687,7 @@ class BuildEnvironment(object): try: return self.domains[domainname] except KeyError: - raise ExtensionError('Domain %r is not registered' % domainname) + raise ExtensionError(__('Domain %r is not registered') % domainname) # --------- RESOLVING REFERENCES AND TOCTREES ------------------------------ @@ -881,7 +797,7 @@ class BuildEnvironment(object): def traverse_toctree(parent, docname): # type: (unicode, unicode) -> Iterator[Tuple[unicode, unicode]] if parent == docname: - logger.warning('self referenced toctree found. Ignored.', location=docname) + logger.warning(__('self referenced toctree found. Ignored.'), location=docname) return # traverse toctree by pre-order @@ -921,10 +837,30 @@ class BuildEnvironment(object): continue if 'orphan' in self.metadata[docname]: continue - logger.warning('document isn\'t included in any toctree', + logger.warning(__('document isn\'t included in any toctree'), location=docname) # call check-consistency for all extensions for domain in self.domains.values(): domain.check_consistency() self.app.emit('env-check-consistency', self) + + # --------- METHODS FOR COMPATIBILITY -------------------------------------- + + def update(self, config, srcdir, doctreedir): + # type: (Config, unicode, unicode) -> List[unicode] + warnings.warn('env.update() is deprecated. Please use builder.read() instead.', + RemovedInSphinx30Warning) + return self.app.builder.read() + + def _read_serial(self, docnames, app): + # type: (List[unicode], Sphinx) -> None + warnings.warn('env._read_serial() is deprecated. Please use builder.read() instead.', + RemovedInSphinx30Warning) + return self.app.builder._read_serial(docnames) + + def _read_parallel(self, docnames, app, nproc): + # type: (List[unicode], Sphinx, int) -> None + warnings.warn('env._read_parallel() is deprecated. Please use builder.read() instead.', + RemovedInSphinx30Warning) + return self.app.builder._read_parallel(docnames, nproc) diff --git a/sphinx/environment/adapters/indexentries.py b/sphinx/environment/adapters/indexentries.py index 5ac8ff1d6..7c31fc3d5 100644 --- a/sphinx/environment/adapters/indexentries.py +++ b/sphinx/environment/adapters/indexentries.py @@ -15,7 +15,7 @@ from itertools import groupby from six import text_type, iteritems -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import split_into, logging if False: @@ -89,7 +89,7 @@ class IndexEntries(object): add_entry(first, _('see also %s') % second, None, link=False, key=index_key) else: - logger.warning('unknown index entry type %r', type, location=fn) + logger.warning(__('unknown index entry type %r'), type, location=fn) except ValueError as err: logger.warning(str(err), location=fn) diff --git a/sphinx/environment/adapters/toctree.py b/sphinx/environment/adapters/toctree.py index af5d85f23..f1261bcdc 100644 --- a/sphinx/environment/adapters/toctree.py +++ b/sphinx/environment/adapters/toctree.py @@ -13,6 +13,7 @@ from docutils import nodes from six import iteritems from sphinx import addnodes +from sphinx.locale import __ from sphinx.util import url_re, logging from sphinx.util.nodes import clean_astext, process_only_nodes @@ -147,8 +148,8 @@ class TocTree(object): toc = nodes.bullet_list('', item) else: if ref in parents: - logger.warning('circular toctree references ' - 'detected, ignoring: %s <- %s', + logger.warning(__('circular toctree references ' + 'detected, ignoring: %s <- %s'), ref, ' <- '.join(parents), location=ref) continue @@ -166,12 +167,12 @@ class TocTree(object): refnode.children = [nodes.Text(title)] if not toc.children: # empty toc means: no titles will show up in the toctree - logger.warning('toctree contains reference to document %r that ' - 'doesn\'t have a title: no link will be generated', + logger.warning(__('toctree contains reference to document %r that ' + 'doesn\'t have a title: no link will be generated'), ref, location=toctreenode) except KeyError: # this is raised if the included file does not exist - logger.warning('toctree contains reference to nonexisting document %r', + logger.warning(__('toctree contains reference to nonexisting document %r'), ref, location=toctreenode) else: # if titles_only is given, only keep the main title and diff --git a/sphinx/environment/collectors/asset.py b/sphinx/environment/collectors/asset.py index 2504a7ea4..70faf108c 100644 --- a/sphinx/environment/collectors/asset.py +++ b/sphinx/environment/collectors/asset.py @@ -19,6 +19,7 @@ from six import iteritems, itervalues from sphinx import addnodes from sphinx.environment.collectors import EnvironmentCollector +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.i18n import get_image_filename_for_language, search_image_for_language from sphinx.util.images import guess_mimetype @@ -89,7 +90,7 @@ class ImageCollector(EnvironmentCollector): for imgpath in itervalues(candidates): app.env.dependencies[docname].add(imgpath) if not os.access(path.join(app.srcdir, imgpath), os.R_OK): - logger.warning('image file not readable: %s' % imgpath, + logger.warning(__('image file not readable: %s') % imgpath, location=node, type='image', subtype='not_readable') continue app.env.images.add_file(docname, imgpath) @@ -105,7 +106,7 @@ class ImageCollector(EnvironmentCollector): if mimetype not in candidates: globbed.setdefault(mimetype, []).append(new_imgpath) except (OSError, IOError) as err: - logger.warning('image file %s not readable: %s' % (filename, err), + logger.warning(__('image file %s not readable: %s') % (filename, err), location=node, type='image', subtype='not_readable') for key, files in iteritems(globbed): candidates[key] = sorted(files, key=len)[0] # select by similarity @@ -130,7 +131,7 @@ class DownloadFileCollector(EnvironmentCollector): rel_filename, filename = app.env.relfn2path(targetname, app.env.docname) app.env.dependencies[app.env.docname].add(rel_filename) if not os.access(filename, os.R_OK): - logger.warning('download file not readable: %s' % filename, + logger.warning(__('download file not readable: %s') % filename, location=node, type='download', subtype='not_readable') continue node['filename'] = app.env.dlfiles.add_file(app.env.docname, filename) diff --git a/sphinx/environment/collectors/indexentries.py b/sphinx/environment/collectors/indexentries.py index dec5dbc75..a9ba897d0 100644 --- a/sphinx/environment/collectors/indexentries.py +++ b/sphinx/environment/collectors/indexentries.py @@ -9,6 +9,8 @@ :license: BSD, see LICENSE for details. """ +from six import text_type + from sphinx import addnodes from sphinx.environment.collectors import EnvironmentCollector from sphinx.util import split_index_msg, logging @@ -44,7 +46,7 @@ class IndexEntriesCollector(EnvironmentCollector): for entry in node['entries']: split_index_msg(entry[0], entry[1]) except ValueError as exc: - logger.warning(str(exc), location=node) + logger.warning(text_type(exc), location=node) node.parent.remove(node) else: for entry in node['entries']: diff --git a/sphinx/environment/collectors/toctree.py b/sphinx/environment/collectors/toctree.py index a7556eadd..245433002 100644 --- a/sphinx/environment/collectors/toctree.py +++ b/sphinx/environment/collectors/toctree.py @@ -15,6 +15,7 @@ from six import iteritems from sphinx import addnodes from sphinx.environment.adapters.toctree import TocTree from sphinx.environment.collectors import EnvironmentCollector +from sphinx.locale import __ from sphinx.transforms import SphinxContentsFilter from sphinx.util import url_re, logging @@ -66,6 +67,7 @@ class TocTreeCollector(EnvironmentCollector): numentries = [0] # nonlocal again... def traverse_in_section(node, cls): + # type: (nodes.Node, Any) -> List[nodes.Node] """Like traverse(), but stay within the same section.""" result = [] if isinstance(node, cls): @@ -77,6 +79,7 @@ class TocTreeCollector(EnvironmentCollector): return result def build_toc(node, depth=1): + # type: (nodes.Node, int) -> List[nodes.Node] entries = [] for sectionnode in node: # find all toctree nodes in this section and add them @@ -86,7 +89,7 @@ class TocTreeCollector(EnvironmentCollector): onlynode = addnodes.only(expr=sectionnode['expr']) blist = build_toc(sectionnode, depth) if blist: - onlynode += blist.children + onlynode += blist.children # type: ignore entries.append(onlynode) continue if not isinstance(sectionnode, nodes.section): @@ -145,6 +148,7 @@ class TocTreeCollector(EnvironmentCollector): env.toc_secnumbers = {} def _walk_toc(node, secnums, depth, titlenode=None): + # type: (nodes.Node, Dict, int, nodes.Node) -> None # titlenode is the title of the document, it will get assigned a # secnumber too, so that it shows up in next/prev/parent rellinks for subnode in node.children: @@ -177,6 +181,7 @@ class TocTreeCollector(EnvironmentCollector): _walk_toctree(subnode, depth) def _walk_toctree(toctreenode, depth): + # type: (nodes.Node, int) -> None if depth == 0: return for (title, ref) in toctreenode['entries']: @@ -184,8 +189,8 @@ class TocTreeCollector(EnvironmentCollector): # don't mess with those continue elif ref in assigned: - logger.warning('%s is already assigned section numbers ' - '(nested numbered toctree?)', ref, + logger.warning(__('%s is already assigned section numbers ' + '(nested numbered toctree?)'), ref, location=toctreenode, type='toc', subtype='secnum') elif ref in env.tocs: secnums = env.toc_secnumbers[ref] = {} @@ -216,9 +221,10 @@ class TocTreeCollector(EnvironmentCollector): assigned = set() # type: Set[unicode] old_fignumbers = env.toc_fignumbers env.toc_fignumbers = {} - fignum_counter = {} # type: Dict[unicode, Dict[Tuple[int], int]] + fignum_counter = {} # type: Dict[unicode, Dict[Tuple[int, ...], int]] def get_section_number(docname, section): + # type: (unicode, nodes.Node) -> Tuple[int, ...] anchorname = '#' + section['ids'][0] secnumbers = env.toc_secnumbers.get(docname, {}) if anchorname in secnumbers: @@ -229,6 +235,7 @@ class TocTreeCollector(EnvironmentCollector): return secnum or tuple() def get_next_fignumber(figtype, secnum): + # type: (unicode, Tuple[int, ...]) -> Tuple[int, ...] counter = fignum_counter.setdefault(figtype, {}) secnum = secnum[:env.config.numfig_secnum_depth] @@ -236,6 +243,7 @@ class TocTreeCollector(EnvironmentCollector): return secnum + (counter[secnum],) def register_fignumber(docname, secnum, figtype, fignode): + # type: (unicode, Tuple[int], unicode, nodes.Node) -> None env.toc_fignumbers.setdefault(docname, {}) fignumbers = env.toc_fignumbers[docname].setdefault(figtype, {}) figure_id = fignode['ids'][0] @@ -243,6 +251,7 @@ class TocTreeCollector(EnvironmentCollector): fignumbers[figure_id] = get_next_fignumber(figtype, secnum) def _walk_doctree(docname, doctree, secnum): + # type: (unicode, nodes.Node, Tuple[int, ...]) -> None for subnode in doctree.children: if isinstance(subnode, nodes.section): next_secnum = get_section_number(docname, subnode) @@ -268,13 +277,14 @@ class TocTreeCollector(EnvironmentCollector): _walk_doctree(docname, subnode, secnum) def _walk_doc(docname, secnum): + # type: (unicode, Tuple[int]) -> None if docname not in assigned: assigned.add(docname) doctree = env.get_doctree(docname) _walk_doctree(docname, doctree, secnum) if env.config.numfig: - _walk_doc(env.config.master_doc, tuple()) + _walk_doc(env.config.master_doc, tuple()) # type: ignore for docname, fignums in iteritems(env.toc_fignumbers): if fignums != old_fignumbers.get(docname): rewrite_needed.append(docname) diff --git a/sphinx/errors.py b/sphinx/errors.py index eef1a157a..7652e93cb 100644 --- a/sphinx/errors.py +++ b/sphinx/errors.py @@ -16,20 +16,40 @@ if False: class SphinxError(Exception): - """ - Base class for Sphinx errors that are shown to the user in a nicer - way than normal exceptions. + """Base class for Sphinx errors. + + This is the base class for "nice" exceptions. When such an exception is + raised, Sphinx will abort the build and present the exception category and + message to the user. + + Extensions are encouraged to derive from this exception for their custom + errors. + + Exceptions *not* derived from :exc:`SphinxError` are treated as unexpected + and shown to the user with a part of the traceback (and the full traceback + saved in a temporary file). + + .. attribute:: category + + Description of the exception "category", used in converting the + exception to a string ("category: message"). Should be set accordingly + in subclasses. """ category = 'Sphinx error' class SphinxWarning(SphinxError): - """Raised for warnings if warnings are treated as errors.""" + """Warning, treated as error.""" category = 'Warning, treated as error' +class ApplicationError(SphinxError): + """Application initialization error.""" + category = 'Application error' + + class ExtensionError(SphinxError): - """Raised if something's wrong with the configuration.""" + """Extension error.""" category = 'Extension error' def __init__(self, message, orig_exc=None): @@ -53,27 +73,22 @@ class ExtensionError(SphinxError): class ConfigError(SphinxError): + """Configuration error.""" category = 'Configuration error' class ThemeError(SphinxError): + """Theme error.""" category = 'Theme error' class VersionRequirementError(SphinxError): + """Incompatible Sphinx version error.""" category = 'Sphinx version error' -class PycodeError(Exception): - def __str__(self): - # type: () -> str - res = self.args[0] - if len(self.args) > 1: - res += ' (exception was: %r)' % self.args[1] - return res - - class SphinxParallelError(SphinxError): + """Sphinx parallel build error.""" category = 'Sphinx parallel build error' @@ -85,3 +100,14 @@ class SphinxParallelError(SphinxError): def __str__(self): # type: () -> str return self.message + + +class PycodeError(Exception): + """Pycode Python source code analyser error.""" + + def __str__(self): + # type: () -> str + res = self.args[0] + if len(self.args) > 1: + res += ' (exception was: %r)' % self.args[1] + return res diff --git a/sphinx/events.py b/sphinx/events.py index 097f61fc6..fb62d1776 100644 --- a/sphinx/events.py +++ b/sphinx/events.py @@ -27,6 +27,7 @@ if False: # List of all known core events. Maps name to arguments description. core_events = { 'builder-inited': '', + 'config-inited': 'config', 'env-get-outdated': 'env, added, changed, removed', 'env-get-updated': 'env', 'env-purge-doc': 'env, docname', diff --git a/sphinx/ext/apidoc.py b/sphinx/ext/apidoc.py index 07c399f25..056835863 100644 --- a/sphinx/ext/apidoc.py +++ b/sphinx/ext/apidoc.py @@ -9,7 +9,7 @@ This is derived from the "sphinx-autopackage" script, which is: Copyright 2008 Société des arts technologiques (SAT), - http://www.sat.qc.ca/ + https://sat.qc.ca/ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. @@ -19,6 +19,7 @@ from __future__ import print_function import argparse import glob +import locale import os import sys from fnmatch import fnmatch @@ -26,8 +27,10 @@ from os import path from six import binary_type -from sphinx import __display_version__ +import sphinx.locale +from sphinx import __display_version__, package_dir from sphinx.cmd.quickstart import EXTENSIONS +from sphinx.locale import __ from sphinx.util import rst from sphinx.util.osutil import FileAvoidWrite, ensuredir, walk @@ -68,12 +71,12 @@ def write_file(name, text, opts): """Write the output file for module/package <name>.""" fname = path.join(opts.destdir, '%s.%s' % (name, opts.suffix)) if opts.dryrun: - print('Would create file %s.' % fname) + print(__('Would create file %s.') % fname) return if not opts.force and path.isfile(fname): - print('File %s already exists, skipping.' % fname) + print(__('File %s already exists, skipping.') % fname) else: - print('Creating file %s.' % fname) + print(__('Creating file %s.') % fname) with FileAvoidWrite(fname) as f: f.write(text) @@ -284,7 +287,7 @@ def is_excluded(root, excludes): """Check if the directory is in the exclude list. Note: by having trailing slashes, we avoid common prefix issues, like - e.g. an exlude "foo" also accidentally excluding "foobar". + e.g. an exclude "foo" also accidentally excluding "foobar". """ for exclude in excludes: if fnmatch(root, exclude): @@ -297,84 +300,84 @@ def get_parser(): parser = argparse.ArgumentParser( usage='usage: %(prog)s [OPTIONS] -o <OUTPUT_PATH> <MODULE_PATH> ' '[EXCLUDE_PATTERN, ...]', - epilog='For more information, visit <http://sphinx-doc.org/>.', - description=""" + epilog=__('For more information, visit <http://sphinx-doc.org/>.'), + description=__(""" Look recursively in <MODULE_PATH> for Python modules and packages and create one reST file with automodule directives per package in the <OUTPUT_PATH>. The <EXCLUDE_PATTERN>s can be file and/or directory patterns that will be excluded from generation. -Note: By default this script will not overwrite already created files.""") +Note: By default this script will not overwrite already created files.""")) parser.add_argument('--version', action='version', dest='show_version', version='%%(prog)s %s' % __display_version__) parser.add_argument('module_path', - help='path to module to document') + help=__('path to module to document')) parser.add_argument('exclude_pattern', nargs='*', - help='fnmatch-style file and/or directory patterns ' - 'to exclude from generation') + help=__('fnmatch-style file and/or directory patterns ' + 'to exclude from generation')) parser.add_argument('-o', '--output-dir', action='store', dest='destdir', required=True, - help='directory to place all output') + help=__('directory to place all output')) parser.add_argument('-d', '--maxdepth', action='store', dest='maxdepth', type=int, default=4, - help='maximum depth of submodules to show in the TOC ' - '(default: 4)') + help=__('maximum depth of submodules to show in the TOC ' + '(default: 4)')) parser.add_argument('-f', '--force', action='store_true', dest='force', - help='overwrite existing files') + help=__('overwrite existing files')) parser.add_argument('-l', '--follow-links', action='store_true', dest='followlinks', default=False, - help='follow symbolic links. Powerful when combined ' - 'with collective.recipe.omelette.') + help=__('follow symbolic links. Powerful when combined ' + 'with collective.recipe.omelette.')) parser.add_argument('-n', '--dry-run', action='store_true', dest='dryrun', - help='run the script without creating files') + help=__('run the script without creating files')) parser.add_argument('-e', '--separate', action='store_true', dest='separatemodules', - help='put documentation for each module on its own page') + help=__('put documentation for each module on its own page')) parser.add_argument('-P', '--private', action='store_true', dest='includeprivate', - help='include "_private" modules') + help=__('include "_private" modules')) parser.add_argument('-T', '--no-toc', action='store_true', dest='notoc', - help="don't create a table of contents file") + help=__("don't create a table of contents file")) parser.add_argument('-E', '--no-headings', action='store_true', dest='noheadings', - help="don't create headings for the module/package " - "packages (e.g. when the docstrings already " - "contain them)") + help=__("don't create headings for the module/package " + "packages (e.g. when the docstrings already " + "contain them)")) parser.add_argument('-M', '--module-first', action='store_true', dest='modulefirst', - help='put module documentation before submodule ' - 'documentation') + help=__('put module documentation before submodule ' + 'documentation')) parser.add_argument('--implicit-namespaces', action='store_true', dest='implicit_namespaces', - help='interpret module paths according to PEP-0420 ' - 'implicit namespaces specification') + help=__('interpret module paths according to PEP-0420 ' + 'implicit namespaces specification')) parser.add_argument('-s', '--suffix', action='store', dest='suffix', default='rst', - help='file suffix (default: rst)') + help=__('file suffix (default: rst)')) parser.add_argument('-F', '--full', action='store_true', dest='full', - help='generate a full project with sphinx-quickstart') + help=__('generate a full project with sphinx-quickstart')) parser.add_argument('-a', '--append-syspath', action='store_true', dest='append_syspath', - help='append module_path to sys.path, used when --full is given') + help=__('append module_path to sys.path, used when --full is given')) parser.add_argument('-H', '--doc-project', action='store', dest='header', - help='project name (default: root module name)') + help=__('project name (default: root module name)')) parser.add_argument('-A', '--doc-author', action='store', dest='author', - help='project author(s), used when --full is given') + help=__('project author(s), used when --full is given')) parser.add_argument('-V', '--doc-version', action='store', dest='version', - help='project version, used when --full is given') + help=__('project version, used when --full is given')) parser.add_argument('-R', '--doc-release', action='store', dest='release', - help='project release, used when --full is given, ' - 'defaults to --doc-version') + help=__('project release, used when --full is given, ' + 'defaults to --doc-version')) - group = parser.add_argument_group('extension options') + group = parser.add_argument_group(__('extension options')) for ext in EXTENSIONS: group.add_argument('--ext-%s' % ext, action='append_const', const='sphinx.ext.%s' % ext, dest='extensions', - help='enable %s extension' % ext) + help=__('enable %s extension') % ext) return parser @@ -382,6 +385,9 @@ Note: By default this script will not overwrite already created files.""") def main(argv=sys.argv[1:]): # type: (List[str]) -> int """Parse and check the command line arguments.""" + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') + parser = get_parser() args = parser.parse_args(argv) @@ -394,7 +400,7 @@ def main(argv=sys.argv[1:]): if args.suffix.startswith('.'): args.suffix = args.suffix[1:] if not path.isdir(rootpath): - print('%s is not a directory.' % rootpath, file=sys.stderr) + print(__('%s is not a directory.') % rootpath, file=sys.stderr) sys.exit(1) if not args.dryrun: ensuredir(args.destdir) diff --git a/sphinx/ext/autodoc/__init__.py b/sphinx/ext/autodoc/__init__.py index 89ca1f44c..132116d55 100644 --- a/sphinx/ext/autodoc/__init__.py +++ b/sphinx/ext/autodoc/__init__.py @@ -25,7 +25,7 @@ from sphinx.deprecation import RemovedInSphinx20Warning from sphinx.ext.autodoc.importer import mock, import_object, get_object_members from sphinx.ext.autodoc.importer import _MockImporter # to keep compatibility # NOQA from sphinx.ext.autodoc.inspector import format_annotation, formatargspec # to keep compatibility # NOQA -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.pycode import ModuleAnalyzer, PycodeError from sphinx.util import logging from sphinx.util import rpartition, force_decode @@ -340,7 +340,7 @@ class Documenter(object): explicit_modname, path, base, args, retann = \ py_ext_sig_re.match(self.name).groups() # type: ignore except AttributeError: - logger.warning('invalid signature for auto%s (%r)' % (self.objtype, self.name)) + logger.warning(__('invalid signature for auto%s (%r)') % (self.objtype, self.name)) return False # support explicit module and class name separation via :: @@ -437,7 +437,7 @@ class Documenter(object): try: args = self.format_args() except Exception as err: - logger.warning('error while formatting arguments for %s: %s' % + logger.warning(__('error while formatting arguments for %s: %s') % (self.fullname, err)) args = None @@ -560,7 +560,7 @@ class Documenter(object): if name in members: selected.append((name, members[name].value)) else: - logger.warning('missing attribute %s in object %s' % + logger.warning(__('missing attribute %s in object %s') % (name, self.fullname)) return False, sorted(selected) elif self.options.inherited_members: @@ -731,9 +731,9 @@ class Documenter(object): if not self.parse_name(): # need a module to import logger.warning( - 'don\'t know which module to import for autodocumenting ' - '%r (try placing a "module" or "currentmodule" directive ' - 'in the document, or giving an explicit module name)' % + __('don\'t know which module to import for autodocumenting ' + '%r (try placing a "module" or "currentmodule" directive ' + 'in the document, or giving an explicit module name)') % self.name) return @@ -820,15 +820,15 @@ class ModuleDocumenter(Documenter): def resolve_name(self, modname, parents, path, base): # type: (str, Any, str, Any) -> Tuple[str, List[unicode]] if modname is not None: - logger.warning('"::" in automodule name doesn\'t make sense') + logger.warning(__('"::" in automodule name doesn\'t make sense')) return (path or '') + base, [] def parse_name(self): # type: () -> bool ret = Documenter.parse_name(self) if self.args or self.retann: - logger.warning('signature arguments or return annotation ' - 'given for automodule %s' % self.fullname) + logger.warning(__('signature arguments or return annotation ' + 'given for automodule %s') % self.fullname) return ret def add_directive_header(self, sig): @@ -861,8 +861,8 @@ class ModuleDocumenter(Documenter): if not isinstance(memberlist, (list, tuple)) or not \ all(isinstance(entry, string_types) for entry in memberlist): logger.warning( - '__all__ should be a list of strings, not %r ' - '(in module %s) -- ignoring __all__' % + __('__all__ should be a list of strings, not %r ' + '(in module %s) -- ignoring __all__') % (memberlist, self.fullname)) # fall back to all members return True, safe_getmembers(self.object) @@ -874,8 +874,8 @@ class ModuleDocumenter(Documenter): ret.append((mname, safe_getattr(self.object, mname))) except AttributeError: logger.warning( - 'missing attribute mentioned in :members: or __all__: ' - 'module %s, attribute %s' % + __('missing attribute mentioned in :members: or __all__: ' + 'module %s, attribute %s') % (safe_getattr(self.object, '__name__', '???'), mname)) return False, ret @@ -1193,6 +1193,7 @@ class ClassDocumenter(DocstringSignatureMixin, ModuleLevelDocumenter): # type: def generate(self, more_content=None, real_modname=None, check_module=False, all_members=False): + # type: (Any, str, bool, bool) -> None # Do not pass real_modname and use the name from the __module__ # attribute of the class. # If a class gets imported into the module real_modname @@ -1325,6 +1326,7 @@ class AttributeDocumenter(DocstringStripSignatureMixin, ClassLevelDocumenter): @staticmethod def is_function_or_method(obj): + # type: (Any) -> bool return inspect.isfunction(obj) or inspect.isbuiltin(obj) or inspect.ismethod(obj) @classmethod @@ -1425,18 +1427,22 @@ class InstanceAttributeDocumenter(AttributeDocumenter): class DeprecatedDict(dict): def __init__(self, message): + # type: (str) -> None self.message = message super(DeprecatedDict, self).__init__() def __setitem__(self, key, value): + # type: (unicode, Any) -> None warnings.warn(self.message, RemovedInSphinx20Warning) super(DeprecatedDict, self).__setitem__(key, value) def setdefault(self, key, default=None): + # type: (unicode, Any) -> None warnings.warn(self.message, RemovedInSphinx20Warning) super(DeprecatedDict, self).setdefault(key, default) - def update(self, other=None): + def update(self, other=None): # type: ignore + # type: (Dict) -> None warnings.warn(self.message, RemovedInSphinx20Warning) super(DeprecatedDict, self).update(other) diff --git a/sphinx/ext/autodoc/importer.py b/sphinx/ext/autodoc/importer.py index 00a55c4c5..2cc8cbd1f 100644 --- a/sphinx/ext/autodoc/importer.py +++ b/sphinx/ext/autodoc/importer.py @@ -90,6 +90,7 @@ class _MockImporter(object): sys.meta_path.insert(0, self) def disable(self): + # type: () -> None # remove `self` from `sys.meta_path` to disable import hook sys.meta_path = [i for i in sys.meta_path if i is not self] # remove mocked modules from sys.modules to avoid side effects after @@ -130,6 +131,7 @@ def mock(names): def import_module(modname, warningiserror=False): + # type: (str, bool) -> Any """ Call __import__(modname), convert exceptions to ImportError """ diff --git a/sphinx/ext/autosectionlabel.py b/sphinx/ext/autosectionlabel.py index 6a3ff1422..5713828a4 100644 --- a/sphinx/ext/autosectionlabel.py +++ b/sphinx/ext/autosectionlabel.py @@ -11,9 +11,16 @@ from docutils import nodes +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.nodes import clean_astext +if False: + # For type annotation + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + + logger = logging.getLogger(__name__) if False: @@ -37,8 +44,8 @@ def register_sections_as_label(app, document): sectname = clean_astext(node[0]) if name in labels: - logger.warning('duplicate label %s, ' % name + 'other instance ' - 'in ' + app.env.doc2path(labels[name][0]), + logger.warning(__('duplicate label %s, other instance in %s'), + name, app.env.doc2path(labels[name][0]), location=node) anonlabels[name] = docname, labelid diff --git a/sphinx/ext/autosummary/__init__.py b/sphinx/ext/autosummary/__init__.py index bf2c6fba0..d9cf47535 100644 --- a/sphinx/ext/autosummary/__init__.py +++ b/sphinx/ext/autosummary/__init__.py @@ -75,6 +75,7 @@ from sphinx.environment.adapters.toctree import TocTree from sphinx.ext.autodoc import get_documenters from sphinx.ext.autodoc.directive import DocumenterBridge, Options from sphinx.ext.autodoc.importer import import_module +from sphinx.locale import __ from sphinx.pycode import ModuleAnalyzer, PycodeError from sphinx.util import import_object, rst, logging from sphinx.util.docutils import NullReporter, new_document @@ -108,6 +109,7 @@ def process_autosummary_toc(app, doctree): crawled = {} def crawl_toc(node, depth=1): + # type: (nodes.Node, int) -> None crawled[node] = True for j, subnode in enumerate(node): try: @@ -166,6 +168,7 @@ _app = None # type: Sphinx class FakeDirective(DocumenterBridge): def __init__(self): + # type: () -> None super(FakeDirective, self).__init__({}, None, Options(), 0) # type: ignore @@ -655,14 +658,14 @@ def process_generate_options(app): from sphinx.ext.autosummary.generate import generate_autosummary_docs - ext = app.config.source_suffix + ext = list(app.config.source_suffix) genfiles = [genfile + (not genfile.endswith(tuple(ext)) and ext[0] or '') for genfile in genfiles] suffix = get_rst_suffix(app) if suffix is None: - logger.warning('autosummary generats .rst files internally. ' - 'But your source_suffix does not contain .rst. Skipped.') + logger.warning(__('autosummary generats .rst files internally. ' + 'But your source_suffix does not contain .rst. Skipped.')) return generate_autosummary_docs(genfiles, builder=app.builder, diff --git a/sphinx/ext/autosummary/generate.py b/sphinx/ext/autosummary/generate.py index 2818ab3c7..4c9175a5d 100644 --- a/sphinx/ext/autosummary/generate.py +++ b/sphinx/ext/autosummary/generate.py @@ -21,6 +21,7 @@ from __future__ import print_function import argparse import codecs +import locale import os import pydoc import re @@ -29,10 +30,12 @@ import sys from jinja2 import FileSystemLoader, TemplateNotFound from jinja2.sandbox import SandboxedEnvironment +import sphinx.locale from sphinx import __display_version__ from sphinx import package_dir from sphinx.ext.autosummary import import_by_name, get_documenter from sphinx.jinja2glue import BuiltinTemplateLoader +from sphinx.locale import __ from sphinx.registry import SphinxComponentRegistry from sphinx.util.inspect import safe_getattr from sphinx.util.osutil import ensuredir @@ -82,6 +85,7 @@ def _simple_warn(msg): def _underline(title, line='='): + # type: (unicode, unicode) -> unicode if '\n' in title: raise ValueError('Can only underline single lines') return title + '\n' + line * len(title) @@ -98,11 +102,11 @@ def generate_autosummary_docs(sources, output_dir=None, suffix='.rst', showed_sources = list(sorted(sources)) if len(showed_sources) > 20: showed_sources = showed_sources[:10] + ['...'] + showed_sources[-10:] - info('[autosummary] generating autosummary for: %s' % + info(__('[autosummary] generating autosummary for: %s') % ', '.join(showed_sources)) if output_dir: - info('[autosummary] writing to %s' % output_dir) + info(__('[autosummary] writing to %s') % output_dir) if base_path is not None: sources = [os.path.join(base_path, filename) for filename in sources] @@ -360,8 +364,8 @@ def get_parser(): # type: () -> argparse.ArgumentParser parser = argparse.ArgumentParser( usage='%(prog)s [OPTIONS] <SOURCE_FILE>...', - epilog='For more information, visit <http://sphinx-doc.org/>.', - description=""" + epilog=__('For more information, visit <http://sphinx-doc.org/>.'), + description=__(""" Generate ReStructuredText using autosummary directives. sphinx-autogen is a frontend to sphinx.ext.autosummary.generate. It generates @@ -372,35 +376,38 @@ The format of the autosummary directive is documented in the ``sphinx.ext.autosummary`` Python module and can be read using:: pydoc sphinx.ext.autosummary -""") +""")) parser.add_argument('--version', action='version', dest='show_version', version='%%(prog)s %s' % __display_version__) parser.add_argument('source_file', nargs='+', - help='source files to generate rST files for') + help=__('source files to generate rST files for')) parser.add_argument('-o', '--output-dir', action='store', dest='output_dir', - help='directory to place all output in') + help=__('directory to place all output in')) parser.add_argument('-s', '--suffix', action='store', dest='suffix', default='rst', - help='default suffix for files (default: ' - '%(default)s)') + help=__('default suffix for files (default: ' + '%(default)s)')) parser.add_argument('-t', '--templates', action='store', dest='templates', default=None, - help='custom template directory (default: ' - '%(default)s)') + help=__('custom template directory (default: ' + '%(default)s)')) parser.add_argument('-i', '--imported-members', action='store_true', dest='imported_members', default=False, - help='document imported members (default: ' - '%(default)s)') + help=__('document imported members (default: ' + '%(default)s)')) return parser def main(argv=sys.argv[1:]): # type: (List[str]) -> None + locale.setlocale(locale.LC_ALL, '') + sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx') + app = DummyApplication() setup_documenters(app) args = get_parser().parse_args(argv) diff --git a/sphinx/ext/coverage.py b/sphinx/ext/coverage.py index 6c9acfc7d..351108faa 100644 --- a/sphinx/ext/coverage.py +++ b/sphinx/ext/coverage.py @@ -20,6 +20,7 @@ from six.moves import cPickle as pickle import sphinx from sphinx.builders import Builder +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.inspect import safe_getattr @@ -45,7 +46,7 @@ def compile_regex_list(name, exps): try: lst.append(re.compile(exp)) except Exception: - logger.warning('invalid regex %r in %s', exp, name) + logger.warning(__('invalid regex %r in %s'), exp, name) return lst @@ -54,8 +55,8 @@ class CoverageBuilder(Builder): Evaluates coverage of code in the documentation. """ name = 'coverage' - epilog = ('Testing of coverage in the sources finished, look at the ' - 'results in %(outdir)s/python.txt.') + epilog = __('Testing of coverage in the sources finished, look at the ' + 'results in %(outdir)s/python.txt.') def init(self): # type: () -> None @@ -69,7 +70,7 @@ class CoverageBuilder(Builder): try: self.c_regexes.append((name, re.compile(exp))) except Exception: - logger.warning('invalid regex %r in coverage_c_regexes', exp) + logger.warning(__('invalid regex %r in coverage_c_regexes'), exp) self.c_ignorexps = {} # type: Dict[unicode, List[Pattern]] for (name, exps) in iteritems(self.config.coverage_ignore_c_items): @@ -151,7 +152,7 @@ class CoverageBuilder(Builder): try: mod = __import__(mod_name, fromlist=['foo']) except ImportError as err: - logger.warning('module %s could not be imported: %s', mod_name, err) + logger.warning(__('module %s could not be imported: %s'), mod_name, err) self.py_undoc[mod_name] = {'error': err} continue diff --git a/sphinx/ext/doctest.py b/sphinx/ext/doctest.py index 2a889900d..255a38f6d 100644 --- a/sphinx/ext/doctest.py +++ b/sphinx/ext/doctest.py @@ -26,7 +26,7 @@ from six import itervalues, StringIO, binary_type, text_type, PY2 import sphinx from sphinx.builders import Builder -from sphinx.locale import _ +from sphinx.locale import __ from sphinx.util import force_decode, logging from sphinx.util.console import bold # type: ignore from sphinx.util.nodes import set_source_info @@ -129,12 +129,12 @@ class TestDirective(Directive): prefix, option_name = option[0], option[1:] if prefix not in '+-': self.state.document.reporter.warning( - _("missing '+' or '-' in '%s' option.") % option, + __("missing '+' or '-' in '%s' option.") % option, line=self.lineno) continue if option_name not in doctest.OPTIONFLAGS_BY_NAME: self.state.document.reporter.warning( - _("'%s' is not a valid option.") % option_name, + __("'%s' is not a valid option.") % option_name, line=self.lineno) continue flag = doctest.OPTIONFLAGS_BY_NAME[option[1:]] @@ -148,7 +148,7 @@ class TestDirective(Directive): node['options'][flag] = True # Skip the test except InvalidSpecifier: self.state.document.reporter.warning( - _("'%s' is not a valid pyversion option") % spec, + __("'%s' is not a valid pyversion option") % spec, line=self.lineno) return [node] @@ -214,7 +214,7 @@ class TestGroup(object): if self.tests and len(self.tests[-1]) == 2: self.tests[-1][1] = code else: - raise RuntimeError('invalid TestCode type') + raise RuntimeError(__('invalid TestCode type')) def __repr__(self): # type: ignore # type: () -> unicode @@ -275,8 +275,8 @@ class DocTestBuilder(Builder): Runs test snippets in the documentation. """ name = 'doctest' - epilog = ('Testing of doctests in the sources finished, look at the ' - 'results in %(outdir)s/output.txt.') + epilog = __('Testing of doctests in the sources finished, look at the ' + 'results in %(outdir)s/output.txt.') def init(self): # type: () -> None @@ -427,7 +427,7 @@ Doctest summary filename = self.get_filename_for_node(node, docname) line_number = self.get_line_number(node) if not source: - logger.warning('no code/output in %s block at %s:%s', + logger.warning(__('no code/output in %s block at %s:%s'), node.get('testnodetype', 'doctest'), filename, line_number) code = TestCode(source, type=node.get('testnodetype', 'doctest'), @@ -518,7 +518,7 @@ Doctest summary doctest_encode(code[0].code, self.env.config.source_encoding), {}, # type: ignore # NOQA group.name, code[0].filename, code[0].lineno) except Exception: - logger.warning('ignoring invalid doctest code: %r', code[0].code, + logger.warning(__('ignoring invalid doctest code: %r'), code[0].code, location=(code[0].filename, code[0].lineno)) continue if not test.examples: diff --git a/sphinx/ext/extlinks.py b/sphinx/ext/extlinks.py index dbd55e4b4..53b051c45 100644 --- a/sphinx/ext/extlinks.py +++ b/sphinx/ext/extlinks.py @@ -8,11 +8,11 @@ This adds a new config value called ``extlinks`` that is created like this:: - extlinks = {'exmpl': ('http://example.com/%s.html', prefix), ...} + extlinks = {'exmpl': ('https://example.invalid/%s.html', prefix), ...} Now you can use e.g. :exmpl:`foo` in your documents. This will create a - link to ``http://example.com/foo.html``. The link caption depends on the - *prefix* value given: + link to ``https://example.invalid/foo.html``. The link caption depends on + the *prefix* value given: - If it is ``None``, the caption will be the full URL. - If it is a string (empty or not), the caption will be the prefix prepended @@ -24,15 +24,25 @@ :license: BSD, see LICENSE for details. """ +from typing import TYPE_CHECKING + from docutils import nodes, utils from six import iteritems import sphinx from sphinx.util.nodes import split_explicit_title +if TYPE_CHECKING: + from typing import Any, Dict, List, Tuple # NOQA + from docutils.parsers.rst.states import Inliner # NOQA + from sphinx.application import Sphinx # NOQA + from sphinx.util.typing import RoleFunction # NOQA + def make_link_role(base_url, prefix): + # type: (unicode, unicode) -> RoleFunction def role(typ, rawtext, text, lineno, inliner, options={}, content=[]): + # type: (unicode, unicode, unicode, int, Inliner, Dict, List[unicode]) -> Tuple[List[nodes.Node], List[nodes.Node]] # NOQA text = utils.unescape(text) has_explicit_title, title, part = split_explicit_title(text) try: @@ -54,11 +64,13 @@ def make_link_role(base_url, prefix): def setup_link_roles(app): + # type: (Sphinx) -> None for name, (base_url, prefix) in iteritems(app.config.extlinks): app.add_role(name, make_link_role(base_url, prefix)) def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] app.add_config_value('extlinks', {}, 'env') app.connect('builder-inited', setup_link_roles) return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/githubpages.py b/sphinx/ext/githubpages.py index b267b1740..f97c810f8 100644 --- a/sphinx/ext/githubpages.py +++ b/sphinx/ext/githubpages.py @@ -10,16 +10,24 @@ """ import os +from typing import TYPE_CHECKING import sphinx +if TYPE_CHECKING: + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + from sphinx.environment import BuildEnvironment # NOQA + def create_nojekyll(app, env): + # type: (Sphinx, BuildEnvironment) -> None if app.builder.format == 'html': path = os.path.join(app.builder.outdir, '.nojekyll') open(path, 'wt').close() def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] app.connect('env-updated', create_nojekyll) return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/graphviz.py b/sphinx/ext/graphviz.py index 08707b733..bc348f4e1 100644 --- a/sphinx/ext/graphviz.py +++ b/sphinx/ext/graphviz.py @@ -277,7 +277,7 @@ def render_dot_html(self, node, code, options, prefix='graphviz', "'svg', but is %r") % format) fname, outfn = render_dot(self, code, options, format, prefix) except GraphvizError as exc: - logger.warning('dot code %r: ' % code + str(exc)) + logger.warning(__('dot code %r: %s'), code, text_type(exc)) raise nodes.SkipNode if fname is None: @@ -321,7 +321,7 @@ def render_dot_latex(self, node, code, options, prefix='graphviz'): try: fname, outfn = render_dot(self, code, options, 'pdf', prefix) except GraphvizError as exc: - logger.warning('dot code %r: ' % code + str(exc)) + logger.warning(__('dot code %r: %s'), code, text_type(exc)) raise nodes.SkipNode is_inline = self.is_inline(node) @@ -359,7 +359,7 @@ def render_dot_texinfo(self, node, code, options, prefix='graphviz'): try: fname, outfn = render_dot(self, code, options, 'png', prefix) except GraphvizError as exc: - logger.warning('dot code %r: ' % code + str(exc)) + logger.warning(__('dot code %r: %s'), code, text_type(exc)) raise nodes.SkipNode if fname is not None: self.body.append('@image{%s,,,[graphviz],png}\n' % fname[:-4]) diff --git a/sphinx/ext/imgconverter.py b/sphinx/ext/imgconverter.py index 95f579e36..8546593a7 100644 --- a/sphinx/ext/imgconverter.py +++ b/sphinx/ext/imgconverter.py @@ -28,6 +28,7 @@ logger = logging.getLogger(__name__) class ImagemagickConverter(ImageConverter): conversion_rules = [ ('image/svg+xml', 'image/png'), + ('image/gif', 'image/png'), ('application/pdf', 'image/png'), ] @@ -52,6 +53,10 @@ class ImagemagickConverter(ImageConverter): # type: (unicode, unicode) -> bool """Converts the image to expected one.""" try: + if _from.lower().endswith('.gif'): + # when target is GIF format, pick the first frame + _from += '[0]' + args = ([self.config.image_converter] + self.config.image_converter_args + [_from, _to]) diff --git a/sphinx/ext/imgmath.py b/sphinx/ext/imgmath.py index 5f9d7a1e2..a1faf2c1b 100644 --- a/sphinx/ext/imgmath.py +++ b/sphinx/ext/imgmath.py @@ -25,7 +25,7 @@ import sphinx from sphinx.errors import SphinxError, ExtensionError from sphinx.ext.mathbase import get_node_equation_number from sphinx.ext.mathbase import setup_math as mathbase_setup, wrap_displaymath -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.osutil import ensuredir, ENOENT, cd from sphinx.util.png import read_png_depth, write_png_depth @@ -141,8 +141,8 @@ def compile_math(latex, builder): except OSError as err: if err.errno != ENOENT: # No such file or directory raise - logger.warning('LaTeX command %r cannot be run (needed for math ' - 'display), check the imgmath_latex setting', + logger.warning(__('LaTeX command %r cannot be run (needed for math ' + 'display), check the imgmath_latex setting'), builder.config.imgmath_latex) raise InvokeError @@ -161,8 +161,8 @@ def convert_dvi_to_image(command, name): except OSError as err: if err.errno != ENOENT: # No such file or directory raise - logger.warning('%s command %r cannot be run (needed for math ' - 'display), check the imgmath_%s setting', + logger.warning(__('%s command %r cannot be run (needed for math ' + 'display), check the imgmath_%s setting'), name, command[0], name) raise InvokeError @@ -300,7 +300,7 @@ def html_visit_math(self, node): sm = nodes.system_message(msg, type='WARNING', level=2, backrefs=[], source=node['latex']) sm.walkabout(self) - logger.warning('display latex %r: %s', node['latex'], msg) + logger.warning(__('display latex %r: %s'), node['latex'], msg) raise nodes.SkipNode if fname is None: # something failed -- use text-only as a bad substitute @@ -328,7 +328,7 @@ def html_visit_displaymath(self, node): sm = nodes.system_message(msg, type='WARNING', level=2, backrefs=[], source=node['latex']) sm.walkabout(self) - logger.warning('inline latex %r: %s', node['latex'], msg) + logger.warning(__('inline latex %r: %s'), node['latex'], msg) raise nodes.SkipNode self.body.append(self.starttag(node, 'div', CLASS='math')) self.body.append('<p>') diff --git a/sphinx/ext/intersphinx.py b/sphinx/ext/intersphinx.py index 372f7baab..af219d16a 100644 --- a/sphinx/ext/intersphinx.py +++ b/sphinx/ext/intersphinx.py @@ -41,7 +41,7 @@ from six.moves.urllib.parse import urlsplit, urlunsplit import sphinx from sphinx.builders.html import INVENTORY_FILENAME from sphinx.deprecation import RemovedInSphinx20Warning -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import requests, logging from sphinx.util.inventory import InventoryFile @@ -64,31 +64,33 @@ class InventoryAdapter(object): """Inventory adapter for environment""" def __init__(self, env): + # type: (BuildEnvironment) -> None self.env = env if not hasattr(env, 'intersphinx_cache'): - self.env.intersphinx_cache = {} - self.env.intersphinx_inventory = {} - self.env.intersphinx_named_inventory = {} + self.env.intersphinx_cache = {} # type: ignore + self.env.intersphinx_inventory = {} # type: ignore + self.env.intersphinx_named_inventory = {} # type: ignore @property def cache(self): # type: () -> Dict[unicode, Tuple[unicode, int, Inventory]] - return self.env.intersphinx_cache + return self.env.intersphinx_cache # type: ignore @property def main_inventory(self): # type: () -> Inventory - return self.env.intersphinx_inventory + return self.env.intersphinx_inventory # type: ignore @property def named_inventory(self): # type: () -> Dict[unicode, Inventory] - return self.env.intersphinx_named_inventory + return self.env.intersphinx_named_inventory # type: ignore def clear(self): - self.env.intersphinx_inventory.clear() - self.env.intersphinx_named_inventory.clear() + # type: () -> None + self.env.intersphinx_inventory.clear() # type: ignore + self.env.intersphinx_named_inventory.clear() # type: ignore def _strip_basic_auth(url): @@ -221,7 +223,7 @@ def load_mappings(app): # new format name, (uri, inv) = key, value if not isinstance(name, string_types): - logger.warning('intersphinx identifier %r is not string. Ignored', name) + logger.warning(__('intersphinx identifier %r is not string. Ignored'), name) continue else: # old format, no name @@ -263,8 +265,8 @@ def load_mappings(app): for fail in failures: logger.info(*fail) else: - logger.warning("failed to reach any of the inventories " - "with the following issues:") + logger.warning(__("failed to reach any of the inventories " + "with the following issues:")) for fail in failures: logger.warning(*fail) @@ -371,7 +373,11 @@ def setup(app): app.add_config_value('intersphinx_timeout', None, False) app.connect('missing-reference', missing_reference) app.connect('builder-inited', load_mappings) - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} + return { + 'version': sphinx.__display_version__, + 'env_version': 1, + 'parallel_read_safe': True + } def debug(argv): @@ -401,6 +407,7 @@ def inspect_main(argv): config = MockConfig() def warn(self, msg): + # type: (unicode) -> None print(msg, file=sys.stderr) filename = argv[0] diff --git a/sphinx/ext/jsmath.py b/sphinx/ext/jsmath.py index 05d5dc290..e523b54c8 100644 --- a/sphinx/ext/jsmath.py +++ b/sphinx/ext/jsmath.py @@ -10,6 +10,8 @@ :license: BSD, see LICENSE for details. """ +from typing import TYPE_CHECKING + from docutils import nodes import sphinx @@ -19,13 +21,20 @@ from sphinx.ext.mathbase import setup_math as mathbase_setup from sphinx.locale import _ +if TYPE_CHECKING: + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + + def html_visit_math(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None self.body.append(self.starttag(node, 'span', '', CLASS='math notranslate')) self.body.append(self.encode(node['latex']) + '</span>') raise nodes.SkipNode def html_visit_displaymath(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None if node['nowrap']: self.body.append(self.starttag(node, 'div', CLASS='math notranslate')) self.body.append(self.encode(node['latex'])) @@ -53,6 +62,7 @@ def html_visit_displaymath(self, node): def builder_inited(app): + # type: (Sphinx) -> None if not app.config.jsmath_path: raise ExtensionError('jsmath_path config value must be set for the ' 'jsmath extension to work') @@ -60,6 +70,7 @@ def builder_inited(app): def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] try: mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None)) except ExtensionError: diff --git a/sphinx/ext/mathbase.py b/sphinx/ext/mathbase.py index 3cc734537..939247638 100644 --- a/sphinx/ext/mathbase.py +++ b/sphinx/ext/mathbase.py @@ -24,6 +24,7 @@ if False: # For type annotation from typing import Any, Callable, Dict, Iterable, List, Tuple # NOQA from docutils.parsers.rst.states import Inliner # NOQA + from docutils.writers.html4css1 import Writer # NOQA from sphinx.application import Sphinx # NOQA from sphinx.builders import Builder # NOQA from sphinx.environment import BuildEnvironment # NOQA @@ -97,7 +98,7 @@ class MathDomain(Domain): eqref_format = env.config.math_eqref_format or "({number})" title = nodes.Text(eqref_format.format(number=number)) except KeyError as exc: - logger.warning('Invalid math_eqref_format: %r', exc, + logger.warning(__('Invalid math_eqref_format: %r'), exc, location=node) title = nodes.Text("(%d)" % number) title = nodes.Text("(%d)" % number) @@ -136,6 +137,7 @@ class MathDomain(Domain): def get_node_equation_number(writer, node): + # type: (Writer, nodes.Node) -> unicode if writer.builder.config.math_numfig and writer.builder.config.numfig: figtype = 'displaymath' if writer.builder.name == 'singlehtml': @@ -307,7 +309,7 @@ def latex_visit_eqref(self, node): ref = '\\ref{%s}' % label self.body.append(eqref_format.format(number=ref)) except KeyError as exc: - logger.warning('Invalid math_eqref_format: %r', exc, + logger.warning(__('Invalid math_eqref_format: %r'), exc, location=node) self.body.append('\\eqref{%s}' % label) else: diff --git a/sphinx/ext/mathjax.py b/sphinx/ext/mathjax.py index 6c5d0a642..9f8013b2b 100644 --- a/sphinx/ext/mathjax.py +++ b/sphinx/ext/mathjax.py @@ -3,7 +3,7 @@ sphinx.ext.mathjax ~~~~~~~~~~~~~~~~~~ - Allow `MathJax <http://mathjax.org/>`_ to be used to display math in + Allow `MathJax <https://www.mathjax.org/>`_ to be used to display math in Sphinx's HTML writer -- requires the MathJax JavaScript library on your webserver/computer. @@ -11,6 +11,8 @@ :license: BSD, see LICENSE for details. """ +from typing import TYPE_CHECKING + from docutils import nodes import sphinx @@ -19,8 +21,13 @@ from sphinx.ext.mathbase import get_node_equation_number from sphinx.ext.mathbase import setup_math as mathbase_setup from sphinx.locale import _ +if TYPE_CHECKING: + from typing import Any, Dict # NOQA + from sphinx.application import Sphinx # NOQA + def html_visit_math(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None self.body.append(self.starttag(node, 'span', '', CLASS='math notranslate')) self.body.append(self.builder.config.mathjax_inline[0] + self.encode(node['latex']) + @@ -29,6 +36,7 @@ def html_visit_math(self, node): def html_visit_displaymath(self, node): + # type: (nodes.NodeVisitor, nodes.Node) -> None self.body.append(self.starttag(node, 'div', CLASS='math notranslate')) if node['nowrap']: self.body.append(self.encode(node['latex'])) @@ -61,6 +69,7 @@ def html_visit_displaymath(self, node): def builder_inited(app): + # type: (Sphinx) -> None if not app.config.mathjax_path: raise ExtensionError('mathjax_path config value must be set for the ' 'mathjax extension to work') @@ -68,13 +77,14 @@ def builder_inited(app): def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] try: mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None)) except ExtensionError: raise ExtensionError('sphinx.ext.mathjax: other math package is already loaded') # more information for mathjax secure url is here: - # http://docs.mathjax.org/en/latest/start.html#secure-access-to-the-cdn + # https://docs.mathjax.org/en/latest/start.html#secure-access-to-the-cdn app.add_config_value('mathjax_path', 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?' 'config=TeX-AMS-MML_HTMLorMML', False) diff --git a/sphinx/ext/napoleon/__init__.py b/sphinx/ext/napoleon/__init__.py index b65f7f2a1..b968f5948 100644 --- a/sphinx/ext/napoleon/__init__.py +++ b/sphinx/ext/napoleon/__init__.py @@ -47,9 +47,10 @@ class Config(object): napoleon_use_param = True napoleon_use_rtype = True napoleon_use_keyword = True + napoleon_custom_sections = None .. _Google style: - http://google.github.io/styleguide/pyguide.html + https://google.github.io/styleguide/pyguide.html .. _NumPy style: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt @@ -241,6 +242,19 @@ class Config(object): :returns: *bool* -- True if successful, False otherwise + napoleon_custom_sections : :obj:`list` (Defaults to None) + Add a list of custom sections to include, expanding the list of parsed sections. + + The entries can either be strings or tuples, depending on the intention: + * To create a custom "generic" section, just pass a string. + * To create an alias for an existing section, pass a tuple containing the + alias name and the original, in that order. + + If an entry is just a string, it is interpreted as a header for a generic + section. If the entry is a tuple/list/indexed container, the first entry + is the name of the section, the second is the section key to emulate. + + """ _config_values = { 'napoleon_google_docstring': (True, 'env'), @@ -254,7 +268,8 @@ class Config(object): 'napoleon_use_ivar': (False, 'env'), 'napoleon_use_param': (True, 'env'), 'napoleon_use_rtype': (True, 'env'), - 'napoleon_use_keyword': (True, 'env') + 'napoleon_use_keyword': (True, 'env'), + 'napoleon_custom_sections': (None, 'env') } def __init__(self, **settings): @@ -310,14 +325,13 @@ def _patch_python_domain(): pass else: import sphinx.domains.python - import sphinx.locale - l_ = sphinx.locale.lazy_gettext + from sphinx.locale import _ for doc_field in sphinx.domains.python.PyObject.doc_field_types: if doc_field.name == 'parameter': doc_field.names = ('param', 'parameter', 'arg', 'argument') break sphinx.domains.python.PyObject.doc_field_types.append( - PyTypedField('keyword', label=l_('Keyword Arguments'), + PyTypedField('keyword', label=_('Keyword Arguments'), names=('keyword', 'kwarg', 'kwparam'), typerolename='obj', typenames=('paramtype', 'kwtype'), can_collapse=True)) diff --git a/sphinx/ext/napoleon/docstring.py b/sphinx/ext/napoleon/docstring.py index b349c761f..530cec203 100644 --- a/sphinx/ext/napoleon/docstring.py +++ b/sphinx/ext/napoleon/docstring.py @@ -14,6 +14,7 @@ import collections import inspect import re +from functools import partial from six import string_types, u from six.moves import range @@ -140,13 +141,19 @@ class GoogleDocstring(UnicodeMixin): self._sections = { 'args': self._parse_parameters_section, 'arguments': self._parse_parameters_section, + 'attention': partial(self._parse_admonition, 'attention'), 'attributes': self._parse_attributes_section, + 'caution': partial(self._parse_admonition, 'caution'), + 'danger': partial(self._parse_admonition, 'danger'), + 'error': partial(self._parse_admonition, 'error'), 'example': self._parse_examples_section, 'examples': self._parse_examples_section, + 'hint': partial(self._parse_admonition, 'hint'), + 'important': partial(self._parse_admonition, 'important'), 'keyword args': self._parse_keyword_arguments_section, 'keyword arguments': self._parse_keyword_arguments_section, 'methods': self._parse_methods_section, - 'note': self._parse_note_section, + 'note': partial(self._parse_admonition, 'note'), 'notes': self._parse_notes_section, 'other parameters': self._parse_other_parameters_section, 'parameters': self._parse_parameters_section, @@ -155,13 +162,17 @@ class GoogleDocstring(UnicodeMixin): 'raises': self._parse_raises_section, 'references': self._parse_references_section, 'see also': self._parse_see_also_section, - 'todo': self._parse_todo_section, - 'warning': self._parse_warning_section, - 'warnings': self._parse_warning_section, + 'tip': partial(self._parse_admonition, 'tip'), + 'todo': partial(self._parse_admonition, 'todo'), + 'warning': partial(self._parse_admonition, 'warning'), + 'warnings': partial(self._parse_admonition, 'warning'), 'warns': self._parse_warns_section, 'yield': self._parse_yields_section, 'yields': self._parse_yields_section, } # type: Dict[unicode, Callable] + + self._load_custom_sections() + self._parse() def __unicode__(self): @@ -522,6 +533,23 @@ class GoogleDocstring(UnicodeMixin): line and not self._is_indented(line, self._section_indent))) + def _load_custom_sections(self): + # type: () -> None + + if self._config.napoleon_custom_sections is not None: + for entry in self._config.napoleon_custom_sections: + if isinstance(entry, string_types): + # if entry is just a label, add to sections list, + # using generic section logic. + self._sections[entry.lower()] = self._parse_custom_generic_section + else: + # otherwise, assume entry is container; + # [0] is new section, [1] is the section to alias. + # in the case of key mismatch, just handle as generic section. + self._sections[entry[0].lower()] = \ + self._sections.get(entry[1].lower(), + self._parse_custom_generic_section) + def _parse(self): # type: () -> None self._parsed_lines = self._consume_empty() @@ -550,10 +578,18 @@ class GoogleDocstring(UnicodeMixin): lines = self._consume_to_next_section() self._parsed_lines.extend(lines) + def _parse_admonition(self, admonition, section): + # type (unicode, unicode) -> List[unicode] + lines = self._consume_to_next_section() + return self._format_admonition(admonition, lines) + def _parse_attribute_docstring(self): # type: () -> List[unicode] _type, _desc = self._consume_inline_attribute() - return self._format_field('', _type, _desc) + lines = self._format_field('', '', _desc) + if _type: + lines.extend(['', ':type: %s' % _type]) + return lines def _parse_attributes_section(self, section): # type: (unicode) -> List[unicode] @@ -566,8 +602,11 @@ class GoogleDocstring(UnicodeMixin): lines.append(':vartype %s: %s' % (_name, _type)) else: lines.extend(['.. attribute:: ' + _name, '']) - fields = self._format_field('', _type, _desc) + fields = self._format_field('', '', _desc) lines.extend(self._indent(fields, 3)) + if _type: + lines.append('') + lines.extend(self._indent([':type: %s' % _type], 3)) lines.append('') if self._config.napoleon_use_ivar: lines.append('') @@ -578,6 +617,10 @@ class GoogleDocstring(UnicodeMixin): use_admonition = self._config.napoleon_use_admonition_for_examples return self._parse_generic_section(section, use_admonition) + def _parse_custom_generic_section(self, section): + # for now, no admonition for simple custom sections + return self._parse_generic_section(section, False) + def _parse_usage_section(self, section): # type: (unicode) -> List[unicode] header = ['.. rubric:: Usage:', ''] # type: List[unicode] @@ -621,11 +664,6 @@ class GoogleDocstring(UnicodeMixin): lines.append('') return lines - def _parse_note_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('note', lines) - def _parse_notes_section(self, section): # type: (unicode) -> List[unicode] use_admonition = self._config.napoleon_use_admonition_for_notes @@ -717,19 +755,8 @@ class GoogleDocstring(UnicodeMixin): return lines def _parse_see_also_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('seealso', lines) - - def _parse_todo_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('todo', lines) - - def _parse_warning_section(self, section): - # type: (unicode) -> List[unicode] - lines = self._consume_to_next_section() - return self._format_admonition('warning', lines) + # type (unicode) -> List[unicode] + return self._parse_admonition('seealso', section) def _parse_warns_section(self, section): # type: (unicode) -> List[unicode] @@ -963,8 +990,9 @@ class NumpyDocstring(GoogleDocstring): items = [] def parse_item_name(text): + # type: (unicode) -> Tuple[unicode, unicode] """Match ':role:`name`' or 'name'""" - m = self._name_rgx.match(text) + m = self._name_rgx.match(text) # type: ignore if m: g = m.groups() if g[1] is None: @@ -974,6 +1002,7 @@ class NumpyDocstring(GoogleDocstring): raise ValueError("%s is not a item name" % text) def push_item(name, rest): + # type: (unicode, List[unicode]) -> None if not name: return name, role = parse_item_name(name) diff --git a/sphinx/ext/pngmath.py b/sphinx/ext/pngmath.py deleted file mode 100644 index ebb05e615..000000000 --- a/sphinx/ext/pngmath.py +++ /dev/null @@ -1,276 +0,0 @@ -# -*- coding: utf-8 -*- -""" - sphinx.ext.pngmath - ~~~~~~~~~~~~~~~~~~ - - Render math in HTML via dvipng. This extension has been deprecated; please - use sphinx.ext.imgmath instead. - - :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. - :license: BSD, see LICENSE for details. -""" - -import codecs -import posixpath -import re -import shutil -import tempfile -from hashlib import sha1 -from os import path -from subprocess import Popen, PIPE - -from docutils import nodes -from six import text_type - -import sphinx -from sphinx.errors import SphinxError, ExtensionError -from sphinx.ext.mathbase import get_node_equation_number -from sphinx.ext.mathbase import setup_math as mathbase_setup, wrap_displaymath -from sphinx.util import logging -from sphinx.util.osutil import ensuredir, ENOENT, cd -from sphinx.util.png import read_png_depth, write_png_depth -from sphinx.util.pycompat import sys_encoding - -if False: - # For type annotation - from typing import Any, Dict, Tuple # NOQA - from sphinx.application import Sphinx # NOQA - from sphinx.ext.mathbase import math as math_node, displaymath # NOQA - -logger = logging.getLogger(__name__) - - -class MathExtError(SphinxError): - category = 'Math extension error' - - def __init__(self, msg, stderr=None, stdout=None): - # type: (unicode, unicode, unicode) -> None - if stderr: - msg += '\n[stderr]\n' + stderr.decode(sys_encoding, 'replace') - if stdout: - msg += '\n[stdout]\n' + stdout.decode(sys_encoding, 'replace') - SphinxError.__init__(self, msg) - - -DOC_HEAD = r''' -\documentclass[12pt]{article} -\usepackage[utf8x]{inputenc} -\usepackage{amsmath} -\usepackage{amsthm} -\usepackage{amssymb} -\usepackage{amsfonts} -\usepackage{bm} -\pagestyle{empty} -''' - -DOC_BODY = r''' -\begin{document} -%s -\end{document} -''' - -DOC_BODY_PREVIEW = r''' -\usepackage[active]{preview} -\begin{document} -\begin{preview} -%s -\end{preview} -\end{document} -''' - -depth_re = re.compile(br'\[\d+ depth=(-?\d+)\]') - - -def render_math(self, math): - # type: (nodes.NodeVisitor, unicode) -> Tuple[unicode, int] - """Render the LaTeX math expression *math* using latex and dvipng. - - Return the filename relative to the built document and the "depth", - that is, the distance of image bottom and baseline in pixels, if the - option to use preview_latex is switched on. - - Error handling may seem strange, but follows a pattern: if LaTeX or - dvipng aren't available, only a warning is generated (since that enables - people on machines without these programs to at least build the rest - of the docs successfully). If the programs are there, however, they - may not fail since that indicates a problem in the math source. - """ - use_preview = self.builder.config.pngmath_use_preview - latex = DOC_HEAD + self.builder.config.pngmath_latex_preamble - latex += (use_preview and DOC_BODY_PREVIEW or DOC_BODY) % math - - shasum = "%s.png" % sha1(latex.encode('utf-8')).hexdigest() - relfn = posixpath.join(self.builder.imgpath, 'math', shasum) - outfn = path.join(self.builder.outdir, self.builder.imagedir, 'math', shasum) - if path.isfile(outfn): - depth = read_png_depth(outfn) - return relfn, depth - - # if latex or dvipng has failed once, don't bother to try again - if hasattr(self.builder, '_mathpng_warned_latex') or \ - hasattr(self.builder, '_mathpng_warned_dvipng'): - return None, None - - # use only one tempdir per build -- the use of a directory is cleaner - # than using temporary files, since we can clean up everything at once - # just removing the whole directory (see cleanup_tempdir) - if not hasattr(self.builder, '_mathpng_tempdir'): - tempdir = self.builder._mathpng_tempdir = tempfile.mkdtemp() - else: - tempdir = self.builder._mathpng_tempdir - - with codecs.open(path.join(tempdir, 'math.tex'), 'w', 'utf-8') as tf: - tf.write(latex) - - # build latex command; old versions of latex don't have the - # --output-directory option, so we have to manually chdir to the - # temp dir to run it. - ltx_args = [self.builder.config.pngmath_latex, '--interaction=nonstopmode'] - # add custom args from the config file - ltx_args.extend(self.builder.config.pngmath_latex_args) - ltx_args.append('math.tex') - - with cd(tempdir): - try: - p = Popen(ltx_args, stdout=PIPE, stderr=PIPE) - except OSError as err: - if err.errno != ENOENT: # No such file or directory - raise - logger.warning('LaTeX command %r cannot be run (needed for math ' - 'display), check the pngmath_latex setting', - self.builder.config.pngmath_latex) - self.builder._mathpng_warned_latex = True - return None, None - - stdout, stderr = p.communicate() - if p.returncode != 0: - raise MathExtError('latex exited with error', stderr, stdout) - - ensuredir(path.dirname(outfn)) - # use some standard dvipng arguments - dvipng_args = [self.builder.config.pngmath_dvipng] - dvipng_args += ['-o', outfn, '-T', 'tight', '-z9'] - # add custom ones from config value - dvipng_args.extend(self.builder.config.pngmath_dvipng_args) - if use_preview: - dvipng_args.append('--depth') - # last, the input file name - dvipng_args.append(path.join(tempdir, 'math.dvi')) - try: - p = Popen(dvipng_args, stdout=PIPE, stderr=PIPE) - except OSError as err: - if err.errno != ENOENT: # No such file or directory - raise - logger.warning('dvipng command %r cannot be run (needed for math ' - 'display), check the pngmath_dvipng setting', - self.builder.config.pngmath_dvipng) - self.builder._mathpng_warned_dvipng = True - return None, None - stdout, stderr = p.communicate() - if p.returncode != 0: - raise MathExtError('dvipng exited with error', stderr, stdout) - depth = None - if use_preview: - for line in stdout.splitlines(): - m = depth_re.match(line) - if m: - depth = int(m.group(1)) - write_png_depth(outfn, depth) - break - - return relfn, depth - - -def cleanup_tempdir(app, exc): - # type: (Sphinx, Exception) -> None - if exc: - return - if not hasattr(app.builder, '_mathpng_tempdir'): - return - try: - shutil.rmtree(app.builder._mathpng_tempdir) # type: ignore - except Exception: - pass - - -def get_tooltip(self, node): - # type: (nodes.NodeVisitor, math_node) -> unicode - if self.builder.config.pngmath_add_tooltips: - return ' alt="%s"' % self.encode(node['latex']).strip() - return '' - - -def html_visit_math(self, node): - # type: (nodes.NodeVisitor, math_node) -> None - try: - fname, depth = render_math(self, '$' + node['latex'] + '$') - except MathExtError as exc: - msg = text_type(exc) - sm = nodes.system_message(msg, type='WARNING', level=2, - backrefs=[], source=node['latex']) - sm.walkabout(self) - logger.warning('display latex %r: %s', node['latex'], msg) - raise nodes.SkipNode - if fname is None: - # something failed -- use text-only as a bad substitute - self.body.append('<span class="math">%s</span>' % - self.encode(node['latex']).strip()) - else: - c = ('<img class="math" src="%s"' % fname) + get_tooltip(self, node) - if depth is not None: - c += ' style="vertical-align: %dpx"' % (-depth) - self.body.append(c + '/>') - raise nodes.SkipNode - - -def html_visit_displaymath(self, node): - # type: (nodes.NodeVisitor, displaymath) -> None - if node['nowrap']: - latex = node['latex'] - else: - latex = wrap_displaymath(node['latex'], None, - self.builder.config.math_number_all) - try: - fname, depth = render_math(self, latex) - except MathExtError as exc: - msg = text_type(exc) - sm = nodes.system_message(msg, type='WARNING', level=2, - backrefs=[], source=node['latex']) - sm.walkabout(self) - logger.warning('inline latex %r: %s', node['latex'], msg) - raise nodes.SkipNode - self.body.append(self.starttag(node, 'div', CLASS='math')) - self.body.append('<p>') - if node['number']: - number = get_node_equation_number(self, node) - self.body.append('<span class="eqno">(%s)</span>' % number) - if fname is None: - # something failed -- use text-only as a bad substitute - self.body.append('<span class="math">%s</span></p>\n</div>' % - self.encode(node['latex']).strip()) - else: - self.body.append(('<img src="%s"' % fname) + get_tooltip(self, node) + - '/></p>\n</div>') - raise nodes.SkipNode - - -def setup(app): - # type: (Sphinx) -> Dict[unicode, Any] - logger.warning('sphinx.ext.pngmath has been deprecated. ' - 'Please use sphinx.ext.imgmath instead.') - try: - mathbase_setup(app, (html_visit_math, None), (html_visit_displaymath, None)) - except ExtensionError: - raise ExtensionError('sphinx.ext.pngmath: other math package is already loaded') - - app.add_config_value('pngmath_dvipng', 'dvipng', 'html') - app.add_config_value('pngmath_latex', 'latex', 'html') - app.add_config_value('pngmath_use_preview', False, 'html') - app.add_config_value('pngmath_dvipng_args', - ['-gamma', '1.5', '-D', '110', '-bg', 'Transparent'], - 'html') - app.add_config_value('pngmath_latex_args', [], 'html') - app.add_config_value('pngmath_latex_preamble', '', 'html') - app.add_config_value('pngmath_add_tooltips', True, 'html') - app.connect('build-finished', cleanup_tempdir) - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} diff --git a/sphinx/ext/todo.py b/sphinx/ext/todo.py index fd3d5c062..c7ef16b51 100644 --- a/sphinx/ext/todo.py +++ b/sphinx/ext/todo.py @@ -19,7 +19,7 @@ from docutils.parsers.rst.directives.admonitions import BaseAdmonition import sphinx from sphinx.environment import NoUri -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.nodes import set_source_info from sphinx.util.texescape import tex_escape_map @@ -103,7 +103,7 @@ def process_todos(app, doctree): }) if env.config.todo_emit_warnings: - logger.warning("TODO entry found: %s", node[1].astext(), + logger.warning(__("TODO entry found: %s"), node[1].astext(), location=node) @@ -258,4 +258,8 @@ def setup(app): app.connect('doctree-resolved', process_todo_nodes) app.connect('env-purge-doc', purge_todos) app.connect('env-merge-info', merge_info) - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} + return { + 'version': sphinx.__display_version__, + 'env_version': 1, + 'parallel_read_safe': True + } diff --git a/sphinx/ext/viewcode.py b/sphinx/ext/viewcode.py index 059aa5ef1..d20ad78d5 100644 --- a/sphinx/ext/viewcode.py +++ b/sphinx/ext/viewcode.py @@ -240,4 +240,8 @@ def setup(app): app.connect('missing-reference', missing_reference) # app.add_config_value('viewcode_include_modules', [], 'env') # app.add_config_value('viewcode_exclude_modules', [], 'env') - return {'version': sphinx.__display_version__, 'parallel_read_safe': True} + return { + 'version': sphinx.__display_version__, + 'env_version': 1, + 'parallel_read_safe': True + } diff --git a/sphinx/extension.py b/sphinx/extension.py index aa1157ec8..732ea327c 100644 --- a/sphinx/extension.py +++ b/sphinx/extension.py @@ -17,14 +17,16 @@ from sphinx.util import logging if False: # For type annotation - from typing import Dict # NOQA + from typing import Any, Dict # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA logger = logging.getLogger(__name__) class Extension(object): def __init__(self, name, module, **kwargs): + # type: (unicode, Any, Any) -> None self.name = name self.module = module self.metadata = kwargs @@ -41,13 +43,13 @@ class Extension(object): self.parallel_write_safe = kwargs.pop('parallel_write_safe', True) -def verify_required_extensions(app, requirements): - # type: (Sphinx, Dict[unicode, unicode]) -> None +def verify_needs_extensions(app, config): + # type: (Sphinx, Config) -> None """Verify the required Sphinx extensions are loaded.""" - if requirements is None: + if config.needs_extensions is None: return - for extname, reqversion in iteritems(requirements): + for extname, reqversion in iteritems(config.needs_extensions): extension = app.extensions.get(extname) if extension is None: logger.warning(__('The %s extension is required by needs_extensions settings, ' @@ -59,3 +61,14 @@ def verify_required_extensions(app, requirements): 'version %s and therefore cannot be built with ' 'the loaded version (%s).') % (extname, reqversion, extension.version)) + + +def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] + app.connect('config-inited', verify_needs_extensions) + + return { + 'version': 'builtin', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/highlighting.py b/sphinx/highlighting.py index ac1d1118e..7d5956570 100644 --- a/sphinx/highlighting.py +++ b/sphinx/highlighting.py @@ -21,6 +21,7 @@ from pygments.util import ClassNotFound from six import text_type from sphinx.ext import doctest +from sphinx.locale import __ from sphinx.pygments_styles import SphinxStyle, NoneStyle from sphinx.util import logging from sphinx.util.pycompat import htmlescape @@ -131,7 +132,7 @@ class PygmentsBridge(object): try: lexer = lexers[lang] = get_lexer_by_name(lang, **(opts or {})) except ClassNotFound: - logger.warning('Pygments lexer name %r is not known', lang, + logger.warning(__('Pygments lexer name %r is not known'), lang, location=location) lexer = lexers['none'] else: @@ -152,8 +153,8 @@ class PygmentsBridge(object): if lang == 'default': pass # automatic highlighting failed. else: - logger.warning('Could not lex literal_block as "%s". ' - 'Highlighting skipped.', lang, + logger.warning(__('Could not lex literal_block as "%s". ' + 'Highlighting skipped.'), lang, type='misc', subtype='highlighting_failure', location=location) hlsource = highlight(source, lexers['none'], formatter) diff --git a/sphinx/io.py b/sphinx/io.py index 8f1da22bd..826da1812 100644 --- a/sphinx/io.py +++ b/sphinx/io.py @@ -16,9 +16,10 @@ from docutils.io import FileInput, NullOutput from docutils.readers import standalone from docutils.statemachine import StringList, string2lines from docutils.writers import UnfilteredWriter -from six import text_type +from six import text_type, iteritems from typing import Any, Union # NOQA +from sphinx.locale import __ from sphinx.transforms import ( ApplySourceWorkaround, ExtraTranslatableNodes, CitationReferences, DefaultSubstitutions, MoveModuleTargets, HandleCodeBlocks, SortIds, @@ -149,6 +150,7 @@ class SphinxDummyWriter(UnfilteredWriter): def SphinxDummySourceClass(source, *args, **kwargs): + # type: (Any, Any, Any) -> Any """Bypass source object as is to cheat Publisher.""" return source @@ -157,7 +159,7 @@ class SphinxBaseFileInput(FileInput): """A base class of SphinxFileInput. It supports to replace unknown Unicode characters to '?'. And it also emits - Sphinx events ``source-read`` on reading. + Sphinx events :event:`source-read` on reading. """ def __init__(self, app, env, *args, **kwds): @@ -198,7 +200,7 @@ class SphinxBaseFileInput(FileInput): if lineend == -1: lineend = len(error.object) lineno = error.object.count(b'\n', 0, error.start) + 1 - logger.warning('undecodable source characters, replacing with "?": %r', + logger.warning(__('undecodable source characters, replacing with "?": %r'), (error.object[linestart + 1:error.start] + b'>>>' + error.object[error.start:error.end] + b'<<<' + error.object[error.end:lineend]), @@ -274,14 +276,29 @@ class SphinxRSTFileInput(SphinxBaseFileInput): return lineno +class FiletypeNotFoundError(Exception): + pass + + +def get_filetype(source_suffix, filename): + # type: (Dict[unicode, unicode], unicode) -> unicode + for suffix, filetype in iteritems(source_suffix): + if filename.endswith(suffix): + # If default filetype (None), considered as restructuredtext. + return filetype or 'restructuredtext' + else: + raise FiletypeNotFoundError + + def read_doc(app, env, filename): # type: (Sphinx, BuildEnvironment, unicode) -> nodes.document """Parse a document and convert to doctree.""" - input_class = app.registry.get_source_input(filename) + filetype = get_filetype(app.config.source_suffix, filename) + input_class = app.registry.get_source_input(filetype) reader = SphinxStandaloneReader(app) source = input_class(app, env, source=None, source_path=filename, encoding=env.config.source_encoding) - parser = app.registry.create_source_parser(app, filename) + parser = app.registry.create_source_parser(app, filetype) pub = Publisher(reader=reader, parser=parser, diff --git a/sphinx/jinja2glue.py b/sphinx/jinja2glue.py index cc935d577..24dd0e0fd 100644 --- a/sphinx/jinja2glue.py +++ b/sphinx/jinja2glue.py @@ -24,7 +24,7 @@ from sphinx.util.osutil import mtimes_of_files if False: # For type annotation - from typing import Any, Callable, Dict, List, Iterator, Tuple # NOQA + from typing import Any, Callable, Dict, List, Iterator, Tuple, Union # NOQA from jinja2.environment import Environment # NOQA from sphinx.builders import Builder # NOQA from sphinx.theming import Theme # NOQA @@ -46,7 +46,7 @@ def _toint(val): def _todim(val): - # type (int or unicode) -> unicode + # type: (Union[int, unicode]) -> unicode """ Make val a css dimension. In particular the following transformations are performed: @@ -61,7 +61,7 @@ def _todim(val): return 'initial' elif str(val).isdigit(): return '0' if int(val) == 0 else '%spx' % val - return val + return val # type: ignore def _slice_index(values, slices): diff --git a/sphinx/locale/__init__.py b/sphinx/locale/__init__.py index e148f2c12..b0bf0ead1 100644 --- a/sphinx/locale/__init__.py +++ b/sphinx/locale/__init__.py @@ -10,10 +10,16 @@ """ import gettext +import locale +import warnings +from collections import defaultdict +from gettext import NullTranslations -from six import PY3, text_type +from six import text_type from six.moves import UserString +from sphinx.deprecation import RemovedInSphinx30Warning + if False: # For type annotation from typing import Any, Callable, Dict, Iterator, List, Tuple # NOQA @@ -180,6 +186,8 @@ def mygettext(string): """Used instead of _ when creating TranslationProxies, because _ is not bound yet at that time. """ + warnings.warn('sphinx.locale.mygettext() is deprecated. Please use `_()` instead.', + RemovedInSphinx30Warning) return _(string) @@ -188,119 +196,156 @@ def lazy_gettext(string): """A lazy version of `gettext`.""" # if isinstance(string, _TranslationProxy): # return string + warnings.warn('sphinx.locale.laxy_gettext() is deprecated. Please use `_()` instead.', + RemovedInSphinx30Warning) return _TranslationProxy(mygettext, string) # type: ignore -l_ = lazy_gettext - - -admonitionlabels = { - 'attention': l_('Attention'), - 'caution': l_('Caution'), - 'danger': l_('Danger'), - 'error': l_('Error'), - 'hint': l_('Hint'), - 'important': l_('Important'), - 'note': l_('Note'), - 'seealso': l_('See also'), - 'tip': l_('Tip'), - 'warning': l_('Warning'), -} # type: Dict[unicode, unicode] - -versionlabels = { - 'versionadded': l_('New in version %s'), - 'versionchanged': l_('Changed in version %s'), - 'deprecated': l_('Deprecated since version %s'), -} # type: Dict[unicode, unicode] - -# XXX Python specific -pairindextypes = { - 'module': l_('module'), - 'keyword': l_('keyword'), - 'operator': l_('operator'), - 'object': l_('object'), - 'exception': l_('exception'), - 'statement': l_('statement'), - 'builtin': l_('built-in function'), -} # Dict[unicode, _TranslationProxy] - -translators = {} # type: Dict[unicode, Any] - -if PY3: - def _(message, *args): - # type: (unicode, *Any) -> unicode - try: - if len(args) <= 1: - return translators['sphinx'].gettext(message) - else: # support pluralization - return translators['sphinx'].ngettext(message, args[0], args[1]) - except KeyError: - return message -else: - def _(message, *args): - # type: (unicode, *Any) -> unicode - try: - if len(args) <= 1: - return translators['sphinx'].ugettext(message) - else: # support pluralization - return translators['sphinx'].ungettext(message, args[0], args[1]) - except KeyError: - return message - - -def __(message, *args): - # type: (unicode, *Any) -> unicode - """A dummy wrapper to i18n'ize exceptions and command line messages. - - In future, the messages are translated using LC_MESSAGES or any other - locale settings. - """ - return message if len(args) <= 1 else args[0] +translators = defaultdict(NullTranslations) # type: Dict[Tuple[unicode, unicode], NullTranslations] # NOQA -def init(locale_dirs, language, catalog='sphinx'): - # type: (List, unicode, unicode) -> Tuple[Any, bool] +def init(locale_dirs, language, catalog='sphinx', namespace='general'): + # type: (List[unicode], unicode, unicode, unicode) -> Tuple[NullTranslations, bool] """Look for message catalogs in `locale_dirs` and *ensure* that there is at least a NullTranslations catalog set in `translators`. If called multiple times or if several ``.mo`` files are found, their contents are merged together (thus making ``init`` reentrable). """ global translators - translator = translators.get(catalog) + translator = translators.get((namespace, catalog)) # ignore previously failed attempts to find message catalogs - if isinstance(translator, gettext.NullTranslations): + if translator.__class__ is NullTranslations: translator = None # the None entry is the system's default locale path has_translation = True + if language and '_' in language: + # for language having country code (like "de_AT") + languages = [language, language.split('_')[0]] + else: + languages = [language] + # loading for dir_ in locale_dirs: try: trans = gettext.translation(catalog, localedir=dir_, # type: ignore - languages=[language]) # type: ignore + languages=languages) if translator is None: translator = trans else: - translator._catalog.update(trans._catalog) # type: ignore + translator.add_fallback(trans) except Exception: # Language couldn't be found in the specified path pass - # guarantee translators[catalog] exists + # guarantee translators[(namespace, catalog)] exists if translator is None: - translator = gettext.NullTranslations() + translator = NullTranslations() has_translation = False - translators[catalog] = translator + translators[(namespace, catalog)] = translator if hasattr(translator, 'ugettext'): - translator.gettext = translator.ugettext + translator.gettext = translator.ugettext # type: ignore return translator, has_translation -def get_translator(catalog='sphinx'): - # type: (unicode) -> gettext.NullTranslations - global translators - translator = translators.get(catalog) - if translator is None: - translator = gettext.NullTranslations() - if hasattr(translator, 'ugettext'): - translator.gettext = translator.ugettext - return translator +def init_console(locale_dir, catalog): + # type: (unicode, unicode) -> Tuple[NullTranslations, bool] + """Initialize locale for console. + + .. versionadded:: 1.8 + """ + language, _ = locale.getlocale(locale.LC_MESSAGES) # encoding is ignored + return init([locale_dir], language, catalog, 'console') + + +def get_translator(catalog='sphinx', namespace='general'): + # type: (unicode, unicode) -> NullTranslations + return translators[(namespace, catalog)] + + +def is_translator_registered(catalog='sphinx', namespace='general'): + # type: (unicode, unicode) -> bool + return (namespace, catalog) in translators + + +def _lazy_translate(catalog, namespace, message): + # type: (unicode, unicode, unicode) -> unicode + """Used instead of _ when creating TranslationProxy, because _ is + not bound yet at that time. + """ + translator = get_translator(catalog, namespace) + return translator.gettext(message) # type: ignore + + +def get_translation(catalog, namespace='general'): + """Get a translation function based on the *catalog* and *namespace*. + + The extension can use this API to translate the messages on the + extension:: + + import os + from sphinx.locale import get_translation + + _ = get_translation(__name__) + text = _('Hello Sphinx!') + + + def setup(app): + package_dir = path.abspath(path.dirname(__file__)) + locale_dir = os.path.join(package_dir, 'locales') + app.add_message_catalog(__name__, locale_dir) + + With this code, sphinx searches a message catalog from + ``${package_dir}/locales/${language}/LC_MESSAGES/${__name__}.mo`` + The :confval:`language` is used for the searching. + + .. versionadded:: 1.8 + """ + def gettext(message, *args): + # type: (unicode, *Any) -> unicode + if not is_translator_registered(catalog, namespace): + # not initialized yet + return _TranslationProxy(_lazy_translate, catalog, namespace, message) # type: ignore # NOQA + else: + translator = get_translator(catalog, namespace) + if len(args) <= 1: + return translator.gettext(message) # type: ignore + else: # support pluralization + return translator.ngettext(message, args[0], args[1]) # type: ignore + + return gettext + + +# A shortcut for sphinx-core +#: Translation function for messages on documentation (menu, labels, themes and so on). +#: This function follows :confval:`language` setting. +_ = get_translation('sphinx') +#: Translation function for console messages +#: This function follows locale setting (`LC_ALL`, `LC_MESSAGES` and so on). +__ = get_translation('sphinx', 'console') + + +def l_(*args): + warnings.warn('sphinx.locale.l_() is deprecated. Please use `_()` instead.', + RemovedInSphinx30Warning) + return _(*args) + + +# labels +admonitionlabels = { + 'attention': _('Attention'), + 'caution': _('Caution'), + 'danger': _('Danger'), + 'error': _('Error'), + 'hint': _('Hint'), + 'important': _('Important'), + 'note': _('Note'), + 'seealso': _('See also'), + 'tip': _('Tip'), + 'warning': _('Warning'), +} # type: Dict[unicode, unicode] + +# Moved to sphinx.directives.other (will be overrided later) +versionlabels = {} # type: Dict[unicode, unicode] + +# Moved to sphinx.domains.python (will be overrided later) +pairindextypes = {} # type: Dict[unicode, unicode] diff --git a/sphinx/make_mode.py b/sphinx/make_mode.py index aecad31b4..23fb4ce64 100644 --- a/sphinx/make_mode.py +++ b/sphinx/make_mode.py @@ -22,7 +22,7 @@ import sys from os import path import sphinx -from sphinx import cmdline +from sphinx.cmd.build import build_main from sphinx.util.console import color_terminal, nocolor, bold, blue # type: ignore from sphinx.util.osutil import cd, rmtree @@ -151,7 +151,7 @@ class Make(object): '-d', doctreedir, self.srcdir, self.builddir_join(builder)] - return cmdline.main(args + opts) + return build_main(args + opts) def run_make_mode(args): diff --git a/sphinx/parsers.py b/sphinx/parsers.py index 34822898f..db62e7154 100644 --- a/sphinx/parsers.py +++ b/sphinx/parsers.py @@ -91,7 +91,7 @@ class RSTParser(docutils.parsers.rst.Parser): def setup(app): # type: (Sphinx) -> Dict[unicode, Any] - app.add_source_parser('*', RSTParser) # register as a special parser + app.add_source_parser(RSTParser) return { 'version': 'builtin', diff --git a/sphinx/pycode/__init__.py b/sphinx/pycode/__init__.py index de951a19f..04353e805 100644 --- a/sphinx/pycode/__init__.py +++ b/sphinx/pycode/__init__.py @@ -27,17 +27,19 @@ class ModuleAnalyzer(object): @classmethod def for_string(cls, string, modname, srcname='<string>'): + # type: (unicode, unicode, unicode) -> ModuleAnalyzer if isinstance(string, bytes): return cls(BytesIO(string), modname, srcname) - return cls(StringIO(string), modname, srcname, decoded=True) + return cls(StringIO(string), modname, srcname, decoded=True) # type: ignore @classmethod def for_file(cls, filename, modname): + # type: (unicode, unicode) -> ModuleAnalyzer if ('file', filename) in cls.cache: return cls.cache['file', filename] try: with open(filename, 'rb') as f: - obj = cls(f, modname, filename) + obj = cls(f, modname, filename) # type: ignore cls.cache['file', filename] = obj except Exception as err: raise PycodeError('error opening %r' % filename, err) @@ -45,6 +47,7 @@ class ModuleAnalyzer(object): @classmethod def for_module(cls, modname): + # type: (str) -> ModuleAnalyzer if ('module', modname) in cls.cache: entry = cls.cache['module', modname] if isinstance(entry, PycodeError): @@ -126,7 +129,7 @@ if __name__ == '__main__': # ma = ModuleAnalyzer.for_file(__file__.rstrip('c'), 'sphinx.builders.html') ma = ModuleAnalyzer.for_file('sphinx/environment.py', 'sphinx.environment') - ma.tokenize() + ma.tokenize() # type: ignore x1 = time.time() ma.parse() x2 = time.time() diff --git a/sphinx/pycode/parser.py b/sphinx/pycode/parser.py index f15da664a..8f6615b54 100644 --- a/sphinx/pycode/parser.py +++ b/sphinx/pycode/parser.py @@ -77,6 +77,7 @@ def dedent_docstring(s): # type: (unicode) -> unicode """Remove common leading indentation from docstring.""" def dummy(): + # type: () -> None # dummy function to mock `inspect.getdoc`. pass diff --git a/sphinx/quickstart.py b/sphinx/quickstart.py index ab4cf92da..dbdfb47c1 100644 --- a/sphinx/quickstart.py +++ b/sphinx/quickstart.py @@ -10,12 +10,17 @@ """ import warnings +from typing import TYPE_CHECKING from sphinx.cmd.quickstart import main as _main from sphinx.deprecation import RemovedInSphinx20Warning +if TYPE_CHECKING: + from typing import Any # NOQA + def main(*args, **kwargs): + # type: (Any, Any) -> None warnings.warn( '`sphinx.quickstart.main()` has moved to `sphinx.cmd.quickstart.' 'main()`.', diff --git a/sphinx/registry.py b/sphinx/registry.py index cc012b3f7..0b95f16b9 100644 --- a/sphinx/registry.py +++ b/sphinx/registry.py @@ -11,10 +11,15 @@ from __future__ import print_function import traceback +import warnings +from inspect import isclass +from types import MethodType +from docutils.parsers.rst import Directive from pkg_resources import iter_entry_points -from six import iteritems, itervalues, string_types +from six import iteritems, itervalues +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.domains import ObjType from sphinx.domains.std import GenericObject, Target from sphinx.errors import ExtensionError, SphinxError, VersionRequirementError @@ -22,14 +27,13 @@ from sphinx.extension import Extension from sphinx.locale import __ from sphinx.parsers import Parser as SphinxParser from sphinx.roles import XRefRole -from sphinx.util import import_object from sphinx.util import logging from sphinx.util.console import bold # type: ignore from sphinx.util.docutils import directive_helper if False: # For type annotation - from typing import Any, Callable, Dict, Iterator, List, Type, Union # NOQA + from typing import Any, Callable, Dict, Iterator, List, Tuple, Type, Union # NOQA from docutils import nodes # NOQA from docutils.io import Input # NOQA from docutils.parsers import Parser # NOQA @@ -39,7 +43,7 @@ if False: from sphinx.domains import Domain, Index # NOQA from sphinx.environment import BuildEnvironment # NOQA from sphinx.ext.autodoc import Documenter # NOQA - from sphinx.util.typing import RoleFunction # NOQA + from sphinx.util.typing import RoleFunction, TitleGetter # NOQA logger = logging.getLogger(__name__) @@ -52,18 +56,62 @@ EXTENSION_BLACKLIST = { class SphinxComponentRegistry(object): def __init__(self): + # type: () -> None + #: special attrgetter for autodoc; class object -> attrgetter self.autodoc_attrgettrs = {} # type: Dict[Type, Callable[[Any, unicode, Any], Any]] + + #: builders; a dict of builder name -> bulider class self.builders = {} # type: Dict[unicode, Type[Builder]] + + #: autodoc documenters; a dict of documenter name -> documenter class self.documenters = {} # type: Dict[unicode, Type[Documenter]] + + #: domains; a dict of domain name -> domain class self.domains = {} # type: Dict[unicode, Type[Domain]] + + #: additional directives for domains + #: a dict of domain name -> dict of directive name -> directive self.domain_directives = {} # type: Dict[unicode, Dict[unicode, Any]] + + #: additional indices for domains + #: a dict of domain name -> list of index class self.domain_indices = {} # type: Dict[unicode, List[Type[Index]]] + + #: additional object types for domains + #: a dict of domain name -> dict of objtype name -> objtype self.domain_object_types = {} # type: Dict[unicode, Dict[unicode, ObjType]] + + #: additional roles for domains + #: a dict of domain name -> dict of role name -> role impl. self.domain_roles = {} # type: Dict[unicode, Dict[unicode, Union[RoleFunction, XRefRole]]] # NOQA + + #: additional enumerable nodes + #: a dict of node class -> tuple of figtype and title_getter function + self.enumerable_nodes = {} # type: Dict[nodes.Node, Tuple[unicode, TitleGetter]] + + #: LaTeX packages; list of package names and its options + self.latex_packages = [] # type: List[Tuple[unicode, unicode]] + + #: post transforms; list of transforms self.post_transforms = [] # type: List[Type[Transform]] - self.source_parsers = {} # type: Dict[unicode, Parser] + + #: source paresrs; file type -> parser class + self.source_parsers = {} # type: Dict[unicode, Type[Parser]] + + #: source inputs; file type -> input class self.source_inputs = {} # type: Dict[unicode, Input] + + #: source suffix: suffix -> file type + self.source_suffix = {} # type: Dict[unicode, unicode] + + #: custom translators; builder name -> translator class self.translators = {} # type: Dict[unicode, nodes.NodeVisitor] + + #: custom handlers for translators + #: a dict of builder name -> dict of node name -> visitor and departure functions + self.translation_handlers = {} # type: Dict[unicode, Dict[unicode, Tuple[Callable, Callable]]] # NOQA + + #: additional transforms; list of transforms self.transforms = [] # type: List[Type[Transform]] def add_builder(self, builder): @@ -140,8 +188,12 @@ class SphinxComponentRegistry(object): (domain, name, obj, has_content, argument_spec, option_spec)) if domain not in self.domains: raise ExtensionError(__('domain %s not yet registered') % domain) + directives = self.domain_directives.setdefault(domain, {}) - directives[name] = directive_helper(obj, has_content, argument_spec, **option_spec) + if not isclass(obj) or not issubclass(obj, Directive): + directives[name] = directive_helper(obj, has_content, argument_spec, **option_spec) + else: + directives[name] = obj def add_role_to_domain(self, domain, name, role): # type: (unicode, unicode, Union[RoleFunction, XRefRole]) -> None @@ -197,28 +249,57 @@ class SphinxComponentRegistry(object): object_types = self.domain_object_types.setdefault('std', {}) object_types[directivename] = ObjType(objname or directivename, rolename) - def add_source_parser(self, suffix, parser): - # type: (unicode, Type[Parser]) -> None - logger.debug('[app] adding search source_parser: %r, %r', suffix, parser) - if suffix in self.source_parsers: + def add_source_suffix(self, suffix, filetype): + # type: (unicode, unicode) -> None + logger.debug('[app] adding source_suffix: %r, %r', suffix, filetype) + if suffix in self.source_suffix: raise ExtensionError(__('source_parser for %r is already registered') % suffix) - self.source_parsers[suffix] = parser - - def get_source_parser(self, filename): - # type: (unicode) -> Type[Parser] - for suffix, parser_class in iteritems(self.source_parsers): - if filename.endswith(suffix): - break else: - # use special parser for unknown file-extension '*' (if exists) - parser_class = self.source_parsers.get('*') - - if parser_class is None: - raise SphinxError(__('source_parser for %s not registered') % filename) + self.source_suffix[suffix] = filetype + + def add_source_parser(self, *args): + # type: (Any) -> None + logger.debug('[app] adding search source_parser: %r', args) + if len(args) == 1: + # new sytle arguments: (source_parser) + suffix = None # type: unicode + parser = args[0] # type: Type[Parser] else: - if isinstance(parser_class, string_types): - parser_class = import_object(parser_class, 'source parser') # type: ignore - return parser_class + # old style arguments: (suffix, source_parser) + warnings.warn('app.add_source_parser() does not support suffix argument. ' + 'Use app.add_source_suffix() instead.', + RemovedInSphinx30Warning) + suffix = args[0] + parser = args[1] + + if suffix: + self.add_source_suffix(suffix, suffix) + + if len(parser.supported) == 0: + warnings.warn('Old source_parser has been detected. Please fill Parser.supported ' + 'attribute: %s' % parser.__name__, + RemovedInSphinx30Warning) + + # create a map from filetype to parser + for filetype in parser.supported: + if filetype in self.source_parsers: + raise ExtensionError(__('source_parser for %r is already registered') % + filetype) + else: + self.source_parsers[filetype] = parser + + # also maps suffix to parser + # + # This rescues old styled parsers which does not have ``supported`` filetypes. + if suffix: + self.source_parsers[suffix] = parser + + def get_source_parser(self, filetype): + # type: (unicode) -> Type[Parser] + try: + return self.source_parsers[filetype] + except KeyError: + raise SphinxError(__('Source parser for %s not registered') % filetype) def get_source_parsers(self): # type: () -> Dict[unicode, Parser] @@ -240,36 +321,57 @@ class SphinxComponentRegistry(object): filetype) self.source_inputs[filetype] = input_class - def get_source_input(self, filename): + def get_source_input(self, filetype): # type: (unicode) -> Type[Input] - parser = self.get_source_parser(filename) - for filetype in parser.supported: - if filetype in self.source_inputs: - input_class = self.source_inputs[filetype] - break - else: - # use special source_input for unknown file-type '*' (if exists) - input_class = self.source_inputs.get('*') - - if input_class is None: - raise SphinxError(__('source_input for %s not registered') % filename) - else: - return input_class + try: + return self.source_inputs[filetype] + except KeyError: + try: + # use special source_input for unknown filetype + return self.source_inputs['*'] + except KeyError: + raise SphinxError(__('source_input for %s not registered') % filetype) def add_translator(self, name, translator): # type: (unicode, Type[nodes.NodeVisitor]) -> None logger.info(bold(__('Change of translator for the %s builder.') % name)) self.translators[name] = translator + def add_translation_handlers(self, node, **kwargs): + # type: (nodes.Node, Any) -> None + logger.debug('[app] adding translation_handlers: %r, %r', node, kwargs) + for builder_name, handlers in iteritems(kwargs): + translation_handlers = self.translation_handlers.setdefault(builder_name, {}) + try: + visit, depart = handlers # unpack once for assertion + translation_handlers[node.__name__] = (visit, depart) + except ValueError: + raise ExtensionError(__('kwargs for add_node() must be a (visit, depart) ' + 'function tuple: %r=%r') % builder_name, handlers) + def get_translator_class(self, builder): # type: (Builder) -> Type[nodes.NodeVisitor] return self.translators.get(builder.name, builder.default_translator_class) - def create_translator(self, builder, document): - # type: (Builder, nodes.Node) -> nodes.NodeVisitor + def create_translator(self, builder, *args): + # type: (Builder, Any) -> nodes.NodeVisitor translator_class = self.get_translator_class(builder) - return translator_class(builder, document) + assert translator_class, "translator not found for %s" % builder.name + translator = translator_class(*args) + + # transplant handlers for custom nodes to translator instance + handlers = self.translation_handlers.get(builder.name, None) + if handlers is None: + # retry with builder.format + handlers = self.translation_handlers.get(builder.format, {}) + + for name, (visit, depart) in iteritems(handlers): + setattr(translator, 'visit_' + name, MethodType(visit, translator)) + if depart: + setattr(translator, 'depart_' + name, MethodType(depart, translator)) + + return translator def add_transform(self, transform): # type: (Type[Transform]) -> None @@ -297,6 +399,16 @@ class SphinxComponentRegistry(object): # type: (Type, Callable[[Any, unicode, Any], Any]) -> None self.autodoc_attrgettrs[typ] = attrgetter + def add_latex_package(self, name, options): + # type: (unicode, unicode) -> None + logger.debug('[app] adding latex package: %r', name) + self.latex_packages.append((name, options)) + + def add_enumerable_node(self, node, figtype, title_getter=None): + # type: (nodes.Node, unicode, TitleGetter) -> None + logger.debug('[app] adding enumerable node: (%r, %r, %r)', node, figtype, title_getter) + self.enumerable_nodes[node] = (figtype, title_getter) + def load_extension(self, app, extname): # type: (Sphinx, unicode) -> None """Load a Sphinx extension.""" @@ -334,8 +446,6 @@ class SphinxComponentRegistry(object): if metadata is None: metadata = {} - if extname == 'rst2pdf.pdfbuilder': - metadata['parallel_read_safe'] = True elif not isinstance(metadata, dict): logger.warning(__('extension %r returned an unsupported object from ' 'its setup() function; it should return None or a ' @@ -343,3 +453,37 @@ class SphinxComponentRegistry(object): app.extensions[extname] = Extension(extname, mod, **metadata) app._setting_up_extension.pop() + + def get_envversion(self, app): + # type: (Sphinx) -> Dict[unicode, unicode] + from sphinx.environment import ENV_VERSION + envversion = {ext.name: ext.metadata['env_version'] for ext in app.extensions.values() + if ext.metadata.get('env_version')} + envversion['sphinx'] = ENV_VERSION + return envversion + + +def merge_source_suffix(app): + # type: (Sphinx) -> None + """Merge source_suffix which specified by user and added by extensions.""" + for suffix, filetype in iteritems(app.registry.source_suffix): + if suffix not in app.config.source_suffix: + app.config.source_suffix[suffix] = filetype + elif app.config.source_suffix[suffix] is None: + # filetype is not specified (default filetype). + # So it overrides default filetype by extensions setting. + app.config.source_suffix[suffix] = filetype + + # copy config.source_suffix to registry + app.registry.source_suffix = app.config.source_suffix + + +def setup(app): + # type: (Sphinx) -> Dict[unicode, Any] + app.connect('builder-inited', merge_source_suffix) + + return { + 'version': 'builtin', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/sphinx/search/da.py b/sphinx/search/da.py index b3611152a..26ac428fe 100644 --- a/sphinx/search/da.py +++ b/sphinx/search/da.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + danish_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/danish/stop.txt og | and @@ -125,7 +130,9 @@ class SearchDanish(SearchLanguage): stopwords = danish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('danish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/de.py b/sphinx/search/de.py index 89cbe3de5..00e36354b 100644 --- a/sphinx/search/de.py +++ b/sphinx/search/de.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + german_stopwords = parse_stop_word(u''' |source: http://snowball.tartarus.org/algorithms/german/stop.txt aber | but @@ -308,7 +313,9 @@ class SearchGerman(SearchLanguage): stopwords = german_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('german') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/es.py b/sphinx/search/es.py index 2777811d8..971d0c149 100644 --- a/sphinx/search/es.py +++ b/sphinx/search/es.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + spanish_stopwords = parse_stop_word(u''' |source: http://snowball.tartarus.org/algorithms/spanish/stop.txt de | from, of @@ -368,7 +373,9 @@ class SearchSpanish(SearchLanguage): stopwords = spanish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('spanish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/fi.py b/sphinx/search/fi.py index ca63b0021..b2b47781e 100644 --- a/sphinx/search/fi.py +++ b/sphinx/search/fi.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + finnish_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/finnish/stop.txt | forms of BE @@ -118,7 +123,9 @@ class SearchFinnish(SearchLanguage): stopwords = finnish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('finnish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/fr.py b/sphinx/search/fr.py index 615a47383..9976c1ca7 100644 --- a/sphinx/search/fr.py +++ b/sphinx/search/fr.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + french_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/french/stop.txt au | a + le @@ -204,7 +209,9 @@ class SearchFrench(SearchLanguage): stopwords = french_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('french') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word.lower()) diff --git a/sphinx/search/hu.py b/sphinx/search/hu.py index 85d15bac7..9cdee61fe 100644 --- a/sphinx/search/hu.py +++ b/sphinx/search/hu.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + hungarian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/hungarian/stop.txt | prepared by Anna Tordai @@ -232,7 +237,9 @@ class SearchHungarian(SearchLanguage): stopwords = hungarian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('hungarian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/it.py b/sphinx/search/it.py index b613ce33c..db86e386e 100644 --- a/sphinx/search/it.py +++ b/sphinx/search/it.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + italian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/italian/stop.txt ad | a (to) before vowel @@ -321,7 +326,9 @@ class SearchItalian(SearchLanguage): stopwords = italian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('italian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/nl.py b/sphinx/search/nl.py index b2ab4e0f0..49003d2d1 100644 --- a/sphinx/search/nl.py +++ b/sphinx/search/nl.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + dutch_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/dutch/stop.txt de | the @@ -132,7 +137,9 @@ class SearchDutch(SearchLanguage): stopwords = dutch_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('dutch') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/no.py b/sphinx/search/no.py index 32f7b38c5..7f5a73858 100644 --- a/sphinx/search/no.py +++ b/sphinx/search/no.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + norwegian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/norwegian/stop.txt og | and @@ -207,7 +212,9 @@ class SearchNorwegian(SearchLanguage): stopwords = norwegian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('norwegian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/pt.py b/sphinx/search/pt.py index ba9f2529b..44c7e2118 100644 --- a/sphinx/search/pt.py +++ b/sphinx/search/pt.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + portuguese_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/portuguese/stop.txt de | of, from @@ -267,7 +272,9 @@ class SearchPortuguese(SearchLanguage): stopwords = portuguese_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('portuguese') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/ru.py b/sphinx/search/ru.py index 8eb6c3dbb..b628b9866 100644 --- a/sphinx/search/ru.py +++ b/sphinx/search/ru.py @@ -13,6 +13,11 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any # NOQA + + russian_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/russian/stop.txt и | and @@ -256,7 +261,9 @@ class SearchRussian(SearchLanguage): stopwords = russian_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('russian') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/search/sv.py b/sphinx/search/sv.py index 64883381e..953ae94b5 100644 --- a/sphinx/search/sv.py +++ b/sphinx/search/sv.py @@ -13,6 +13,10 @@ from sphinx.search import SearchLanguage, parse_stop_word import snowballstemmer +if False: + # For type annotation + from typing import Any + swedish_stopwords = parse_stop_word(u''' | source: http://snowball.tartarus.org/algorithms/swedish/stop.txt och | and @@ -145,7 +149,9 @@ class SearchSwedish(SearchLanguage): stopwords = swedish_stopwords def init(self, options): + # type: (Any) -> None self.stemmer = snowballstemmer.stemmer('swedish') def stem(self, word): + # type: (unicode) -> unicode return self.stemmer.stemWord(word) diff --git a/sphinx/setup_command.py b/sphinx/setup_command.py index 1f160fec1..37fc2d042 100644 --- a/sphinx/setup_command.py +++ b/sphinx/setup_command.py @@ -116,11 +116,11 @@ class BuildDoc(Command): for root, dirnames, filenames in os.walk(guess): if 'conf.py' in filenames: return root - return None + return os.curdir # Overriding distutils' Command._ensure_stringlike which doesn't support # unicode, causing finalize_options to fail if invoked again. Workaround - # for http://bugs.python.org/issue19570 + # for https://bugs.python.org/issue19570 def _ensure_stringlike(self, option, what, default=None): # type: (unicode, unicode, Any) -> Any val = getattr(self, option) @@ -134,30 +134,26 @@ class BuildDoc(Command): def finalize_options(self): # type: () -> None + self.ensure_string_list('builder') + if self.source_dir is None: self.source_dir = self._guess_source_dir() self.announce('Using source directory %s' % self.source_dir) + self.ensure_dirname('source_dir') - if self.source_dir is None: - self.source_dir = os.curdir - self.source_dir = abspath(self.source_dir) + if self.config_dir is None: self.config_dir = self.source_dir - self.config_dir = abspath(self.config_dir) - self.ensure_string_list('builder') if self.build_dir is None: build = self.get_finalized_command('build') self.build_dir = os.path.join(abspath(build.build_base), 'sphinx') # type: ignore - self.mkpath(self.build_dir) # type: ignore - self.build_dir = abspath(self.build_dir) + self.doctree_dir = os.path.join(self.build_dir, 'doctrees') - self.mkpath(self.doctree_dir) # type: ignore + self.builder_target_dirs = [ (builder, os.path.join(self.build_dir, builder)) for builder in self.builder] # type: List[Tuple[str, unicode]] - for _, builder_target_dir in self.builder_target_dirs: - self.mkpath(builder_target_dir) # type: ignore def run(self): # type: () -> None diff --git a/sphinx/testing/fixtures.py b/sphinx/testing/fixtures.py index be0037b70..76ed154fd 100644 --- a/sphinx/testing/fixtures.py +++ b/sphinx/testing/fixtures.py @@ -22,16 +22,19 @@ from six import StringIO, string_types from . import util if False: - from typing import Dict, Union # NOQA + # For type annotation + from typing import Any, Dict, Union # NOQA @pytest.fixture(scope='session') def rootdir(): + # type: () -> None return None @pytest.fixture def app_params(request, test_params, shared_result, sphinx_test_tempdir, rootdir): + # type: (Any, Any, Any, Any, Any) -> None """ parameters that is specified by 'pytest.mark.sphinx' for sphinx.application.Sphinx initialization @@ -137,12 +140,14 @@ def warning(app): @pytest.fixture() -def make_app(test_params): +def make_app(test_params, monkeypatch): """ provides make_app function to initialize SphinxTestApp instance. if you want to initialize 'app' in your test function. please use this instead of using SphinxTestApp class directory. """ + monkeypatch.setattr('sphinx.application.abspath', lambda x: x) + apps = [] syspath = sys.path[:] @@ -153,7 +158,7 @@ def make_app(test_params): app_ = util.SphinxTestApp(*args, **kwargs) # type: Union[util.SphinxTestApp, util.SphinxTestAppWrapperForSkipBuilding] # NOQA apps.append(app_) if test_params['shared_result']: - app_ = util.SphinxTestAppWrapperForSkipBuilding(app_) + app_ = util.SphinxTestAppWrapperForSkipBuilding(app_) # type: ignore return app_ yield make diff --git a/sphinx/testing/path.py b/sphinx/testing/path.py index 5836d2bc0..5f59ad30b 100644 --- a/sphinx/testing/path.py +++ b/sphinx/testing/path.py @@ -10,9 +10,13 @@ import os import shutil import sys from io import open +from typing import TYPE_CHECKING from six import PY2, text_type +if TYPE_CHECKING: + from typing import Any, Callable, IO, List # NOQA + FILESYSTEMENCODING = sys.getfilesystemencoding() or sys.getdefaultencoding() @@ -23,58 +27,68 @@ class path(text_type): """ if PY2: def __new__(cls, s, encoding=FILESYSTEMENCODING, errors='strict'): + # type: (unicode, unicode, unicode) -> path if isinstance(s, str): s = s.decode(encoding, errors) - return text_type.__new__(cls, s) # type: ignore + return text_type.__new__(cls, s) return text_type.__new__(cls, s) # type: ignore @property def parent(self): + # type: () -> path """ The name of the directory the file or directory is in. """ return self.__class__(os.path.dirname(self)) def basename(self): + # type: () -> unicode return os.path.basename(self) def abspath(self): + # type: () -> path """ Returns the absolute path. """ return self.__class__(os.path.abspath(self)) def isabs(self): + # type: () -> bool """ Returns ``True`` if the path is absolute. """ return os.path.isabs(self) def isdir(self): + # type: () -> bool """ Returns ``True`` if the path is a directory. """ return os.path.isdir(self) def isfile(self): + # type: () -> bool """ Returns ``True`` if the path is a file. """ return os.path.isfile(self) def islink(self): + # type: () -> bool """ Returns ``True`` if the path is a symbolic link. """ return os.path.islink(self) def ismount(self): + # type: () -> bool """ Returns ``True`` if the path is a mount point. """ return os.path.ismount(self) def rmtree(self, ignore_errors=False, onerror=None): + # type: (bool, Callable) -> None """ Removes the file or directory and any files or directories it may contain. @@ -93,6 +107,7 @@ class path(text_type): shutil.rmtree(self, ignore_errors=ignore_errors, onerror=onerror) def copytree(self, destination, symlinks=False): + # type: (unicode, bool) -> None """ Recursively copy a directory to the given `destination`. If the given `destination` does not exist it will be created. @@ -105,6 +120,7 @@ class path(text_type): shutil.copytree(self, destination, symlinks=symlinks) def movetree(self, destination): + # type: (unicode) -> None """ Recursively move the file or directory to the given `destination` similar to the Unix "mv" command. @@ -117,24 +133,29 @@ class path(text_type): move = movetree def unlink(self): + # type: () -> None """ Removes a file. """ os.unlink(self) def stat(self): + # type: () -> Any """ Returns a stat of the file. """ return os.stat(self) def utime(self, arg): + # type: (Any) -> None os.utime(self, arg) def open(self, mode='r', **kwargs): + # type: (unicode, Any) -> IO return open(self, mode, **kwargs) def write_text(self, text, encoding='utf-8', **kwargs): + # type: (unicode, unicode, Any) -> None """ Writes the given `text` to the file. """ @@ -144,6 +165,7 @@ class path(text_type): f.write(text) def text(self, encoding='utf-8', **kwargs): + # type: (unicode, Any) -> unicode """ Returns the text in the file. """ @@ -152,6 +174,7 @@ class path(text_type): return f.read() def bytes(self): + # type: () -> str """ Returns the bytes in the file. """ @@ -159,6 +182,7 @@ class path(text_type): return f.read() def write_bytes(self, bytes, append=False): + # type: (str, bool) -> None """ Writes the given `bytes` to the file. @@ -173,12 +197,14 @@ class path(text_type): f.write(bytes) def exists(self): + # type: () -> bool """ Returns ``True`` if the path exist. """ return os.path.exists(self) def lexists(self): + # type: () -> bool """ Returns ``True`` if the path exists unless it is a broken symbolic link. @@ -186,21 +212,25 @@ class path(text_type): return os.path.lexists(self) def makedirs(self, mode=0o777): + # type: (int) -> None """ Recursively create directories. """ os.makedirs(self, mode) def joinpath(self, *args): + # type: (Any) -> path """ Joins the path with the argument given and returns the result. """ - return self.__class__(os.path.join(self, *map(self.__class__, args))) + return self.__class__(os.path.join(self, *map(self.__class__, args))) # type: ignore # NOQA def listdir(self): + # type: () -> List[unicode] return os.listdir(self) __div__ = __truediv__ = joinpath def __repr__(self): + # type: () -> str return '%s(%s)' % (self.__class__.__name__, text_type.__repr__(self)) diff --git a/sphinx/testing/util.py b/sphinx/testing/util.py index 60544cfa3..e55007d80 100644 --- a/sphinx/testing/util.py +++ b/sphinx/testing/util.py @@ -18,14 +18,16 @@ from docutils import nodes from docutils.parsers.rst import directives, roles from six import string_types -from sphinx import application +from sphinx import application, locale from sphinx.builders.latex import LaTeXBuilder from sphinx.ext.autodoc import AutoDirective from sphinx.pycode import ModuleAnalyzer from sphinx.testing.path import path if False: + # For type annotation from typing import List # NOQA + from typing import Any, Dict, Generator, IO, List, Pattern # NOQA __all__ = [ @@ -36,21 +38,25 @@ __all__ = [ def assert_re_search(regex, text, flags=0): + # type: (Pattern, unicode, int) -> None if not re.search(regex, text, flags): assert False, '%r did not match %r' % (regex, text) def assert_not_re_search(regex, text, flags=0): + # type: (Pattern, unicode, int) -> None if re.search(regex, text, flags): assert False, '%r did match %r' % (regex, text) def assert_startswith(thing, prefix): + # type: (unicode, unicode) -> None if not thing.startswith(prefix): assert False, '%r does not start with %r' % (thing, prefix) def assert_node(node, cls=None, xpath="", **kwargs): + # type: (nodes.Node, Any, unicode, Any) -> None if cls: if isinstance(cls, list): assert_node(node, cls[0], xpath=xpath, **kwargs) @@ -80,13 +86,15 @@ def assert_node(node, cls=None, xpath="", **kwargs): def etree_parse(path): + # type: (unicode) -> Any with warnings.catch_warnings(record=False): warnings.filterwarnings("ignore", category=DeprecationWarning) - return ElementTree.parse(path) + return ElementTree.parse(path) # type: ignore class Struct(object): def __init__(self, **kwds): + # type: (Any) -> None self.__dict__.update(kwds) @@ -99,6 +107,7 @@ class SphinxTestApp(application.Sphinx): def __init__(self, buildername='html', srcdir=None, freshenv=False, confoverrides=None, status=None, warning=None, tags=None, docutilsconf=None): + # type: (unicode, path, bool, Dict, IO, IO, unicode, unicode) -> None if docutilsconf is not None: (srcdir / 'docutils.conf').write_text(docutilsconf) @@ -127,7 +136,7 @@ class SphinxTestApp(application.Sphinx): if v.startswith('visit_')) try: - application.Sphinx.__init__(self, srcdir, confdir, outdir, doctreedir, + application.Sphinx.__init__(self, srcdir, confdir, outdir, doctreedir, # type: ignore # NOQA buildername, confoverrides, status, warning, freshenv, warningiserror, tags) except Exception: @@ -135,9 +144,11 @@ class SphinxTestApp(application.Sphinx): raise def cleanup(self, doctrees=False): + # type: (bool) -> None AutoDirective._registry.clear() ModuleAnalyzer.cache.clear() LaTeXBuilder.usepackages = [] + locale.translators.clear() sys.path[:] = self._saved_path sys.modules.pop('autodoc_fodder', None) directives._directives = self._saved_directives @@ -149,6 +160,7 @@ class SphinxTestApp(application.Sphinx): delattr(nodes.GenericNodeVisitor, 'depart_' + method[6:]) def __repr__(self): + # type: () -> str return '<%s buildername=%r>' % (self.__class__.__name__, self.builder.name) @@ -160,13 +172,16 @@ class SphinxTestAppWrapperForSkipBuilding(object): """ def __init__(self, app_): + # type: (SphinxTestApp) -> None self.app = app_ def __getattr__(self, name): + # type: (str) -> Any return getattr(self.app, name) def build(self, *args, **kw): - if not self.app.outdir.listdir(): + # type: (Any, Any) -> None + if not self.app.outdir.listdir(): # type: ignore # if listdir is empty, do build. self.app.build(*args, **kw) # otherwise, we can use built cache @@ -176,16 +191,19 @@ _unicode_literals_re = re.compile(r'u(".*?")|u(\'.*?\')') def remove_unicode_literals(s): - return _unicode_literals_re.sub(lambda x: x.group(1) or x.group(2), s) + # type: (unicode) -> unicode + return _unicode_literals_re.sub(lambda x: x.group(1) or x.group(2), s) # type: ignore def find_files(root, suffix=None): + # type: (unicode, bool) -> Generator for dirpath, dirs, files in os.walk(root, followlinks=True): dirpath = path(dirpath) - for f in [f for f in files if not suffix or f.endswith(suffix)]: + for f in [f for f in files if not suffix or f.endswith(suffix)]: # type: ignore fpath = dirpath / f yield os.path.relpath(fpath, root) def strip_escseq(text): + # type: (unicode) -> unicode return re.sub('\x1b.*?m', '', text) diff --git a/sphinx/texinputs/footnotehyper-sphinx.sty b/sphinx/texinputs/footnotehyper-sphinx.sty index 5995f012d..b6692cfb8 100644 --- a/sphinx/texinputs/footnotehyper-sphinx.sty +++ b/sphinx/texinputs/footnotehyper-sphinx.sty @@ -4,10 +4,10 @@ %% %% Package: footnotehyper-sphinx %% Version: based on footnotehyper.sty 2017/03/07 v1.0 -%% as available at http://www.ctan.org/pkg/footnotehyper +%% as available at https://www.ctan.org/pkg/footnotehyper %% License: the one applying to Sphinx %% -%% Refer to the PDF documentation at http://www.ctan.org/pkg/footnotehyper for +%% Refer to the PDF documentation at https://www.ctan.org/pkg/footnotehyper for %% the code comments. %% %% Differences: diff --git a/sphinx/theming.py b/sphinx/theming.py index 33c4c76be..7287f7fc4 100644 --- a/sphinx/theming.py +++ b/sphinx/theming.py @@ -133,7 +133,7 @@ class Theme(object): for option, value in iteritems(overrides): if option not in options: - logger.warning('unsupported theme option %r given' % option) + logger.warning(__('unsupported theme option %r given') % option) else: options[option] = value diff --git a/sphinx/transforms/__init__.py b/sphinx/transforms/__init__.py index a2e92223d..d82b02149 100644 --- a/sphinx/transforms/__init__.py +++ b/sphinx/transforms/__init__.py @@ -19,7 +19,7 @@ from docutils.utils import normalize_language_tag from docutils.utils.smartquotes import smartchars from sphinx import addnodes -from sphinx.locale import _ +from sphinx.locale import _, __ from sphinx.util import logging from sphinx.util.docutils import new_document from sphinx.util.i18n import format_date @@ -27,6 +27,7 @@ from sphinx.util.nodes import apply_source_workaround, is_smartquotable if False: # For type annotation + from typing import Generator, List # NOQA from sphinx.application import Sphinx # NOQA from sphinx.config import Config # NOQA from sphinx.domain.std import StandardDomain # NOQA @@ -124,7 +125,7 @@ class DefaultSubstitutions(SphinxTransform): text = self.config[refname] if refname == 'today' and not text: # special handling: can also specify a strftime format - text = format_date(self.config.today_fmt or _('%b %d, %Y'), # type: ignore + text = format_date(self.config.today_fmt or _('%b %d, %Y'), language=self.config.language) ref.replace_self(nodes.Text(text, text)) @@ -261,8 +262,8 @@ class AutoIndexUpgrader(SphinxTransform): # type: () -> None for node in self.document.traverse(addnodes.index): if 'entries' in node and any(len(entry) == 4 for entry in node['entries']): - msg = ('4 column based index found. ' - 'It might be a bug of extensions you use: %r' % node['entries']) + msg = __('4 column based index found. ' + 'It might be a bug of extensions you use: %r') % node['entries'] logger.warning(msg, location=node) for i, entry in enumerate(node['entries']): if len(entry) == 4: @@ -297,18 +298,19 @@ class UnreferencedFootnotesDetector(SphinxTransform): default_priority = 200 def apply(self): + # type: () -> None for node in self.document.footnotes: if node['names'] == []: # footnote having duplicated number. It is already warned at parser. pass elif node['names'][0] not in self.document.footnote_refs: - logger.warning('Footnote [%s] is not referenced.', node['names'][0], + logger.warning(__('Footnote [%s] is not referenced.'), node['names'][0], type='ref', subtype='footnote', location=node) for node in self.document.autofootnotes: if not any(ref['auto'] == node['auto'] for ref in self.document.autofootnote_refs): - logger.warning('Footnote [#] is not referenced.', + logger.warning(__('Footnote [#] is not referenced.'), type='ref', subtype='footnote', location=node) @@ -391,6 +393,7 @@ class SphinxSmartQuotes(SmartQuotes, SphinxTransform): return self.config.smartquotes_action def get_tokens(self, txtnodes): + # type: (List[nodes.Node]) -> Generator # A generator that yields ``(texttype, nodetext)`` tuples for a list # of "Text" nodes (interface to ``smartquotes.educate_tokens()``). @@ -406,6 +409,7 @@ class ManpageLink(SphinxTransform): default_priority = 999 def apply(self): + # type: () -> None for node in self.document.traverse(addnodes.manpage): manpage = ' '.join([str(x) for x in node.children if isinstance(x, nodes.Text)]) diff --git a/sphinx/transforms/i18n.py b/sphinx/transforms/i18n.py index bb85c76bd..156c5ac4c 100644 --- a/sphinx/transforms/i18n.py +++ b/sphinx/transforms/i18n.py @@ -17,7 +17,7 @@ from docutils.utils import relative_path from sphinx import addnodes from sphinx.domains.std import make_glossary_term, split_term_classifiers -from sphinx.locale import init as init_locale +from sphinx.locale import __, init as init_locale from sphinx.transforms import SphinxTransform from sphinx.util import split_index_msg, logging from sphinx.util.i18n import find_catalog @@ -52,7 +52,7 @@ def publish_msgstr(app, source, source_path, source_line, config, settings): from sphinx.io import SphinxI18nReader reader = SphinxI18nReader(app) reader.set_lineno_for_reporter(source_line) - parser = app.registry.create_source_parser(app, '') + parser = app.registry.create_source_parser(app, 'restructuredtext') doc = reader.read( source=StringInput(source=source, source_path=source_path), parser=parser, @@ -86,6 +86,8 @@ class Locale(SphinxTransform): def apply(self): # type: () -> None settings, source = self.document.settings, self.document['source'] + msgstr = u'' + # XXX check if this is reliable assert source.startswith(self.env.srcdir) docname = path.splitext(relative_path(path.join(self.env.srcdir, 'dummy'), @@ -102,7 +104,7 @@ class Locale(SphinxTransform): # phase1: replace reference ids with translated names for node, msg in extract_messages(self.document): - msgstr = catalog.gettext(msg) + msgstr = catalog.gettext(msg) # type: ignore # XXX add marker to untranslated parts if not msgstr or msgstr == msg or not msgstr.strip(): # as-of-yet untranslated @@ -219,7 +221,7 @@ class Locale(SphinxTransform): if node.get('translated', False): # to avoid double translation continue # skip if the node is already translated by phase1 - msgstr = catalog.gettext(msg) + msgstr = catalog.gettext(msg) # type: ignore # XXX add marker to untranslated parts if not msgstr or msgstr == msg: # as-of-yet untranslated continue @@ -272,8 +274,8 @@ class Locale(SphinxTransform): if len(old_foot_refs) != len(new_foot_refs): old_foot_ref_rawsources = [ref.rawsource for ref in old_foot_refs] new_foot_ref_rawsources = [ref.rawsource for ref in new_foot_refs] - logger.warning('inconsistent footnote references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent footnote references in translated message.' + + ' original: {0}, translated: {1}') .format(old_foot_ref_rawsources, new_foot_ref_rawsources), location=node) old_foot_namerefs = {} # type: Dict[unicode, List[nodes.footnote_reference]] @@ -312,8 +314,8 @@ class Locale(SphinxTransform): if len(old_refs) != len(new_refs): old_ref_rawsources = [ref.rawsource for ref in old_refs] new_ref_rawsources = [ref.rawsource for ref in new_refs] - logger.warning('inconsistent references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent references in translated message.' + + ' original: {0}, translated: {1}') .format(old_ref_rawsources, new_ref_rawsources), location=node) old_ref_names = [r['refname'] for r in old_refs] @@ -343,8 +345,8 @@ class Locale(SphinxTransform): if len(old_foot_refs) != len(new_foot_refs): old_foot_ref_rawsources = [ref.rawsource for ref in old_foot_refs] new_foot_ref_rawsources = [ref.rawsource for ref in new_foot_refs] - logger.warning('inconsistent footnote references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent footnote references in translated message.' + + ' original: {0}, translated: {1}') .format(old_foot_ref_rawsources, new_foot_ref_rawsources), location=node) for old in old_foot_refs: @@ -365,8 +367,8 @@ class Locale(SphinxTransform): if len(old_cite_refs) != len(new_cite_refs): old_cite_ref_rawsources = [ref.rawsource for ref in old_cite_refs] new_cite_ref_rawsources = [ref.rawsource for ref in new_cite_refs] - logger.warning('inconsistent citation references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent citation references in translated message.' + + ' original: {0}, translated: {1}') .format(old_cite_ref_rawsources, new_cite_ref_rawsources), location=node) for old in old_cite_refs: @@ -385,8 +387,8 @@ class Locale(SphinxTransform): if len(old_refs) != len(new_refs): old_ref_rawsources = [ref.rawsource for ref in old_refs] new_ref_rawsources = [ref.rawsource for ref in new_refs] - logger.warning('inconsistent term references in translated message.' + - ' original: {0}, translated: {1}' + logger.warning(__('inconsistent term references in translated message.' + + ' original: {0}, translated: {1}') .format(old_ref_rawsources, new_ref_rawsources), location=node) @@ -438,7 +440,7 @@ class Locale(SphinxTransform): msg_parts = split_index_msg(type, msg) msgstr_parts = [] for part in msg_parts: - msgstr = catalog.gettext(part) + msgstr = catalog.gettext(part) # type: ignore if not msgstr: msgstr = part msgstr_parts.append(msgstr) @@ -450,6 +452,7 @@ class Locale(SphinxTransform): # remove translated attribute that is used for avoiding double translation. def has_translatable(node): + # type: (nodes.Node) -> bool return isinstance(node, nodes.Element) and 'translated' in node for node in self.document.traverse(has_translatable): node.delattr('translated') diff --git a/sphinx/transforms/post_transforms/images.py b/sphinx/transforms/post_transforms/images.py index b8f4b9a5d..7e73332ff 100644 --- a/sphinx/transforms/post_transforms/images.py +++ b/sphinx/transforms/post_transforms/images.py @@ -16,6 +16,7 @@ from math import ceil from docutils import nodes from six import text_type +from sphinx.locale import __ from sphinx.transforms import SphinxTransform from sphinx.util import epoch_to_rfc1123, rfc1123_to_epoch from sphinx.util import logging, requests @@ -78,12 +79,12 @@ class ImageDownloader(BaseImageConverter): try: headers = {} if os.path.exists(path): - timestamp = ceil(os.stat(path).st_mtime) + timestamp = ceil(os.stat(path).st_mtime) # type: float headers['If-Modified-Since'] = epoch_to_rfc1123(timestamp) r = requests.get(node['uri'], headers=headers) if r.status_code >= 400: - logger.warning('Could not fetch remote image: %s [%d]' % + logger.warning(__('Could not fetch remote image: %s [%d]') % (node['uri'], r.status_code)) else: self.app.env.original_image_uri[path] = node['uri'] @@ -111,7 +112,7 @@ class ImageDownloader(BaseImageConverter): node['uri'] = path self.app.env.images.add_file(self.env.docname, path) except Exception as exc: - logger.warning('Could not fetch remote image: %s [%s]' % + logger.warning(__('Could not fetch remote image: %s [%s]') % (node['uri'], text_type(exc))) @@ -132,7 +133,7 @@ class DataURIExtractor(BaseImageConverter): image = parse_data_uri(node['uri']) ext = get_image_extension(image.mimetype) if ext is None: - logger.warning('Unknown image format: %s...', node['uri'][:32], + logger.warning(__('Unknown image format: %s...'), node['uri'][:32], location=node) return diff --git a/sphinx/util/__init__.py b/sphinx/util/__init__.py index dda3fb04c..6a28432e3 100644 --- a/sphinx/util/__init__.py +++ b/sphinx/util/__init__.py @@ -18,6 +18,7 @@ import sys import tempfile import traceback import unicodedata +import warnings from codecs import BOM_UTF8 from collections import deque from datetime import datetime @@ -29,6 +30,7 @@ from six import text_type, binary_type, itervalues from six.moves import range from six.moves.urllib.parse import urlsplit, urlunsplit, quote_plus, parse_qsl, urlencode +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.errors import PycodeError, SphinxParallelError, ExtensionError from sphinx.util import logging from sphinx.util.console import strip_colors, colorize, bold, term_width_line # type: ignore @@ -172,6 +174,9 @@ def copy_static_entry(source, targetdir, builder, context={}, Handles all possible cases of files, directories and subdirectories. """ + warnings.warn('sphinx.util.copy_static_entry is deprecated for removal', + RemovedInSphinx30Warning) + if exclude_matchers: relpath = relative_path(path.join(builder.srcdir, 'dummy'), source) for matcher in exclude_matchers: @@ -609,6 +614,7 @@ def status_iterator(iterable, summary, color="darkgreen", length=0, verbosity=0, def epoch_to_rfc1123(epoch): + # type: (float) -> unicode """Convert datetime format epoch to RFC1123.""" from babel.dates import format_datetime @@ -618,10 +624,12 @@ def epoch_to_rfc1123(epoch): def rfc1123_to_epoch(rfc1123): + # type: (str) -> float return mktime(strptime(rfc1123, '%a, %d %b %Y %H:%M:%S %Z')) def xmlname_checker(): + # type: () -> Pattern # https://www.w3.org/TR/REC-xml/#NT-Name # Only Python 3.3 or newer support character code in regular expression name_start_chars = [ @@ -640,6 +648,7 @@ def xmlname_checker(): ] def convert(entries, splitter=u'|'): + # type: (Any, unicode) -> unicode results = [] for entry in entries: if isinstance(entry, list): diff --git a/sphinx/util/build_phase.py b/sphinx/util/build_phase.py new file mode 100644 index 000000000..e5a53551c --- /dev/null +++ b/sphinx/util/build_phase.py @@ -0,0 +1,24 @@ +# -*- coding: utf-8 -*- +""" + sphinx.util.build_phase + ~~~~~~~~~~~~~~~~~~~~~~~ + + Build phase of Sphinx application. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +try: + from enum import IntEnum +except ImportError: # py27 + IntEnum = object # type: ignore + + +class BuildPhase(IntEnum): + """Build phase of Sphinx application.""" + INITIALIZATION = 1 + READING = 2 + CONSISTENCY_CHECK = 3 + RESOLVING = 3 + WRITING = 4 diff --git a/sphinx/util/compat.py b/sphinx/util/compat.py index e01558077..43ced1f5e 100644 --- a/sphinx/util/compat.py +++ b/sphinx/util/compat.py @@ -8,15 +8,34 @@ :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. :license: BSD, see LICENSE for details. """ + from __future__ import absolute_import import sys +import warnings + +from six import string_types, iteritems +from sphinx.deprecation import RemovedInSphinx30Warning +from sphinx.util import import_object if False: # For type annotation from typing import Any, Dict # NOQA from sphinx.application import Sphinx # NOQA + from sphinx.config import Config # NOQA + + +def deprecate_source_parsers(app, config): + # type: (Sphinx, Config) -> None + if config.source_parsers: + warnings.warn('The config variable "source_parsers" is deprecated. ' + 'Please use app.add_source_parser() API instead.', + RemovedInSphinx30Warning) + for suffix, parser in iteritems(config.source_parsers): + if isinstance(parser, string_types): + parser = import_object(parser, 'source parser') # type: ignore + app.add_source_parser(suffix, parser) def register_application_for_autosummary(app): @@ -35,6 +54,7 @@ def register_application_for_autosummary(app): def setup(app): # type: (Sphinx) -> Dict[unicode, Any] + app.connect('config-inited', deprecate_source_parsers) app.connect('builder-inited', register_application_for_autosummary) return { diff --git a/sphinx/util/docfields.py b/sphinx/util/docfields.py index 2f952d7cc..202616337 100644 --- a/sphinx/util/docfields.py +++ b/sphinx/util/docfields.py @@ -288,6 +288,12 @@ class DocFieldTransformer(object): fieldtype, fieldarg = fieldname.astext(), '' typedesc, is_typefield = typemap.get(fieldtype, (None, None)) + # collect the content, trying not to keep unnecessary paragraphs + if _is_single_paragraph(fieldbody): + content = fieldbody.children[0].children + else: + content = fieldbody.children + # sort out unknown fields if typedesc is None or typedesc.has_arg != bool(fieldarg): # either the field name is unknown, or the argument doesn't @@ -297,16 +303,27 @@ class DocFieldTransformer(object): new_fieldname += ' ' + fieldarg fieldname[0] = nodes.Text(new_fieldname) entries.append(field) + + # but if this has a type then we can at least link it + if typedesc and is_typefield and content: + target = content[0].astext() + xrefs = typedesc.make_xrefs( + typedesc.typerolename, + self.directive.domain, + target, + contnode=content[0], + ) + if _is_single_paragraph(fieldbody): + fieldbody.children[0].clear() + fieldbody.children[0].extend(xrefs) + else: + fieldbody.clear() + fieldbody.extend(xrefs) + continue typename = typedesc.name - # collect the content, trying not to keep unnecessary paragraphs - if _is_single_paragraph(fieldbody): - content = fieldbody.children[0].children - else: - content = fieldbody.children - # if the field specifies a type, put it in the types collection if is_typefield: # filter out only inline nodes; others will result in invalid diff --git a/sphinx/util/docutils.py b/sphinx/util/docutils.py index 38f275824..949bec3b4 100644 --- a/sphinx/util/docutils.py +++ b/sphinx/util/docutils.py @@ -12,6 +12,7 @@ from __future__ import absolute_import import re import types +import warnings from contextlib import contextmanager from copy import copy from distutils.version import LooseVersion @@ -23,6 +24,7 @@ from docutils.parsers.rst import directives, roles, convert_directive_function from docutils.statemachine import StateMachine from docutils.utils import Reporter +from sphinx.deprecation import RemovedInSphinx30Warning from sphinx.errors import ExtensionError from sphinx.locale import __ from sphinx.util import logging @@ -32,13 +34,14 @@ report_re = re.compile('^(.+?:(?:\\d+)?): \\((DEBUG|INFO|WARNING|ERROR|SEVERE)/( if False: # For type annotation - from typing import Any, Callable, Generator, Iterator, List, Tuple # NOQA + from typing import Any, Callable, Generator, Iterator, List, Set, Tuple # NOQA from docutils.statemachine import State, ViewList # NOQA from sphinx.environment import BuildEnvironment # NOQA from sphinx.io import SphinxFileInput # NOQA __version_info__ = tuple(LooseVersion(docutils.__version__).version) +additional_nodes = set() # type: Set[nodes.Node] @contextmanager @@ -54,6 +57,41 @@ def docutils_namespace(): directives._directives = _directives roles._roles = _roles + for node in list(additional_nodes): + unregister_node(node) + additional_nodes.discard(node) + + +def is_node_registered(node): + # type: (nodes.Node) -> bool + """Check the *node* is already registered.""" + return hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__) + + +def register_node(node): + # type: (nodes.Node) -> None + """Register a node to docutils. + + This modifies global state of some visitors. So it is better to use this + inside ``docutils_namespace()`` to prevent side-effects. + """ + if not hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__): + nodes._add_node_class_names([node.__name__]) + additional_nodes.add(node) + + +def unregister_node(node): + # type: (nodes.Node) -> None + """Unregister a node from docutils. + + This is inverse of ``nodes._add_nodes_class_names()``. + """ + if hasattr(nodes.GenericNodeVisitor, 'visit_' + node.__name__): + delattr(nodes.GenericNodeVisitor, "visit_" + node.__name__) + delattr(nodes.GenericNodeVisitor, "depart_" + node.__name__) + delattr(nodes.SparseNodeVisitor, 'visit_' + node.__name__) + delattr(nodes.SparseNodeVisitor, 'depart_' + node.__name__) + def patched_get_language(language_code, reporter=None): # type: (unicode, Reporter) -> Any @@ -200,6 +238,10 @@ def is_html5_writer_available(): def directive_helper(obj, has_content=None, argument_spec=None, **option_spec): # type: (Any, bool, Tuple[int, int, bool], Any) -> Any + warnings.warn('function based directive support is now deprecated. ' + 'Use class based directive instead.', + RemovedInSphinx30Warning) + if isinstance(obj, (types.FunctionType, types.MethodType)): obj.content = has_content # type: ignore obj.arguments = argument_spec or (0, 0, False) # type: ignore diff --git a/sphinx/util/i18n.py b/sphinx/util/i18n.py index 4989de1c9..9b6270bdf 100644 --- a/sphinx/util/i18n.py +++ b/sphinx/util/i18n.py @@ -21,6 +21,7 @@ from babel.messages.mofile import write_mo from babel.messages.pofile import read_po from sphinx.errors import SphinxError +from sphinx.locale import __ from sphinx.util import logging from sphinx.util.osutil import SEP, walk @@ -68,14 +69,14 @@ class CatalogInfo(LocaleFileInfoBase): try: po = read_po(file_po, locale) except Exception as exc: - logger.warning('reading error: %s, %s', self.po_path, exc) + logger.warning(__('reading error: %s, %s'), self.po_path, exc) return with io.open(self.mo_path, 'wb') as file_mo: try: write_mo(file_mo, po) except Exception as exc: - logger.warning('writing error: %s, %s', self.mo_path, exc) + logger.warning(__('writing error: %s, %s'), self.mo_path, exc) def find_catalog(docname, compaction): @@ -208,8 +209,8 @@ def babel_format_date(date, format, locale, formatter=babel.dates.format_date): # fallback to English return formatter(date, format, locale='en') except AttributeError: - logger.warning('Invalid date format. Quote the string by single quote ' - 'if you want to output it directly: %s', format) + logger.warning(__('Invalid date format. Quote the string by single quote ' + 'if you want to output it directly: %s'), format) return format diff --git a/sphinx/util/images.py b/sphinx/util/images.py index dd2f2a9e2..9abe748e4 100644 --- a/sphinx/util/images.py +++ b/sphinx/util/images.py @@ -123,6 +123,7 @@ def parse_data_uri(uri): def test_svg(h, f): + # type: (unicode, IO) -> unicode """An additional imghdr library helper; test the header is SVG's or not.""" try: if '<svg' in h.decode('utf-8').lower(): @@ -130,6 +131,8 @@ def test_svg(h, f): except UnicodeDecodeError: pass + return None + # install test_svg() to imghdr # refs: https://docs.python.org/3.6/library/imghdr.html#imghdr.tests diff --git a/sphinx/util/inspect.py b/sphinx/util/inspect.py index e8427fd84..6b404790c 100644 --- a/sphinx/util/inspect.py +++ b/sphinx/util/inspect.py @@ -580,6 +580,7 @@ else: # of Python 3.5 def _findclass(func): + # type: (Any) -> Any cls = sys.modules.get(func.__module__) if cls is None: return None @@ -593,6 +594,7 @@ else: return cls def _finddoc(obj): + # type: (Any) -> unicode if inspect.isclass(obj): for base in obj.__mro__: if base is not object: @@ -653,6 +655,7 @@ else: return None def getdoc(object): + # type: (Any) -> unicode """Get the documentation string for an object. All tabs are expanded to spaces. To clean up docstrings that are diff --git a/sphinx/util/logging.py b/sphinx/util/logging.py index 09db0028f..1ac4fb327 100644 --- a/sphinx/util/logging.py +++ b/sphinx/util/logging.py @@ -60,11 +60,18 @@ COLOR_MAP.update({ def getLogger(name): # type: (str) -> SphinxLoggerAdapter - """Get logger wrapped by SphinxLoggerAdapter. + """Get logger wrapped by :class:`sphinx.util.logging.SphinxLoggerAdapter`. Sphinx logger always uses ``sphinx.*`` namesapce to be independent from - settings of root logger. It enables to log stably even if 3rd party - extension or imported application resets logger settings. + settings of root logger. It ensure logging is consistent even if a + third-party extension or imported application resets logger settings. + + Example usage:: + + >>> from sphinx.utils import logging + >>> logger = logging.getLogger(__name__) + >>> logger.info('Hello, this is an extension!') + Hello, this is an extension! """ # add sphinx prefix to name forcely logger = logging.getLogger(NAMESPACE + '.' + name) @@ -217,7 +224,10 @@ class MemoryHandler(logging.handlers.BufferingHandler): @contextmanager def pending_warnings(): # type: () -> Generator - """contextmanager to pend logging warnings temporary.""" + """Contextmanager to pend logging warnings temporary. + + Similar to :func:`pending_logging`. + """ logger = logging.getLogger(NAMESPACE) memhandler = MemoryHandler() memhandler.setLevel(logging.WARNING) @@ -243,7 +253,16 @@ def pending_warnings(): @contextmanager def pending_logging(): # type: () -> Generator - """contextmanager to pend logging all logs temporary.""" + """Contextmanager to pend logging all logs temporary. + + For example:: + + >>> with pending_logging(): + >>> logger.warning('Warning message!') # not flushed yet + >>> some_long_process() + >>> + Warning message! # the warning is flushed here + """ logger = logging.getLogger(NAMESPACE) memhandler = MemoryHandler() diff --git a/sphinx/util/nodes.py b/sphinx/util/nodes.py index f439c515e..bdda1369b 100644 --- a/sphinx/util/nodes.py +++ b/sphinx/util/nodes.py @@ -16,7 +16,7 @@ from docutils import nodes from six import text_type from sphinx import addnodes -from sphinx.locale import pairindextypes +from sphinx.locale import __ from sphinx.util import logging if False: @@ -240,6 +240,8 @@ indextypes = [ def process_index_entry(entry, targetid): # type: (unicode, unicode) -> List[Tuple[unicode, unicode, unicode, unicode, unicode]] + from sphinx.domains.python import pairindextypes + indexentries = [] # type: List[Tuple[unicode, unicode, unicode, unicode, unicode]] entry = entry.strip() oentry = entry @@ -295,7 +297,7 @@ def inline_all_toctrees(builder, docnameset, docname, tree, colorfunc, traversed colorfunc, traversed) docnameset.add(includefile) except Exception: - logger.warning('toctree contains ref to nonexisting file %r', + logger.warning(__('toctree contains ref to nonexisting file %r'), includefile, location=docname) else: sof = addnodes.start_of_file(docname=includefile) @@ -368,7 +370,7 @@ def process_only_nodes(document, tags): try: ret = tags.eval_condition(node['expr']) except Exception as err: - logger.warning('exception while evaluating only directive expression: %s', err, + logger.warning(__('exception while evaluating only directive expression: %s'), err, location=node) node.replace_self(node.children or nodes.comment()) else: diff --git a/sphinx/util/osutil.py b/sphinx/util/osutil.py index af87caf9a..21464bbe6 100644 --- a/sphinx/util/osutil.py +++ b/sphinx/util/osutil.py @@ -19,11 +19,14 @@ import re import shutil import sys import time +import warnings from io import BytesIO, StringIO from os import path from six import PY2, PY3, text_type +from sphinx.deprecation import RemovedInSphinx30Warning + if False: # For type annotation from typing import Any, Iterator, List, Tuple, Union # NOQA @@ -181,8 +184,10 @@ def make_filename(string): def ustrftime(format, *args): # type: (unicode, Any) -> unicode - # [DEPRECATED] strftime for unicode strings - # It will be removed at Sphinx-1.5 + """[DEPRECATED] strftime for unicode strings.""" + warnings.warn('sphinx.util.osutil.ustrtime is deprecated for removal', + RemovedInSphinx30Warning) + if not args: # If time is not specified, try to use $SOURCE_DATE_EPOCH variable # See https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal @@ -197,7 +202,7 @@ def ustrftime(format, *args): return time.strftime(text_type(format).encode(enc), *args).decode(enc) else: # Py3 # On Windows, time.strftime() and Unicode characters will raise UnicodeEncodeError. - # http://bugs.python.org/issue8304 + # https://bugs.python.org/issue8304 try: return time.strftime(format, *args) except UnicodeEncodeError: @@ -220,7 +225,12 @@ def abspath(pathdir): # type: (unicode) -> unicode pathdir = path.abspath(pathdir) if isinstance(pathdir, bytes): - pathdir = pathdir.decode(fs_encoding) + try: + pathdir = pathdir.decode(fs_encoding) + except UnicodeDecodeError: + raise UnicodeDecodeError('multibyte filename not supported on ' + 'this filesystem encoding ' + '(%r)' % fs_encoding) return pathdir diff --git a/sphinx/util/rst.py b/sphinx/util/rst.py index f39549276..0e49f991a 100644 --- a/sphinx/util/rst.py +++ b/sphinx/util/rst.py @@ -17,6 +17,7 @@ from docutils.parsers.rst import roles from docutils.parsers.rst.languages import en as english from docutils.utils import Reporter +from sphinx.locale import __ from sphinx.util import logging if False: @@ -43,7 +44,7 @@ def default_role(docname, name): if role_fn: roles._roles[''] = role_fn else: - logger.warning('default role %s not found', name, location=docname) + logger.warning(__('default role %s not found'), name, location=docname) yield diff --git a/sphinx/util/smartypants.py b/sphinx/util/smartypants.py index bca901b18..03fc1816c 100644 --- a/sphinx/util/smartypants.py +++ b/sphinx/util/smartypants.py @@ -19,8 +19,8 @@ notices and this notice are preserved. This file is offered as-is, without any warranty. - .. _SmartyPants: http://daringfireball.net/projects/smartypants/ - .. _2-Clause BSD license: http://www.spdx.org/licenses/BSD-2-Clause + .. _SmartyPants: https://daringfireball.net/projects/smartypants/ + .. _2-Clause BSD license: https://spdx.org/licenses/BSD-2-Clause See the LICENSE file and the original docutils code for details. @@ -70,7 +70,7 @@ langquotes = {'af': u'“”‘’', 'he': u'”“»«', # Hebrew is RTL, test position: 'he-x-altquot': u'„”‚’', # low quotation marks are opening. # 'he-x-altquot': u'“„‘‚', # RTL: low quotation marks opening - 'hr': u'„”‘’', # http://hrvatska-tipografija.com/polunavodnici/ + 'hr': u'„”‘’', # https://hrvatska-tipografija.com/polunavodnici/ 'hr-x-altquot': u'»«›‹', 'hsb': u'„“‚‘', 'hsb-x-altquot': u'»«›‹', diff --git a/sphinx/util/typing.py b/sphinx/util/typing.py index 3cc4a46b5..a26dac473 100644 --- a/sphinx/util/typing.py +++ b/sphinx/util/typing.py @@ -22,3 +22,6 @@ if PY3: # common role functions RoleFunction = Callable[[unicode, unicode, unicode, int, Inliner, Dict, List[unicode]], Tuple[List[nodes.Node], List[nodes.Node]]] + +# title getter functions for enumerable nodes (see sphinx.domains.std) +TitleGetter = Callable[[nodes.Node], unicode] diff --git a/sphinx/util/websupport.py b/sphinx/util/websupport.py index 59133b9e1..59496ec02 100644 --- a/sphinx/util/websupport.py +++ b/sphinx/util/websupport.py @@ -10,5 +10,8 @@ try: from sphinxcontrib.websupport.utils import is_commentable # NOQA except ImportError: + from docutils import nodes # NOQA + def is_commentable(node): + # type: (nodes.Node) -> bool raise RuntimeError diff --git a/sphinx/versioning.py b/sphinx/versioning.py index bd0928775..a855cb8fe 100644 --- a/sphinx/versioning.py +++ b/sphinx/versioning.py @@ -158,6 +158,7 @@ class UIDTransform(SphinxTransform): default_priority = 100 def apply(self): + # type: () -> None env = self.env old_doctree = None if env.versioning_compare: @@ -177,6 +178,7 @@ class UIDTransform(SphinxTransform): def prepare(document): + # type: (nodes.Node) -> None """Simple wrapper for UIDTransform.""" transform = UIDTransform(document) transform.apply() diff --git a/sphinx/writers/html.py b/sphinx/writers/html.py index 0fc4a7bea..84dfb2aad 100644 --- a/sphinx/writers/html.py +++ b/sphinx/writers/html.py @@ -19,7 +19,7 @@ from docutils.writers.html4css1 import Writer, HTMLTranslator as BaseTranslator from six import string_types from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.images import get_image_size @@ -335,7 +335,7 @@ class HTMLTranslator(BaseTranslator): self.body.append('<span class="caption-number">') prefix = self.builder.config.numfig_format.get(figtype) if prefix is None: - msg = 'numfig_format is not defined for %s' % figtype + msg = __('numfig_format is not defined for %s') % figtype logger.warning(msg) else: numbers = self.builder.fignumbers[key][figure_id] @@ -345,7 +345,7 @@ class HTMLTranslator(BaseTranslator): figtype = self.builder.env.domains['std'].get_figtype(node) # type: ignore if figtype: if len(node['ids']) == 0: - msg = 'Any IDs not assigned for %s node' % node.tagname + msg = __('Any IDs not assigned for %s node') % node.tagname logger.warning(msg, location=node) else: append_fignumber(figtype, node['ids'][0]) @@ -625,7 +625,7 @@ class HTMLTranslator(BaseTranslator): if not ('width' in node and 'height' in node): size = get_image_size(os.path.join(self.builder.srcdir, olduri)) if size is None: - logger.warning('Could not obtain image size. :scale: option is ignored.', + logger.warning(__('Could not obtain image size. :scale: option is ignored.'), # NOQA location=node) else: if 'width' not in node: @@ -689,6 +689,7 @@ class HTMLTranslator(BaseTranslator): self.body.append('</td>') def visit_option_group(self, node): + # type: (nodes.Node) -> None BaseTranslator.visit_option_group(self, node) self.context[-2] = self.context[-2].replace(' ', ' ') @@ -845,6 +846,7 @@ class HTMLTranslator(BaseTranslator): node.column = 0 def visit_entry(self, node): + # type: (nodes.Node) -> None BaseTranslator.visit_entry(self, node) if self.body[-1] == ' ': self.body[-1] = ' ' @@ -864,6 +866,7 @@ class HTMLTranslator(BaseTranslator): self.body.append(self.starttag(node, 'tr', '', CLASS='field')) def visit_field_name(self, node): + # type: (nodes.Node) -> None context_count = len(self.context) BaseTranslator.visit_field_name(self, node) if context_count != len(self.context): @@ -871,9 +874,9 @@ class HTMLTranslator(BaseTranslator): def visit_math(self, node, math_env=''): # type: (nodes.Node, unicode) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/ext/math.html'), location=(self.builder.current_docname, node.line)) raise nodes.SkipNode diff --git a/sphinx/writers/html5.py b/sphinx/writers/html5.py index 21d7626ef..b38421db8 100644 --- a/sphinx/writers/html5.py +++ b/sphinx/writers/html5.py @@ -18,7 +18,7 @@ from docutils.writers.html5_polyglot import HTMLTranslator as BaseTranslator from six import string_types from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.images import get_image_size @@ -303,7 +303,7 @@ class HTML5Translator(BaseTranslator): self.body.append('<span class="caption-number">') prefix = self.builder.config.numfig_format.get(figtype) if prefix is None: - msg = 'numfig_format is not defined for %s' % figtype + msg = __('numfig_format is not defined for %s') % figtype logger.warning(msg) else: numbers = self.builder.fignumbers[key][figure_id] @@ -313,7 +313,7 @@ class HTML5Translator(BaseTranslator): figtype = self.builder.env.domains['std'].get_figtype(node) # type: ignore if figtype: if len(node['ids']) == 0: - msg = 'Any IDs not assigned for %s node' % node.tagname + msg = __('Any IDs not assigned for %s node') % node.tagname logger.warning(msg, location=node) else: append_fignumber(figtype, node['ids'][0]) @@ -571,7 +571,7 @@ class HTML5Translator(BaseTranslator): if not ('width' in node and 'height' in node): size = get_image_size(os.path.join(self.builder.srcdir, olduri)) if size is None: - logger.warning('Could not obtain image size. :scale: option is ignored.', + logger.warning(__('Could not obtain image size. :scale: option is ignored.'), # NOQA location=node) else: if 'width' not in node: @@ -825,9 +825,9 @@ class HTML5Translator(BaseTranslator): def visit_math(self, node, math_env=''): # type: (nodes.Node, unicode) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/ext/math.html'), location=(self.builder.current_docname, node.line)) raise nodes.SkipNode diff --git a/sphinx/writers/latex.py b/sphinx/writers/latex.py index ab15ec8b5..d87de1f19 100644 --- a/sphinx/writers/latex.py +++ b/sphinx/writers/latex.py @@ -24,7 +24,7 @@ from six import itervalues, text_type from sphinx import addnodes from sphinx import highlighting from sphinx.errors import SphinxError -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.transforms import SphinxTransform from sphinx.util import split_into, logging from sphinx.util.i18n import format_date @@ -574,13 +574,13 @@ class LaTeXTranslator(nodes.NodeVisitor): self.top_sectionlevel = \ self.sectionnames.index(builder.config.latex_toplevel_sectioning) except ValueError: - logger.warning('unknown %r toplevel_sectioning for class %r' % + logger.warning(__('unknown %r toplevel_sectioning for class %r') % (builder.config.latex_toplevel_sectioning, docclass)) if builder.config.today: self.elements['date'] = builder.config.today else: - self.elements['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), # type: ignore # NOQA + self.elements['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), language=builder.config.language) if builder.config.numfig: @@ -621,7 +621,7 @@ class LaTeXTranslator(nodes.NodeVisitor): if builder.config.language and not self.babel.is_supported_language(): # emit warning if specified language is invalid # (only emitting, nothing changed to processing) - logger.warning('no Babel option known for language %r', + logger.warning(__('no Babel option known for language %r'), builder.config.language) # simply use babel.get_language() always, as get_language() returns @@ -677,7 +677,7 @@ class LaTeXTranslator(nodes.NodeVisitor): self.top_sectionlevel > 0: tocdepth += 1 # because top_sectionlevel is shifted by -1 if tocdepth > len(LATEXSECTIONNAMES) - 2: # default is 5 <-> subparagraph - logger.warning('too large :maxdepth:, ignored.') + logger.warning(__('too large :maxdepth:, ignored.')) tocdepth = len(LATEXSECTIONNAMES) - 2 self.elements['tocdepth'] = '\\setcounter{tocdepth}{%d}' % tocdepth @@ -765,7 +765,7 @@ class LaTeXTranslator(nodes.NodeVisitor): # type: () -> None for key in self.builder.config.latex_elements: if key not in self.elements: - msg = _("Unknown configure key: latex_elements[%r] is ignored.") + msg = __("Unknown configure key: latex_elements[%r] is ignored.") logger.warning(msg % key) def restrict_footnote(self, node): @@ -1091,7 +1091,7 @@ class LaTeXTranslator(nodes.NodeVisitor): if self.this_is_the_title: if len(node.children) != 1 and not isinstance(node.children[0], nodes.Text): - logger.warning('document title is not a single Text node', + logger.warning(__('document title is not a single Text node'), location=(self.curfilestack[-1], node.line)) if not self.elements['title']: # text needs to be escaped since it is inserted into @@ -1131,8 +1131,8 @@ class LaTeXTranslator(nodes.NodeVisitor): self.pushbody([]) self.restrict_footnote(node) else: - logger.warning('encountered title node not in section, topic, table, ' - 'admonition or sidebar', + logger.warning(__('encountered title node not in section, topic, table, ' + 'admonition or sidebar'), location=(self.curfilestack[-1], node.line or '')) self.body.append('\\sphinxstyleothertitle{') self.context.append('}\n') @@ -1741,7 +1741,7 @@ class LaTeXTranslator(nodes.NodeVisitor): try: return rstdim_to_latexdim(width_str) except ValueError: - logger.warning('dimension unit %s is invalid. Ignored.', width_str) + logger.warning(__('dimension unit %s is invalid. Ignored.'), width_str) return None def is_inline(self, node): @@ -1785,7 +1785,7 @@ class LaTeXTranslator(nodes.NodeVisitor): (0, 'center'): ('{\\hspace*{\\fill}', '\\hspace*{\\fill}}'), # These 2 don't exactly do the right thing. The image should # be floated alongside the paragraph. See - # http://www.w3.org/TR/html4/struct/objects.html#adef-align-IMG + # https://www.w3.org/TR/html4/struct/objects.html#adef-align-IMG (0, 'left'): ('{', '\\hspace*{\\fill}}'), (0, 'right'): ('{\\hspace*{\\fill}', '}'), } @@ -1898,9 +1898,11 @@ class LaTeXTranslator(nodes.NodeVisitor): self.unrestrict_footnote(node) def visit_legend(self, node): + # type: (nodes.Node) -> None self.body.append('\n\\begin{sphinxlegend}') def depart_legend(self, node): + # type: (nodes.Node) -> None self.body.append('\\end{sphinxlegend}\n') def visit_admonition(self, node): @@ -2055,7 +2057,7 @@ class LaTeXTranslator(nodes.NodeVisitor): p1, p2 = [self.encode(x) for x in split_into(2, 'seealso', string)] self.body.append(r'\index{%s|see{%s}}' % (p1, p2)) else: - logger.warning('unknown index entry type %s found', type) + logger.warning(__('unknown index entry type %s found'), type) except ValueError as err: logger.warning(str(err)) if not node.get('inline', True): @@ -2627,9 +2629,9 @@ class LaTeXTranslator(nodes.NodeVisitor): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/ext/math.html'), location=(self.curfilestack[-1], node.line)) raise nodes.SkipNode diff --git a/sphinx/writers/manpage.py b/sphinx/writers/manpage.py index 38954ca81..76884cbf6 100644 --- a/sphinx/writers/manpage.py +++ b/sphinx/writers/manpage.py @@ -18,7 +18,7 @@ from docutils.writers.manpage import ( import sphinx.util.docutils from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.i18n import format_date @@ -107,7 +107,7 @@ class ManualPageTranslator(BaseTranslator): if builder.config.today: self._docinfo['date'] = builder.config.today else: - self._docinfo['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), # type: ignore # NOQA + self._docinfo['date'] = format_date(builder.config.today_fmt or _('%b %d, %Y'), language=builder.config.language) self._docinfo['copyright'] = builder.config.copyright self._docinfo['version'] = builder.config.version @@ -513,9 +513,9 @@ class ManualPageTranslator(BaseTranslator): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/ext/math.html') + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/ext/math.html')) raise nodes.SkipNode visit_math_block = visit_math diff --git a/sphinx/writers/texinfo.py b/sphinx/writers/texinfo.py index cce269479..36c48131f 100644 --- a/sphinx/writers/texinfo.py +++ b/sphinx/writers/texinfo.py @@ -19,7 +19,7 @@ from six.moves import range from sphinx import addnodes, __display_version__ from sphinx.errors import ExtensionError -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging from sphinx.util.i18n import format_date from sphinx.writers.latex import collected_footnote @@ -237,7 +237,7 @@ class TexinfoTranslator(nodes.NodeVisitor): 'project': self.escape(self.builder.config.project), 'copyright': self.escape(self.builder.config.copyright), 'date': self.escape(self.builder.config.today or - format_date(self.builder.config.today_fmt or _('%b %d, %Y'), # type: ignore # NOQA + format_date(self.builder.config.today_fmt or _('%b %d, %Y'), language=self.builder.config.language)) }) # title @@ -648,8 +648,8 @@ class TexinfoTranslator(nodes.NodeVisitor): if isinstance(parent, (nodes.Admonition, nodes.sidebar, nodes.topic)): raise nodes.SkipNode elif not isinstance(parent, nodes.section): - logger.warning('encountered title node not in section, topic, table, ' - 'admonition or sidebar', + logger.warning(__('encountered title node not in section, topic, table, ' + 'admonition or sidebar'), location=(self.curfilestack[-1], node.line)) self.visit_rubric(node) else: @@ -1318,7 +1318,7 @@ class TexinfoTranslator(nodes.NodeVisitor): node.parent.get('literal_block'))): self.body.append('\n@caption{') else: - logger.warning('caption not inside a figure.', + logger.warning(__('caption not inside a figure.'), location=(self.curfilestack[-1], node.line)) def depart_caption(self, node): @@ -1421,12 +1421,12 @@ class TexinfoTranslator(nodes.NodeVisitor): def unimplemented_visit(self, node): # type: (nodes.Node) -> None - logger.warning("unimplemented node type: %r", node, + logger.warning(__("unimplemented node type: %r"), node, location=(self.curfilestack[-1], node.line)) def unknown_visit(self, node): # type: (nodes.Node) -> None - logger.warning("unknown node type: %r", node, + logger.warning(__("unknown node type: %r"), node, location=(self.curfilestack[-1], node.line)) def unknown_departure(self, node): @@ -1743,9 +1743,9 @@ class TexinfoTranslator(nodes.NodeVisitor): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/ext/math.html') + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/ext/math.html')) raise nodes.SkipNode visit_math_block = visit_math diff --git a/sphinx/writers/text.py b/sphinx/writers/text.py index 5e870efa2..79789fcca 100644 --- a/sphinx/writers/text.py +++ b/sphinx/writers/text.py @@ -18,7 +18,7 @@ from docutils.utils import column_width from six.moves import zip_longest from sphinx import addnodes -from sphinx.locale import admonitionlabels, _ +from sphinx.locale import admonitionlabels, _, __ from sphinx.util import logging if False: @@ -1183,9 +1183,9 @@ class TextTranslator(nodes.NodeVisitor): def visit_math(self, node): # type: (nodes.Node) -> None - logger.warning('using "math" markup without a Sphinx math extension ' - 'active, please use one of the math extensions ' - 'described at http://sphinx-doc.org/ext/math.html', + logger.warning(__('using "math" markup without a Sphinx math extension ' + 'active, please use one of the math extensions ' + 'described at http://sphinx-doc.org/ext/math.html'), location=(self.builder.current_docname, node.line)) raise nodes.SkipNode diff --git a/tests/etree13/ElementPath.py b/tests/etree13/ElementPath.py deleted file mode 100644 index 8cf1ab578..000000000 --- a/tests/etree13/ElementPath.py +++ /dev/null @@ -1,226 +0,0 @@ -# -# ElementTree -# $Id$ -# -# limited xpath support for element trees -# -# history: -# 2003-05-23 fl created -# 2003-05-28 fl added support for // etc -# 2003-08-27 fl fixed parsing of periods in element names -# 2007-09-10 fl new selection engine -# -# Copyright (c) 2003-2007 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2007 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -## -# Implementation module for XPath support. There's usually no reason -# to import this module directly; the <b>ElementTree</b> does this for -# you, if needed. -## - -import re - -xpath_tokenizer = re.compile( - r"(" - r"'[^']*'|\"[^\"]*\"|" - r"::|" - r"//?|" - r"\.\.|" - r"\(\)|" - r"[/.*:\[\]\(\)@=])|" - r"((?:\{[^}]+\})?[^/:\[\]\(\)@=\s]+)|" - r"\s+" - ).findall - -def prepare_tag(next, token): - tag = token[1] - def select(context, result): - for elem in result: - for e in elem: - if e.tag == tag: - yield e - return select - -def prepare_star(next, token): - def select(context, result): - for elem in result: - for e in elem: - yield e - return select - -def prepare_dot(next, token): - def select(context, result): - for elem in result: - yield elem - return select - -def prepare_iter(next, token): - token = next() - if token[0] == "*": - tag = "*" - elif not token[0]: - tag = token[1] - else: - raise SyntaxError - def select(context, result): - for elem in result: - for e in elem.iter(tag): - if e is not elem: - yield e - return select - -def prepare_dot_dot(next, token): - def select(context, result): - parent_map = context.parent_map - if parent_map is None: - context.parent_map = parent_map = {} - for p in context.root.iter(): - for e in p: - parent_map[e] = p - for elem in result: - if elem in parent_map: - yield parent_map[elem] - return select - -def prepare_predicate(next, token): - # this one should probably be refactored... - token = next() - if token[0] == "@": - # attribute - token = next() - if token[0]: - raise SyntaxError("invalid attribute predicate") - key = token[1] - token = next() - if token[0] == "]": - def select(context, result): - for elem in result: - if elem.get(key) is not None: - yield elem - elif token[0] == "=": - value = next()[0] - if value[:1] == "'" or value[:1] == '"': - value = value[1:-1] - else: - raise SyntaxError("invalid comparision target") - token = next() - def select(context, result): - for elem in result: - if elem.get(key) == value: - yield elem - if token[0] != "]": - raise SyntaxError("invalid attribute predicate") - elif not token[0]: - tag = token[1] - token = next() - if token[0] != "]": - raise SyntaxError("invalid node predicate") - def select(context, result): - for elem in result: - if elem.find(tag) is not None: - yield elem - else: - raise SyntaxError("invalid predicate") - return select - -ops = { - "": prepare_tag, - "*": prepare_star, - ".": prepare_dot, - "..": prepare_dot_dot, - "//": prepare_iter, - "[": prepare_predicate, - } - -_cache = {} - -class _SelectorContext: - parent_map = None - def __init__(self, root): - self.root = root - -# -------------------------------------------------------------------- - -## -# Find first matching object. - -def find(elem, path): - try: - return next(findall(elem, path)) - except StopIteration: - return None - -## -# Find all matching objects. - -def findall(elem, path): - # compile selector pattern - try: - selector = _cache[path] - except KeyError: - if len(_cache) > 100: - _cache.clear() - if path[:1] == "/": - raise SyntaxError("cannot use absolute path on element") - stream = iter(xpath_tokenizer(path)) - next_ = lambda: next(stream); token = next_() - selector = [] - while 1: - try: - selector.append(ops[token[0]](next_, token)) - except StopIteration: - raise SyntaxError("invalid path") - try: - token = next_() - if token[0] == "/": - token = next_() - except StopIteration: - break - _cache[path] = selector - # execute selector pattern - result = [elem] - context = _SelectorContext(elem) - for select in selector: - result = select(context, result) - return result - -## -# Find text for first matching object. - -def findtext(elem, path, default=None): - try: - elem = next(findall(elem, path)) - return elem.text - except StopIteration: - return default diff --git a/tests/etree13/ElementTree.py b/tests/etree13/ElementTree.py deleted file mode 100644 index 134abf313..000000000 --- a/tests/etree13/ElementTree.py +++ /dev/null @@ -1,1553 +0,0 @@ -# -# ElementTree -# $Id$ -# -# light-weight XML support for Python 2.2 and later. -# -# history: -# 2001-10-20 fl created (from various sources) -# 2001-11-01 fl return root from parse method -# 2002-02-16 fl sort attributes in lexical order -# 2002-04-06 fl TreeBuilder refactoring, added PythonDoc markup -# 2002-05-01 fl finished TreeBuilder refactoring -# 2002-07-14 fl added basic namespace support to ElementTree.write -# 2002-07-25 fl added QName attribute support -# 2002-10-20 fl fixed encoding in write -# 2002-11-24 fl changed default encoding to ascii; fixed attribute encoding -# 2002-11-27 fl accept file objects or file names for parse/write -# 2002-12-04 fl moved XMLTreeBuilder back to this module -# 2003-01-11 fl fixed entity encoding glitch for us-ascii -# 2003-02-13 fl added XML literal factory -# 2003-02-21 fl added ProcessingInstruction/PI factory -# 2003-05-11 fl added tostring/fromstring helpers -# 2003-05-26 fl added ElementPath support -# 2003-07-05 fl added makeelement factory method -# 2003-07-28 fl added more well-known namespace prefixes -# 2003-08-15 fl fixed typo in ElementTree.findtext (Thomas Dartsch) -# 2003-09-04 fl fall back on emulator if ElementPath is not installed -# 2003-10-31 fl markup updates -# 2003-11-15 fl fixed nested namespace bug -# 2004-03-28 fl added XMLID helper -# 2004-06-02 fl added default support to findtext -# 2004-06-08 fl fixed encoding of non-ascii element/attribute names -# 2004-08-23 fl take advantage of post-2.1 expat features -# 2004-09-03 fl made Element class visible; removed factory -# 2005-02-01 fl added iterparse implementation -# 2005-03-02 fl fixed iterparse support for pre-2.2 versions -# 2005-11-12 fl added tostringlist/fromstringlist helpers -# 2006-07-05 fl merged in selected changes from the 1.3 sandbox -# 2006-07-05 fl removed support for 2.1 and earlier -# 2007-06-21 fl added deprecation/future warnings -# 2007-08-25 fl added doctype hook, added parser version attribute etc -# 2007-08-26 fl added new serializer code (better namespace handling, etc) -# 2007-08-27 fl warn for broken /tag searches on tree level -# 2007-09-02 fl added html/text methods to serializer (experimental) -# 2007-09-05 fl added method argument to tostring/tostringlist -# 2007-09-06 fl improved error handling -# -# Copyright (c) 1999-2007 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2007 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -from __future__ import generators -from __future__ import absolute_import - -from six import string_types - - -__all__ = [ - # public symbols - "Comment", - "dump", - "Element", "ElementTree", - "fromstring", "fromstringlist", - "iselement", "iterparse", - "parse", "ParseError", - "PI", "ProcessingInstruction", - "QName", - "SubElement", - "tostring", "tostringlist", - "TreeBuilder", - "VERSION", - "XML", - "XMLParser", "XMLTreeBuilder", - ] - -## -# The <b>Element</b> type is a flexible container object, designed to -# store hierarchical data structures in memory. The type can be -# described as a cross between a list and a dictionary. -# <p> -# Each element has a number of properties associated with it: -# <ul> -# <li>a <i>tag</i>. This is a string identifying what kind of data -# this element represents (the element type, in other words).</li> -# <li>a number of <i>attributes</i>, stored in a Python dictionary.</li> -# <li>a <i>text</i> string.</li> -# <li>an optional <i>tail</i> string.</li> -# <li>a number of <i>child elements</i>, stored in a Python sequence</li> -# </ul> -# -# To create an element instance, use the {@link #Element} constructor -# or the {@link #SubElement} factory function. -# <p> -# The {@link #ElementTree} class can be used to wrap an element -# structure, and convert it from and to XML. -## - -import sys, re - -class _SimpleElementPath(object): - # emulate pre-1.2 find/findtext/findall behaviour - def find(self, element, tag): - for elem in element: - if elem.tag == tag: - return elem - return None - def findtext(self, element, tag, default=None): - for elem in element: - if elem.tag == tag: - return elem.text or "" - return default - def findall(self, element, tag): - if tag[:3] == ".//": - return element.getiterator(tag[3:]) - result = [] - for elem in element: - if elem.tag == tag: - result.append(elem) - return result - -try: - from . import ElementPath -except ImportError: - # FIXME: issue warning in this case? - ElementPath = _SimpleElementPath() - -VERSION = "1.3a2" - -class ParseError(SyntaxError): - pass - -# -------------------------------------------------------------------- - -## -# Checks if an object appears to be a valid element object. -# -# @param An element instance. -# @return A true value if this is an element object. -# @defreturn flag - -def iselement(element): - # FIXME: not sure about this; might be a better idea to look - # for tag/attrib/text attributes - return isinstance(element, Element) or hasattr(element, "tag") - -## -# Element class. This class defines the Element interface, and -# provides a reference implementation of this interface. -# <p> -# The element name, attribute names, and attribute values can be -# either 8-bit ASCII strings or Unicode strings. -# -# @param tag The element name. -# @param attrib An optional dictionary, containing element attributes. -# @param **extra Additional attributes, given as keyword arguments. -# @see Element -# @see SubElement -# @see Comment -# @see ProcessingInstruction - -class Element(object): - # <tag attrib>text<child/>...</tag>tail - - ## - # (Attribute) Element tag. - - tag = None - - ## - # (Attribute) Element attribute dictionary. Where possible, use - # {@link #Element.get}, - # {@link #Element.set}, - # {@link #Element.keys}, and - # {@link #Element.items} to access - # element attributes. - - attrib = None - - ## - # (Attribute) Text before first subelement. This is either a - # string or the value None, if there was no text. - - text = None - - ## - # (Attribute) Text after this element's end tag, but before the - # next sibling element's start tag. This is either a string or - # the value None, if there was no text. - - tail = None # text after end tag, if any - - def __init__(self, tag, attrib={}, **extra): - attrib = attrib.copy() - attrib.update(extra) - self.tag = tag - self.attrib = attrib - self._children = [] - - def __repr__(self): - return "<Element %s at %x>" % (repr(self.tag), id(self)) - - ## - # Creates a new element object of the same type as this element. - # - # @param tag Element tag. - # @param attrib Element attributes, given as a dictionary. - # @return A new element instance. - - def makeelement(self, tag, attrib): - return Element(tag, attrib) - - ## - # Returns the number of subelements. - # - # @return The number of subelements. - - def __len__(self): - return len(self._children) - - def __bool__(self): - import warnings - warnings.warn( - "The behavior of this method will change in future versions. " - "Use specific 'len(elem)' or 'elem is not None' test instead.", - FutureWarning - ) - return len(self._children) != 0 # emulate old behaviour - __nonzero__ = __bool__ # for python2 compatibility - - ## - # Returns the given subelement. - # - # @param index What subelement to return. - # @return The given subelement. - # @exception IndexError If the given element does not exist. - - def __getitem__(self, index): - return self._children[index] - - ## - # Replaces the given subelement. - # - # @param index What subelement to replace. - # @param element The new element value. - # @exception IndexError If the given element does not exist. - # @exception AssertionError If element is not a valid object. - - def __setitem__(self, index, element): - assert iselement(element) - self._children[index] = element - - ## - # Deletes the given subelement. - # - # @param index What subelement to delete. - # @exception IndexError If the given element does not exist. - - def __delitem__(self, index): - del self._children[index] - - ## - # Returns a list containing subelements in the given range. - # - # @param start The first subelement to return. - # @param stop The first subelement that shouldn't be returned. - # @return A sequence object containing subelements. - - def __getslice__(self, start, stop): - return self._children[start:stop] - - ## - # Replaces a number of subelements with elements from a sequence. - # - # @param start The first subelement to replace. - # @param stop The first subelement that shouldn't be replaced. - # @param elements A sequence object with zero or more elements. - # @exception AssertionError If a sequence member is not a valid object. - - def __setslice__(self, start, stop, elements): - for element in elements: - assert iselement(element) - self._children[start:stop] = list(elements) - - ## - # Deletes a number of subelements. - # - # @param start The first subelement to delete. - # @param stop The first subelement to leave in there. - - def __delslice__(self, start, stop): - del self._children[start:stop] - - ## - # Adds a subelement to the end of this element. - # - # @param element The element to add. - # @exception AssertionError If a sequence member is not a valid object. - - def append(self, element): - assert iselement(element) - self._children.append(element) - - ## - # Appends subelements from a sequence. - # - # @param elements A sequence object with zero or more elements. - # @exception AssertionError If a subelement is not a valid object. - # @since 1.3 - - def extend(self, elements): - for element in elements: - assert iselement(element) - self._children.extend(elements) - - ## - # Inserts a subelement at the given position in this element. - # - # @param index Where to insert the new subelement. - # @exception AssertionError If the element is not a valid object. - - def insert(self, index, element): - assert iselement(element) - self._children.insert(index, element) - - ## - # Removes a matching subelement. Unlike the <b>find</b> methods, - # this method compares elements based on identity, not on tag - # value or contents. - # - # @param element What element to remove. - # @exception ValueError If a matching element could not be found. - # @exception AssertionError If the element is not a valid object. - - def remove(self, element): - assert iselement(element) - self._children.remove(element) - - ## - # (Deprecated) Returns all subelements. The elements are returned - # in document order. - # - # @return A list of subelements. - # @defreturn list of Element instances - - def getchildren(self): - import warnings - warnings.warn( - "This method will be removed in future versions. " - "Use 'list(elem)' or iteration over elem instead.", - DeprecationWarning - ) - return self._children - - ## - # Finds the first matching subelement, by tag name or path. - # - # @param path What element to look for. - # @return The first matching element, or None if no element was found. - # @defreturn Element or None - - def find(self, path): - return ElementPath.find(self, path) - - ## - # Finds text for the first matching subelement, by tag name or path. - # - # @param path What element to look for. - # @param default What to return if the element was not found. - # @return The text content of the first matching element, or the - # default value no element was found. Note that if the element - # has is found, but has no text content, this method returns an - # empty string. - # @defreturn string - - def findtext(self, path, default=None): - return ElementPath.findtext(self, path, default) - - ## - # Finds all matching subelements, by tag name or path. - # - # @param path What element to look for. - # @return A list or iterator containing all matching elements, - # in document order. - # @defreturn list of Element instances - - def findall(self, path): - return ElementPath.findall(self, path) - - ## - # Resets an element. This function removes all subelements, clears - # all attributes, and sets the text and tail attributes to None. - - def clear(self): - self.attrib.clear() - self._children = [] - self.text = self.tail = None - - ## - # Gets an element attribute. - # - # @param key What attribute to look for. - # @param default What to return if the attribute was not found. - # @return The attribute value, or the default value, if the - # attribute was not found. - # @defreturn string or None - - def get(self, key, default=None): - return self.attrib.get(key, default) - - ## - # Sets an element attribute. - # - # @param key What attribute to set. - # @param value The attribute value. - - def set(self, key, value): - self.attrib[key] = value - - ## - # Gets a list of attribute names. The names are returned in an - # arbitrary order (just like for an ordinary Python dictionary). - # - # @return A list of element attribute names. - # @defreturn list of strings - - def keys(self): - return self.attrib.keys() - - ## - # Gets element attributes, as a sequence. The attributes are - # returned in an arbitrary order. - # - # @return A list of (name, value) tuples for all attributes. - # @defreturn list of (string, string) tuples - - def items(self): - return self.attrib.items() - - ## - # Creates a tree iterator. The iterator loops over this element - # and all subelements, in document order, and returns all elements - # with a matching tag. - # <p> - # If the tree structure is modified during iteration, new or removed - # elements may or may not be included. To get a stable set, use the - # list() function on the iterator, and loop over the resulting list. - # - # @param tag What tags to look for (default is to return all elements). - # @return An iterator containing all the matching elements. - # @defreturn iterator - - def iter(self, tag=None): - if tag == "*": - tag = None - if tag is None or self.tag == tag: - yield self - for e in self._children: - for e in e.iter(tag): - yield e - - # compatibility (FIXME: preserve list behaviour too? see below) - getiterator = iter - - # def getiterator(self, tag=None): - # return list(tag) - - ## - # Creates a text iterator. The iterator loops over this element - # and all subelements, in document order, and returns all inner - # text. - # - # @return An iterator containing all inner text. - # @defreturn iterator - - def itertext(self): - if self.text: - yield self.text - for e in self: - for s in e.itertext(): - yield s - if e.tail: - yield e.tail - -# compatibility -_Element = _ElementInterface = Element - -## -# Subelement factory. This function creates an element instance, and -# appends it to an existing element. -# <p> -# The element name, attribute names, and attribute values can be -# either 8-bit ASCII strings or Unicode strings. -# -# @param parent The parent element. -# @param tag The subelement name. -# @param attrib An optional dictionary, containing element attributes. -# @param **extra Additional attributes, given as keyword arguments. -# @return An element instance. -# @defreturn Element - -def SubElement(parent, tag, attrib={}, **extra): - attrib = attrib.copy() - attrib.update(extra) - element = parent.makeelement(tag, attrib) - parent.append(element) - return element - -## -# Comment element factory. This factory function creates a special -# element that will be serialized as an XML comment by the standard -# serializer. -# <p> -# The comment string can be either an 8-bit ASCII string or a Unicode -# string. -# -# @param text A string containing the comment string. -# @return An element instance, representing a comment. -# @defreturn Element - -def Comment(text=None): - element = Element(Comment) - element.text = text - return element - -## -# PI element factory. This factory function creates a special element -# that will be serialized as an XML processing instruction by the standard -# serializer. -# -# @param target A string containing the PI target. -# @param text A string containing the PI contents, if any. -# @return An element instance, representing a PI. -# @defreturn Element - -def ProcessingInstruction(target, text=None): - element = Element(ProcessingInstruction) - element.text = target - if text: - element.text = element.text + " " + text - return element - -PI = ProcessingInstruction - -## -# QName wrapper. This can be used to wrap a QName attribute value, in -# order to get proper namespace handling on output. -# -# @param text A string containing the QName value, in the form {uri}local, -# or, if the tag argument is given, the URI part of a QName. -# @param tag Optional tag. If given, the first argument is interpreted as -# an URI, and this argument is interpreted as a local name. -# @return An opaque object, representing the QName. - -class QName(object): - def __init__(self, text_or_uri, tag=None): - if tag: - text_or_uri = "{%s}%s" % (text_or_uri, tag) - self.text = text_or_uri - def __str__(self): - return self.text - def __hash__(self): - return hash(self.text) - def __cmp__(self, other): - if isinstance(other, QName): - return cmp(self.text, other.text) - return cmp(self.text, other) - -# -------------------------------------------------------------------- - -## -# ElementTree wrapper class. This class represents an entire element -# hierarchy, and adds some extra support for serialization to and from -# standard XML. -# -# @param element Optional root element. -# @keyparam file Optional file handle or file name. If given, the -# tree is initialized with the contents of this XML file. - -class ElementTree(object): - - def __init__(self, element=None, file=None): - assert element is None or iselement(element) - self._root = element # first node - if file: - self.parse(file) - - ## - # Gets the root element for this tree. - # - # @return An element instance. - # @defreturn Element - - def getroot(self): - return self._root - - ## - # Replaces the root element for this tree. This discards the - # current contents of the tree, and replaces it with the given - # element. Use with care. - # - # @param element An element instance. - - def _setroot(self, element): - assert iselement(element) - self._root = element - - ## - # Loads an external XML document into this element tree. - # - # @param source A file name or file object. - # @keyparam parser An optional parser instance. If not given, the - # standard {@link XMLParser} parser is used. - # @return The document root element. - # @defreturn Element - - def parse(self, source, parser=None): - if not hasattr(source, "read"): - source = open(source, "rb") - if not parser: - parser = XMLParser(target=TreeBuilder()) - while 1: - data = source.read(32768) - if not data: - break - parser.feed(data) - self._root = parser.close() - return self._root - - ## - # Creates a tree iterator for the root element. The iterator loops - # over all elements in this tree, in document order. - # - # @param tag What tags to look for (default is to return all elements) - # @return An iterator. - # @defreturn iterator - - def iter(self, tag=None): - assert self._root is not None - return self._root.iter(tag) - - getiterator = iter - - ## - # Finds the first toplevel element with given tag. - # Same as getroot().find(path). - # - # @param path What element to look for. - # @return The first matching element, or None if no element was found. - # @defreturn Element or None - - def find(self, path): - assert self._root is not None - if path[:1] == "/": - path = "." + path - import warnings - warnings.warn( - "This search is broken in 1.3 and earlier; if you rely " - "on the current behaviour, change it to %r" % path, - FutureWarning - ) - return self._root.find(path) - - ## - # Finds the element text for the first toplevel element with given - # tag. Same as getroot().findtext(path). - # - # @param path What toplevel element to look for. - # @param default What to return if the element was not found. - # @return The text content of the first matching element, or the - # default value no element was found. Note that if the element - # has is found, but has no text content, this method returns an - # empty string. - # @defreturn string - - def findtext(self, path, default=None): - assert self._root is not None - if path[:1] == "/": - path = "." + path - import warnings - warnings.warn( - "This search is broken in 1.3 and earlier; if you rely " - "on the current behaviour, change it to %r" % path, - FutureWarning - ) - return self._root.findtext(path, default) - - ## - # Finds all toplevel elements with the given tag. - # Same as getroot().findall(path). - # - # @param path What element to look for. - # @return A list or iterator containing all matching elements, - # in document order. - # @defreturn list of Element instances - - def findall(self, path): - assert self._root is not None - if path[:1] == "/": - path = "." + path - import warnings - warnings.warn( - "This search is broken in 1.3 and earlier; if you rely " - "on the current behaviour, change it to %r" % path, - FutureWarning - ) - return self._root.findall(path) - - ## - # Writes the element tree to a file, as XML. - # - # @param file A file name, or a file object opened for writing. - # @keyparam encoding Optional output encoding (default is US-ASCII). - # @keyparam method Optional output method ("xml" or "html"; default - # is "xml". - # @keyparam xml_declaration Controls if an XML declaration should - # be added to the file. Use False for never, True for always, - # None for only if not US-ASCII or UTF-8. None is default. - - def write(self, file, - # keyword arguments - encoding="us-ascii", - xml_declaration=None, - default_namespace=None, - method=None): - assert self._root is not None - if not hasattr(file, "write"): - file = open(file, "wb") - write = file.write - if not method: - method = "xml" - if not encoding: - encoding = "us-ascii" - elif xml_declaration or (xml_declaration is None and - encoding not in ("utf-8", "us-ascii")): - write("<?xml version='1.0' encoding='%s'?>\n" % encoding) - if method == "text": - _serialize_text(write, self._root, encoding) - else: - qnames, namespaces = _namespaces( - self._root, encoding, default_namespace - ) - if method == "xml": - _serialize_xml( - write, self._root, encoding, qnames, namespaces - ) - elif method == "html": - _serialize_html( - write, self._root, encoding, qnames, namespaces - ) - else: - raise ValueError("unknown method %r" % method) - -# -------------------------------------------------------------------- -# serialization support - -def _namespaces(elem, encoding, default_namespace=None): - # identify namespaces used in this tree - - # maps qnames to *encoded* prefix:local names - qnames = {None: None} - - # maps uri:s to prefixes - namespaces = {} - if default_namespace: - namespaces[default_namespace] = "" - - def encode(text): - return text.encode(encoding) - - def add_qname(qname): - # calculate serialized qname representation - try: - if qname[:1] == "{": - uri, tag = qname[1:].split("}", 1) - prefix = namespaces.get(uri) - if prefix is None: - prefix = _namespace_map.get(uri) - if prefix is None: - prefix = "ns%d" % len(namespaces) - if prefix != "xml": - namespaces[uri] = prefix - if prefix: - qnames[qname] = encode("%s:%s" % (prefix, tag)) - else: - qnames[qname] = encode(tag) # default element - else: - if default_namespace: - # FIXME: can this be handled in XML 1.0? - raise ValueError( - "cannot use non-qualified names with " - "default_namespace option" - ) - qnames[qname] = encode(qname) - except TypeError: - _raise_serialization_error(qname) - - # populate qname and namespaces table - try: - iterate = elem.iter - except AttributeError: - iterate = elem.getiterator # cET compatibility - for elem in iterate(): - tag = elem.tag - if isinstance(tag, QName) and tag.text not in qnames: - add_qname(tag.text) - elif isinstance(tag, string_types): - if tag not in qnames: - add_qname(tag) - elif tag is not None and tag is not Comment and tag is not PI: - _raise_serialization_error(tag) - for key, value in elem.items(): - if isinstance(key, QName): - key = key.text - if key not in qnames: - add_qname(key) - if isinstance(value, QName) and value.text not in qnames: - add_qname(value.text) - text = elem.text - if isinstance(text, QName) and text.text not in qnames: - add_qname(text.text) - return qnames, namespaces - -def _serialize_xml(write, elem, encoding, qnames, namespaces): - tag = elem.tag - text = elem.text - if tag is Comment: - write("<!--%s-->" % _escape_cdata(text, encoding)) - elif tag is ProcessingInstruction: - write("<?%s?>" % _escape_cdata(text, encoding)) - else: - tag = qnames[tag] - if tag is None: - if text: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_xml(write, e, encoding, qnames, None) - else: - write("<" + tag) - items = elem.items() - if items or namespaces: - items = sorted(items) # lexical order - for k, v in items: - if isinstance(k, QName): - k = k.text - if isinstance(v, QName): - v = qnames[v.text] - else: - v = _escape_attrib(v, encoding) - write(" %s=\"%s\"" % (qnames[k], v)) - if namespaces: - items = namespaces.items() - items = sorted(items, key=lambda x: x[1]) # sort on prefix - for v, k in items: - if k: - k = ":" + k - write(" xmlns%s=\"%s\"" % ( - k.encode(encoding), - _escape_attrib(v, encoding) - )) - if text or len(elem): - write(">") - if text: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_xml(write, e, encoding, qnames, None) - write("</" + tag + ">") - else: - write(" />") - if elem.tail: - write(_escape_cdata(elem.tail, encoding)) - -HTML_EMPTY = ("area", "base", "basefont", "br", "col", "frame", "hr", - "img", "input", "isindex", "link", "meta" "param") - -try: - HTML_EMPTY = set(HTML_EMPTY) -except NameError: - pass - -def _serialize_html(write, elem, encoding, qnames, namespaces): - tag = elem.tag - text = elem.text - if tag is Comment: - write("<!--%s-->" % _escape_cdata(text, encoding)) - elif tag is ProcessingInstruction: - write("<?%s?>" % _escape_cdata(text, encoding)) - else: - tag = qnames[tag] - if tag is None: - if text: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_html(write, e, encoding, qnames, None) - else: - write("<" + tag) - items = elem.items() - if items or namespaces: - items = sorted(items) # lexical order - for k, v in items: - if isinstance(k, QName): - k = k.text - if isinstance(v, QName): - v = qnames[v.text] - else: - v = _escape_attrib_html(v, encoding) - # FIXME: handle boolean attributes - write(" %s=\"%s\"" % (qnames[k], v)) - if namespaces: - items = namespaces.items() - items = sorted(items, key=lambda x: x[1]) # sort on prefix - for v, k in items: - if k: - k = ":" + k - write(" xmlns%s=\"%s\"" % ( - k.encode(encoding), - _escape_attrib(v, encoding) - )) - write(">") - tag = tag.lower() - if text: - if tag == "script" or tag == "style": - write(_encode(text, encoding)) - else: - write(_escape_cdata(text, encoding)) - for e in elem: - _serialize_html(write, e, encoding, qnames, None) - if tag not in HTML_EMPTY: - write("</" + tag + ">") - if elem.tail: - write(_escape_cdata(elem.tail, encoding)) - -def _serialize_text(write, elem, encoding): - for part in elem.itertext(): - write(part.encode(encoding)) - if elem.tail: - write(elem.tail.encode(encoding)) - -## -# Registers a namespace prefix. The registry is global, and any -# existing mapping for either the given prefix or the namespace URI -# will be removed. -# -# @param prefix Namespace prefix. -# @param uri Namespace uri. Tags and attributes in this namespace -# will be serialized with the given prefix, if at all possible. -# @raise ValueError If the prefix is reserved, or is otherwise -# invalid. - -def register_namespace(prefix, uri): - if re.match(r"ns\d+$", prefix): - raise ValueError("Prefix format reserved for internal use") - for k, v in _namespace_map.items(): - if k == uri or v == prefix: - del _namespace_map[k] - _namespace_map[uri] = prefix - -_namespace_map = { - # "well-known" namespace prefixes - "http://www.w3.org/XML/1998/namespace": "xml", - "http://www.w3.org/1999/xhtml": "html", - "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf", - "http://schemas.xmlsoap.org/wsdl/": "wsdl", - # xml schema - "http://www.w3.org/2001/XMLSchema": "xs", - "http://www.w3.org/2001/XMLSchema-instance": "xsi", - # dublic core - "http://purl.org/dc/elements/1.1/": "dc", -} - -def _raise_serialization_error(text): - raise TypeError( - "cannot serialize %r (type %s)" % (text, type(text).__name__) - ) - -def _encode(text, encoding): - try: - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_cdata(text, encoding): - # escape character data - try: - # it's worth avoiding do-nothing calls for strings that are - # shorter than 500 character, or so. assume that's, by far, - # the most common case in most applications. - if "&" in text: - text = text.replace("&", "&") - if "<" in text: - text = text.replace("<", "<") - if ">" in text: - text = text.replace(">", ">") - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_attrib(text, encoding): - # escape attribute value - try: - if "&" in text: - text = text.replace("&", "&") - if "<" in text: - text = text.replace("<", "<") - if ">" in text: - text = text.replace(">", ">") - if "\"" in text: - text = text.replace("\"", """) - if "\n" in text: - text = text.replace("\n", " ") - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_attrib_html(text, encoding): - # escape attribute value - try: - if "&" in text: - text = text.replace("&", "&") - if ">" in text: - text = text.replace(">", ">") - if "\"" in text: - text = text.replace("\"", """) - return text.encode(encoding, "xmlcharrefreplace") - except (TypeError, AttributeError): - _raise_serialization_error(text) - -# -------------------------------------------------------------------- - -## -# Generates a string representation of an XML element, including all -# subelements. -# -# @param element An Element instance. -# @return An encoded string containing the XML data. -# @defreturn string - -def tostring(element, encoding=None, method=None): - class dummy: - pass - data = [] - file = dummy() - file.write = data.append - ElementTree(element).write(file, encoding, method=method) - return "".join(data) - -## -# Generates a string representation of an XML element, including all -# subelements. The string is returned as a sequence of string fragments. -# -# @param element An Element instance. -# @return A sequence object containing the XML data. -# @defreturn sequence -# @since 1.3 - -def tostringlist(element, encoding=None): - class dummy: - pass - data = [] - file = dummy() - file.write = data.append - ElementTree(element).write(file, encoding) - # FIXME: merge small fragments into larger parts - return data - -## -# Writes an element tree or element structure to sys.stdout. This -# function should be used for debugging only. -# <p> -# The exact output format is implementation dependent. In this -# version, it's written as an ordinary XML file. -# -# @param elem An element tree or an individual element. - -def dump(elem): - # debugging - if not isinstance(elem, ElementTree): - elem = ElementTree(elem) - elem.write(sys.stdout) - tail = elem.getroot().tail - if not tail or tail[-1] != "\n": - sys.stdout.write("\n") - -# -------------------------------------------------------------------- -# parsing - -## -# Parses an XML document into an element tree. -# -# @param source A filename or file object containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return An ElementTree instance - -def parse(source, parser=None): - tree = ElementTree() - tree.parse(source, parser) - return tree - -## -# Parses an XML document into an element tree incrementally, and reports -# what's going on to the user. -# -# @param source A filename or file object containing XML data. -# @param events A list of events to report back. If omitted, only "end" -# events are reported. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return A (event, elem) iterator. - -def iterparse(source, events=None, parser=None): - if not hasattr(source, "read"): - source = open(source, "rb") - if not parser: - parser = XMLParser(target=TreeBuilder()) - return _IterParseIterator(source, events, parser) - -class _IterParseIterator(object): - - def __init__(self, source, events, parser): - self._file = source - self._events = [] - self._index = 0 - self.root = self._root = None - self._parser = parser - # wire up the parser for event reporting - parser = self._parser._parser - append = self._events.append - if events is None: - events = ["end"] - for event in events: - if event == "start": - try: - parser.ordered_attributes = 1 - parser.specified_attributes = 1 - def handler(tag, attrib_in, event=event, append=append, - start=self._parser._start_list): - append((event, start(tag, attrib_in))) - parser.StartElementHandler = handler - except AttributeError: - def handler(tag, attrib_in, event=event, append=append, - start=self._parser._start): - append((event, start(tag, attrib_in))) - parser.StartElementHandler = handler - elif event == "end": - def handler(tag, event=event, append=append, - end=self._parser._end): - append((event, end(tag))) - parser.EndElementHandler = handler - elif event == "start-ns": - def handler(prefix, uri, event=event, append=append): - try: - uri = uri.encode("ascii") - except UnicodeError: - pass - append((event, (prefix or "", uri))) - parser.StartNamespaceDeclHandler = handler - elif event == "end-ns": - def handler(prefix, event=event, append=append): - append((event, None)) - parser.EndNamespaceDeclHandler = handler - - def __next__(self): - while 1: - try: - item = self._events[self._index] - except IndexError: - if self._parser is None: - self.root = self._root - raise StopIteration - # load event buffer - del self._events[:] - self._index = 0 - data = self._file.read(16384) - if data: - self._parser.feed(data) - else: - self._root = self._parser.close() - self._parser = None - else: - self._index = self._index + 1 - return item - - next = __next__ # Python 2 compatibility - - def __iter__(self): - return self - -## -# Parses an XML document from a string constant. This function can -# be used to embed "XML literals" in Python code. -# -# @param source A string containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return An Element instance. -# @defreturn Element - -def XML(text, parser=None): - if not parser: - parser = XMLParser(target=TreeBuilder()) - parser.feed(text) - return parser.close() - -## -# Parses an XML document from a string constant, and also returns -# a dictionary which maps from element id:s to elements. -# -# @param source A string containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return A tuple containing an Element instance and a dictionary. -# @defreturn (Element, dictionary) - -def XMLID(text, parser=None): - if not parser: - parser = XMLParser(target=TreeBuilder()) - parser.feed(text) - tree = parser.close() - ids = {} - for elem in tree.getiterator(): - id = elem.get("id") - if id: - ids[id] = elem - return tree, ids - -## -# Parses an XML document from a string constant. Same as {@link #XML}. -# -# @def fromstring(text) -# @param source A string containing XML data. -# @return An Element instance. -# @defreturn Element - -fromstring = XML - -## -# Parses an XML document from a sequence of string fragments. -# -# @param sequence A list or other sequence containing XML data fragments. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLParser} parser is used. -# @return An Element instance. -# @defreturn Element -# @since 1.3 - -def fromstringlist(sequence, parser=None): - if not parser: - parser = XMLParser(target=TreeBuilder()) - for text in sequence: - parser.feed(text) - return parser.close() - -# -------------------------------------------------------------------- - -## -# Generic element structure builder. This builder converts a sequence -# of {@link #TreeBuilder.start}, {@link #TreeBuilder.data}, and {@link -# #TreeBuilder.end} method calls to a well-formed element structure. -# <p> -# You can use this class to build an element structure using a custom XML -# parser, or a parser for some other XML-like format. -# -# @param element_factory Optional element factory. This factory -# is called to create new Element instances, as necessary. - -class TreeBuilder(object): - - def __init__(self, element_factory=None): - self._data = [] # data collector - self._elem = [] # element stack - self._last = None # last element - self._tail = None # true if we're after an end tag - if element_factory is None: - element_factory = Element - self._factory = element_factory - - ## - # Flushes the builder buffers, and returns the toplevel document - # element. - # - # @return An Element instance. - # @defreturn Element - - def close(self): - assert len(self._elem) == 0, "missing end tags" - assert self._last != None, "missing toplevel element" - return self._last - - def _flush(self): - if self._data: - if self._last is not None: - text = "".join(self._data) - if self._tail: - assert self._last.tail is None, "internal error (tail)" - self._last.tail = text - else: - assert self._last.text is None, "internal error (text)" - self._last.text = text - self._data = [] - - ## - # Adds text to the current element. - # - # @param data A string. This should be either an 8-bit string - # containing ASCII text, or a Unicode string. - - def data(self, data): - self._data.append(data) - - ## - # Opens a new element. - # - # @param tag The element name. - # @param attrib A dictionary containing element attributes. - # @return The opened element. - # @defreturn Element - - def start(self, tag, attrs): - self._flush() - self._last = elem = self._factory(tag, attrs) - if self._elem: - self._elem[-1].append(elem) - self._elem.append(elem) - self._tail = 0 - return elem - - ## - # Closes the current element. - # - # @param tag The element name. - # @return The closed element. - # @defreturn Element - - def end(self, tag): - self._flush() - self._last = self._elem.pop() - assert self._last.tag == tag,\ - "end tag mismatch (expected %s, got %s)" % ( - self._last.tag, tag) - self._tail = 1 - return self._last - -## -# Element structure builder for XML source data, based on the -# <b>expat</b> parser. -# -# @keyparam target Target object. If omitted, the builder uses an -# instance of the standard {@link #TreeBuilder} class. -# @keyparam html Predefine HTML entities. This flag is not supported -# by the current implementation. -# @keyparam encoding Optional encoding. If given, the value overrides -# the encoding specified in the XML file. -# @see #ElementTree -# @see #TreeBuilder - -class XMLParser(object): - - def __init__(self, html=0, target=None, encoding=None): - try: - from xml.parsers import expat - except ImportError: - try: - import pyexpat; expat = pyexpat - except ImportError: - raise ImportError( - "No module named expat; use SimpleXMLTreeBuilder instead" - ) - parser = expat.ParserCreate(encoding, "}") - if target is None: - target = TreeBuilder() - # underscored names are provided for compatibility only - self.parser = self._parser = parser - self.target = self._target = target - self._error = expat.error - self._names = {} # name memo cache - # callbacks - parser.DefaultHandlerExpand = self._default - parser.StartElementHandler = self._start - parser.EndElementHandler = self._end - parser.CharacterDataHandler = self._data - # let expat do the buffering, if supported - try: - self._parser.buffer_text = 1 - except AttributeError: - pass - # use new-style attribute handling, if supported - try: - self._parser.ordered_attributes = 1 - self._parser.specified_attributes = 1 - parser.StartElementHandler = self._start_list - except AttributeError: - pass - self._doctype = None - self.entity = {} - try: - self.version = "Expat %d.%d.%d" % expat.version_info - except AttributeError: - pass # unknown - - def _raiseerror(self, value): - err = ParseError(value) - err.code = value.code - err.position = value.lineno, value.offset - raise err - - if sys.version_info >= (3, 0): - def _fixtext(self, text): - return text - else: - def _fixtext(self, text): - # convert text string to ascii, if possible - try: - return text.encode("ascii") - except UnicodeError: - return text - - def _fixname(self, key): - # expand qname, and convert name string to ascii, if possible - try: - name = self._names[key] - except KeyError: - name = key - if "}" in name: - name = "{" + name - self._names[key] = name = self._fixtext(name) - return name - - def _start(self, tag, attrib_in): - fixname = self._fixname - fixtext = self._fixtext - tag = fixname(tag) - attrib = {} - for key, value in attrib_in.items(): - attrib[fixname(key)] = fixtext(value) - return self.target.start(tag, attrib) - - def _start_list(self, tag, attrib_in): - fixname = self._fixname - fixtext = self._fixtext - tag = fixname(tag) - attrib = {} - if attrib_in: - for i in range(0, len(attrib_in), 2): - attrib[fixname(attrib_in[i])] = fixtext(attrib_in[i+1]) - return self.target.start(tag, attrib) - - def _data(self, text): - return self.target.data(self._fixtext(text)) - - def _end(self, tag): - return self.target.end(self._fixname(tag)) - - def _default(self, text): - prefix = text[:1] - if prefix == "&": - # deal with undefined entities - try: - self.target.data(self.entity[text[1:-1]]) - except KeyError: - from xml.parsers import expat - err = expat.error( - "undefined entity %s: line %d, column %d" % - (text, self._parser.ErrorLineNumber, - self._parser.ErrorColumnNumber) - ) - err.code = 11 # XML_ERROR_UNDEFINED_ENTITY - err.lineno = self._parser.ErrorLineNumber - err.offset = self._parser.ErrorColumnNumber - raise err - elif prefix == "<" and text[:9] == "<!DOCTYPE": - self._doctype = [] # inside a doctype declaration - elif self._doctype is not None: - # parse doctype contents - if prefix == ">": - self._doctype = None - return - text = text.strip() - if not text: - return - self._doctype.append(text) - n = len(self._doctype) - if n > 2: - type = self._doctype[1] - if type == "PUBLIC" and n == 4: - name, type, pubid, system = self._doctype - elif type == "SYSTEM" and n == 3: - name, type, system = self._doctype - pubid = None - else: - return - if pubid: - pubid = pubid[1:-1] - if hasattr(self.target, "doctype"): - self.target.doctype(name, pubid, system[1:-1]) - self._doctype = None - - ## - # Feeds data to the parser. - # - # @param data Encoded data. - - def feed(self, data): - try: - self._parser.Parse(data, 0) - except self._error as v: - self._raiseerror(v) - - ## - # Finishes feeding data to the parser. - # - # @return An element structure. - # @defreturn Element - - def close(self): - try: - self._parser.Parse("", 1) # end of data - except self._error as v: - self._raiseerror(v) - tree = self.target.close() - del self.target, self._parser # get rid of circular references - return tree - -# compatibility -XMLTreeBuilder = XMLParser diff --git a/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py b/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py index 6f493c3a3..00a5a7039 100644 --- a/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py +++ b/tests/roots/test-add_source_parser-conflicts-with-users-setting/conf.py @@ -9,7 +9,7 @@ sys.path.insert(0, os.path.abspath('.')) class DummyTestParser(Parser): - pass + supported = ('dummy',) extensions = ['source_parser'] diff --git a/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py b/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py index 0dff7e311..69898ed91 100644 --- a/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py +++ b/tests/roots/test-add_source_parser-conflicts-with-users-setting/source_parser.py @@ -4,8 +4,9 @@ from docutils.parsers import Parser class TestSourceParser(Parser): - pass + supported = ('test',) def setup(app): - app.add_source_parser('.test', TestSourceParser) + app.add_source_suffix('.test', 'test') + app.add_source_parser(TestSourceParser) diff --git a/tests/roots/test-add_source_parser/conf.py b/tests/roots/test-add_source_parser/conf.py index 805f80fcc..eface21e8 100644 --- a/tests/roots/test-add_source_parser/conf.py +++ b/tests/roots/test-add_source_parser/conf.py @@ -9,7 +9,7 @@ sys.path.insert(0, os.path.abspath('.')) class DummyMarkdownParser(Parser): - pass + supported = ('markdown',) extensions = ['source_parser'] diff --git a/tests/roots/test-add_source_parser/source_parser.py b/tests/roots/test-add_source_parser/source_parser.py index 0dff7e311..69898ed91 100644 --- a/tests/roots/test-add_source_parser/source_parser.py +++ b/tests/roots/test-add_source_parser/source_parser.py @@ -4,8 +4,9 @@ from docutils.parsers import Parser class TestSourceParser(Parser): - pass + supported = ('test',) def setup(app): - app.add_source_parser('.test', TestSourceParser) + app.add_source_suffix('.test', 'test') + app.add_source_parser(TestSourceParser) diff --git a/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.mo b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.mo Binary files differnew file mode 100644 index 000000000..6aa00f77a --- /dev/null +++ b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.mo diff --git a/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.po b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.po new file mode 100644 index 000000000..ee1f6c23f --- /dev/null +++ b/tests/roots/test-locale/locale1/en/LC_MESSAGES/myext.po @@ -0,0 +1,2 @@ +msgid "Hello world" +msgstr "HELLO WORLD" diff --git a/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.mo b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.mo Binary files differnew file mode 100644 index 000000000..14c34d0b7 --- /dev/null +++ b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.mo diff --git a/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.po b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.po new file mode 100644 index 000000000..d376cf9bd --- /dev/null +++ b/tests/roots/test-locale/locale2/en/LC_MESSAGES/myext.po @@ -0,0 +1,2 @@ +msgid "Hello sphinx" +msgstr "HELLO SPHINX" diff --git a/tests/roots/test-prolog/prolog_markdown_parser.py b/tests/roots/test-prolog/prolog_markdown_parser.py index f28c37b4e..56ce3cf5a 100644 --- a/tests/roots/test-prolog/prolog_markdown_parser.py +++ b/tests/roots/test-prolog/prolog_markdown_parser.py @@ -4,9 +4,12 @@ from docutils.parsers import Parser class DummyMarkdownParser(Parser): + supported = ('markdown',) + def parse(self, inputstring, document): document.rawsource = inputstring def setup(app): - app.add_source_parser('.md', DummyMarkdownParser) + app.add_source_suffix('.md', 'markdown') + app.add_source_parser(DummyMarkdownParser) diff --git a/tests/roots/test-root/conf.py b/tests/roots/test-root/conf.py index e2d8928fe..38b17f57a 100644 --- a/tests/roots/test-root/conf.py +++ b/tests/roots/test-root/conf.py @@ -20,7 +20,6 @@ templates_path = ['_templates'] master_doc = 'contents' source_suffix = ['.txt', '.add', '.foo'] -source_parsers = {'.foo': 'parsermod.Parser'} project = 'Sphinx <Tests>' copyright = '2010-2016, Georg Brandl & Team' @@ -95,11 +94,6 @@ def userdesc_parse(env, sig, signode): return x -def functional_directive(name, arguments, options, content, lineno, - content_offset, block_text, state, state_machine): - return [nodes.strong(text='from function: %s' % options['opt'])] - - class ClassDirective(Directive): option_spec = {'opt': lambda x: x} @@ -108,9 +102,12 @@ class ClassDirective(Directive): def setup(app): + import parsermod + app.add_config_value('value_from_conf_py', 42, False) - app.add_directive('funcdir', functional_directive, opt=lambda x: x) app.add_directive('clsdir', ClassDirective) app.add_object_type('userdesc', 'userdescrole', '%s (userdesc)', userdesc_parse, objname='user desc') app.add_javascript('file://moo.js') + app.add_source_suffix('.foo', 'foo') + app.add_source_parser(parsermod.Parser) diff --git a/tests/roots/test-root/extapi.txt b/tests/roots/test-root/extapi.txt index 4728e3de1..56be6d8ce 100644 --- a/tests/roots/test-root/extapi.txt +++ b/tests/roots/test-root/extapi.txt @@ -3,8 +3,5 @@ Extension API tests Testing directives: -.. funcdir:: - :opt: Foo - .. clsdir:: :opt: Bar diff --git a/tests/roots/test-root/parsermod.py b/tests/roots/test-root/parsermod.py index dfc699944..de0849c80 100644 --- a/tests/roots/test-root/parsermod.py +++ b/tests/roots/test-root/parsermod.py @@ -3,6 +3,8 @@ from docutils.parsers import Parser class Parser(Parser): + supported = ('foo',) + def parse(self, input, document): section = nodes.section(ids=['id1']) section += nodes.title('Generated section', 'Generated section') diff --git a/tests/test_application.py b/tests/test_application.py index 075acc80c..babf95497 100644 --- a/tests/test_application.py +++ b/tests/test_application.py @@ -83,9 +83,20 @@ def test_domain_override(app, status, warning): @pytest.mark.sphinx(testroot='add_source_parser') def test_add_source_parser(app, status, warning): assert set(app.config.source_suffix) == set(['.rst', '.md', '.test']) - assert set(app.registry.get_source_parsers().keys()) == set(['*', '.md', '.test']) + + # .rst; only in :confval:`source_suffix` + assert '.rst' not in app.registry.get_source_parsers() + assert app.registry.source_suffix['.rst'] is None + + # .md; configured by :confval:`source_suffix` and :confval:`source_parsers` + assert '.md' in app.registry.get_source_parsers() + assert app.registry.source_suffix['.md'] == '.md' assert app.registry.get_source_parsers()['.md'].__name__ == 'DummyMarkdownParser' - assert app.registry.get_source_parsers()['.test'].__name__ == 'TestSourceParser' + + # .test; configured by API + assert app.registry.source_suffix['.test'] == 'test' + assert 'test' in app.registry.get_source_parsers() + assert app.registry.get_source_parsers()['test'].__name__ == 'TestSourceParser' @pytest.mark.sphinx(testroot='extensions') diff --git a/tests/test_build_html.py b/tests/test_build_html.py index d167617c7..7b4a61787 100644 --- a/tests/test_build_html.py +++ b/tests/test_build_html.py @@ -186,7 +186,6 @@ def test_html_warnings(app, warning): (".//dd/p", r'Return spam\.'), ], 'extapi.html': [ - (".//strong", 'from function: Foo'), (".//strong", 'from class: Bar'), ], 'markup.html': [ diff --git a/tests/test_build_html5.py b/tests/test_build_html5.py index 6f0338113..265c42cbd 100644 --- a/tests/test_build_html5.py +++ b/tests/test_build_html5.py @@ -94,7 +94,6 @@ def cached_etree_parse(): (".//dd/p", r'Return spam\.'), ], 'extapi.html': [ - (".//strong", 'from function: Foo'), (".//strong", 'from class: Bar'), ], 'markup.html': [ diff --git a/tests/test_builder.py b/tests/test_builder.py new file mode 100644 index 000000000..d58091e8d --- /dev/null +++ b/tests/test_builder.py @@ -0,0 +1,56 @@ +# -*- coding: utf-8 -*- +""" + test_builder + ~~~~~~~~ + + Test the Builder class. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" +import pytest + + +@pytest.mark.sphinx('dummy', srcdir="test_builder") +def test_incremental_reading(app): + # first reading + updated = app.builder.read() + assert set(updated) == app.env.found_docs == set(app.env.all_docs) + + # test if exclude_patterns works ok + assert 'subdir/excluded' not in app.env.found_docs + + # before second reading, add, modify and remove source files + (app.srcdir / 'new.txt').write_text('New file\n========\n') + app.env.all_docs['contents'] = 0 # mark as modified + (app.srcdir / 'autodoc.txt').unlink() + + # second reading + updated = app.builder.read() + + # "includes" and "images" are in there because they contain references + # to nonexisting downloadable or image files, which are given another + # chance to exist + assert set(updated) == set(['contents', 'new', 'includes', 'images']) + assert 'autodoc' not in app.env.all_docs + assert 'autodoc' not in app.env.found_docs + + +@pytest.mark.sphinx('dummy') +def test_env_read_docs(app): + """By default, docnames are read in alphanumeric order""" + def on_env_read_docs_1(app, env, docnames): + pass + + app.connect('env-before-read-docs', on_env_read_docs_1) + + read_docnames = app.builder.read() + assert len(read_docnames) > 2 and read_docnames == sorted(read_docnames) + + def on_env_read_docs_2(app, env, docnames): + docnames.remove('images') + + app.connect('env-before-read-docs', on_env_read_docs_2) + + read_docnames = app.builder.read() + assert len(read_docnames) == 2 diff --git a/tests/test_domain_std.py b/tests/test_domain_std.py index dce7a5ddf..57d0bf185 100644 --- a/tests/test_domain_std.py +++ b/tests/test_domain_std.py @@ -17,6 +17,7 @@ from sphinx.domains.std import StandardDomain def test_process_doc_handle_figure_caption(): env = mock.Mock(domaindata={}) + env.app.registry.enumerable_nodes = {} figure_node = nodes.figure( '', nodes.caption('caption text', 'caption text'), @@ -40,6 +41,7 @@ def test_process_doc_handle_figure_caption(): def test_process_doc_handle_table_title(): env = mock.Mock(domaindata={}) + env.app.registry.enumerable_nodes = {} table_node = nodes.table( '', nodes.title('title text', 'title text'), @@ -63,6 +65,7 @@ def test_process_doc_handle_table_title(): def test_get_full_qualified_name(): env = mock.Mock(domaindata={}) + env.app.registry.enumerable_nodes = {} domain = StandardDomain(env) # normal references diff --git a/tests/test_environment.py b/tests/test_environment.py index 11364e89f..1ab60b539 100644 --- a/tests/test_environment.py +++ b/tests/test_environment.py @@ -12,38 +12,15 @@ import pytest from sphinx.builders.html import StandaloneHTMLBuilder from sphinx.builders.latex import LaTeXBuilder -from sphinx.testing.util import SphinxTestApp, path -app = env = None - -@pytest.fixture(scope='module', autouse=True) -def setup_module(rootdir, sphinx_test_tempdir): - global app, env - srcdir = sphinx_test_tempdir / 'root-envtest' - if not srcdir.exists(): - (rootdir / 'test-root').copytree(srcdir) - app = SphinxTestApp(srcdir=srcdir) - env = app.env - yield - app.cleanup() - - -# Tests are run in the order they appear in the file, therefore we can -# afford to not run update() in the setup but in its own test - -def test_first_update(): - updated = env.update(app.config, app.srcdir, app.doctreedir) - assert set(updated) == env.found_docs == set(env.all_docs) - # test if exclude_patterns works ok - assert 'subdir/excluded' not in env.found_docs - - -def test_images(): +@pytest.mark.sphinx('dummy') +def test_images(app): + app.build() assert ('image file not readable: foo.png' in app._warning.getvalue()) - tree = env.get_doctree('images') + tree = app.env.get_doctree('images') htmlbuilder = StandaloneHTMLBuilder(app) htmlbuilder.set_environment(app.env) htmlbuilder.init() @@ -67,44 +44,10 @@ def test_images(): 'svgimg.pdf', 'img.foo.png']) -def test_second_update(): - # delete, add and "edit" (change saved mtime) some files and update again - env.all_docs['contents'] = 0 - root = path(app.srcdir) - # important: using "autodoc" because it is the last one to be included in - # the contents.txt toctree; otherwise section numbers would shift - (root / 'autodoc.txt').unlink() - (root / 'new.txt').write_text('New file\n========\n') - updated = env.update(app.config, app.srcdir, app.doctreedir) - # "includes" and "images" are in there because they contain references - # to nonexisting downloadable or image files, which are given another - # chance to exist - assert set(updated) == set(['contents', 'new', 'includes', 'images']) - assert 'autodoc' not in env.all_docs - assert 'autodoc' not in env.found_docs - - -def test_env_read_docs(): - """By default, docnames are read in alphanumeric order""" - def on_env_read_docs_1(app, env, docnames): - pass - - app.connect('env-before-read-docs', on_env_read_docs_1) - - read_docnames = env.update(app.config, app.srcdir, app.doctreedir) - assert len(read_docnames) > 2 and read_docnames == sorted(read_docnames) - - def on_env_read_docs_2(app, env, docnames): - docnames.remove('images') - - app.connect('env-before-read-docs', on_env_read_docs_2) - - read_docnames = env.update(app.config, app.srcdir, app.doctreedir) - assert len(read_docnames) == 2 - - -def test_object_inventory(): - refs = env.domaindata['py']['objects'] +@pytest.mark.sphinx('dummy') +def test_object_inventory(app): + app.build() + refs = app.env.domaindata['py']['objects'] assert 'func_without_module' in refs assert refs['func_without_module'] == ('objects', 'function') @@ -121,8 +64,8 @@ def test_object_inventory(): assert 'func_in_module' not in refs assert 'func_noindex' not in refs - assert env.domaindata['py']['modules']['mod'] == \ + assert app.env.domaindata['py']['modules']['mod'] == \ ('objects', 'Module synopsis.', 'UNIX', False) - assert env.domains['py'].data is env.domaindata['py'] - assert env.domains['c'].data is env.domaindata['c'] + assert app.env.domains['py'].data is app.env.domaindata['py'] + assert app.env.domains['c'].data is app.env.domaindata['c'] diff --git a/tests/test_ext_napoleon_docstring.py b/tests/test_ext_napoleon_docstring.py index 5d0cbcf39..a4d127d0d 100644 --- a/tests/test_ext_napoleon_docstring.py +++ b/tests/test_ext_napoleon_docstring.py @@ -57,15 +57,21 @@ Sample namedtuple subclass .. attribute:: attr1 - *Arbitrary type* -- Quick description of attr1 + Quick description of attr1 + + :type: Arbitrary type .. attribute:: attr2 - *Another arbitrary type* -- Quick description of attr2 + Quick description of attr2 + + :type: Another arbitrary type .. attribute:: attr3 - *Type* -- Adds a newline after the type + Adds a newline after the type + + :type: Type """ self.assertEqual(expected, actual) @@ -265,6 +271,44 @@ class GoogleDocstringTest(BaseDocstringTest): """ )] + def test_sphinx_admonitions(self): + admonition_map = { + 'Attention': 'attention', + 'Caution': 'caution', + 'Danger': 'danger', + 'Error': 'error', + 'Hint': 'hint', + 'Important': 'important', + 'Note': 'note', + 'Tip': 'tip', + 'Todo': 'todo', + 'Warning': 'warning', + 'Warnings': 'warning', + } + config = Config() + for section, admonition in admonition_map.items(): + # Multiline + actual = str(GoogleDocstring(("{}:\n" + " this is the first line\n" + "\n" + " and this is the second line\n" + ).format(section), config)) + expect = (".. {}::\n" + "\n" + " this is the first line\n" + " \n" + " and this is the second line\n" + ).format(admonition) + self.assertEqual(expect, actual) + + # Single line + actual = str(GoogleDocstring(("{}:\n" + " this is a single line\n" + ).format(section), config)) + expect = (".. {}:: this is a single line\n" + ).format(admonition) + self.assertEqual(expect, actual) + def test_docstrings(self): config = Config( napoleon_use_param=False, @@ -322,7 +366,9 @@ Attributes: expected = """\ .. attribute:: in_attr - :class:`numpy.ndarray` -- super-dooper attribute + super-dooper attribute + + :type: :class:`numpy.ndarray` """ self.assertEqual(expected, actual) @@ -335,7 +381,9 @@ Attributes: expected = """\ .. attribute:: in_attr - *numpy.ndarray* -- super-dooper attribute + super-dooper attribute + + :type: numpy.ndarray """ self.assertEqual(expected, actual) @@ -916,6 +964,28 @@ Parameters: actual = str(GoogleDocstring(docstring, config)) self.assertEqual(expected, actual) + def test_custom_generic_sections(self): + + docstrings = (("""\ +Really Important Details: + You should listen to me! +""", """.. rubric:: Really Important Details + +You should listen to me! +"""), + ("""\ +Sooper Warning: + Stop hitting yourself! +""", """:Warns: **Stop hitting yourself!** +""")) + + testConfig = Config(napoleon_custom_sections=['Really Important Details', + ('Sooper Warning', 'warns')]) + + for docstring, expected in docstrings: + actual = str(GoogleDocstring(docstring, testConfig)) + self.assertEqual(expected, actual) + class NumpyDocstringTest(BaseDocstringTest): docstrings = [( @@ -1070,6 +1140,46 @@ class NumpyDocstringTest(BaseDocstringTest): """ )] + def test_sphinx_admonitions(self): + admonition_map = { + 'Attention': 'attention', + 'Caution': 'caution', + 'Danger': 'danger', + 'Error': 'error', + 'Hint': 'hint', + 'Important': 'important', + 'Note': 'note', + 'Tip': 'tip', + 'Todo': 'todo', + 'Warning': 'warning', + 'Warnings': 'warning', + } + config = Config() + for section, admonition in admonition_map.items(): + # Multiline + actual = str(NumpyDocstring(("{}\n" + "{}\n" + " this is the first line\n" + "\n" + " and this is the second line\n" + ).format(section, '-' * len(section)), config)) + expect = (".. {}::\n" + "\n" + " this is the first line\n" + " \n" + " and this is the second line\n" + ).format(admonition) + self.assertEqual(expect, actual) + + # Single line + actual = str(NumpyDocstring(("{}\n" + "{}\n" + " this is a single line\n" + ).format(section, '-' * len(section)), config)) + expect = (".. {}:: this is a single line\n" + ).format(admonition) + self.assertEqual(expect, actual) + def test_docstrings(self): config = Config( napoleon_use_param=False, diff --git a/tests/test_intl.py b/tests/test_intl.py index c8c5a69e9..b94fe85df 100644 --- a/tests/test_intl.py +++ b/tests/test_intl.py @@ -525,7 +525,8 @@ def test_gettext_buildr_ignores_only_directive(app): def test_gettext_dont_rebuild_mo(make_app, app_params, build_mo): # --- don't rebuild by .mo mtime def get_number_of_update_targets(app_): - updated = app_.env.update(app_.config, app_.srcdir, app_.doctreedir) + app_.env.find_files(app_.config, app_.builder) + _, updated, _ = app_.env.get_outdated_files(config_changed=False) return len(updated) args, kwargs = app_params @@ -706,12 +707,14 @@ def test_html_rebuild_mo(app): app.build() # --- rebuild by .mo mtime app.builder.build_update() - updated = app.env.update(app.config, app.srcdir, app.doctreedir) + app.env.find_files(app.config, app.builder) + _, updated, _ = app.env.get_outdated_files(config_changed=False) assert len(updated) == 0 mtime = (app.srcdir / 'xx' / 'LC_MESSAGES' / 'bom.mo').stat().st_mtime (app.srcdir / 'xx' / 'LC_MESSAGES' / 'bom.mo').utime((mtime + 5, mtime + 5)) - updated = app.env.update(app.config, app.srcdir, app.doctreedir) + app.env.find_files(app.config, app.builder) + _, updated, _ = app.env.get_outdated_files(config_changed=False) assert len(updated) == 1 diff --git a/tests/test_locale.py b/tests/test_locale.py new file mode 100644 index 000000000..9b1921bd6 --- /dev/null +++ b/tests/test_locale.py @@ -0,0 +1,66 @@ +# -*- coding: utf-8 -*- +""" + test_locale + ~~~~~~~~~~ + + Test locale. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import pytest + +from sphinx import locale + + +@pytest.fixture(autouse=True) +def cleanup_translations(): + yield + locale.translators.clear() + + +def test_init(rootdir): + # not initialized yet + _ = locale.get_translation('myext') + assert _('Hello world') == 'Hello world' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' + + # load locale1 + locale.init([rootdir / 'test-locale' / 'locale1'], 'en', 'myext') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' + + # load a catalog to unrelated namespace + locale.init([rootdir / 'test-locale' / 'locale2'], 'en', 'myext', 'mynamespace') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'Hello sphinx' # nothing changed here + assert _('Hello reST') == 'Hello reST' + + # load locale2 in addition + locale.init([rootdir / 'test-locale' / 'locale2'], 'en', 'myext') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'HELLO SPHINX' + assert _('Hello reST') == 'Hello reST' + + +def test_init_with_unknown_language(rootdir): + locale.init([rootdir / 'test-locale' / 'locale1'], 'unknown', 'myext') + _ = locale.get_translation('myext') + assert _('Hello world') == 'Hello world' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' + + +def test_add_message_catalog(app, rootdir): + app.config.language = 'en' + app.add_message_catalog('myext', rootdir / 'test-locale' / 'locale1') + _ = locale.get_translation('myext') + assert _('Hello world') == 'HELLO WORLD' + assert _('Hello sphinx') == 'Hello sphinx' + assert _('Hello reST') == 'Hello reST' diff --git a/tests/test_util_docutils.py b/tests/test_util_docutils.py new file mode 100644 index 000000000..31a1d9bd2 --- /dev/null +++ b/tests/test_util_docutils.py @@ -0,0 +1,34 @@ +# -*- coding: utf-8 -*- +""" + test_util_docutils + ~~~~~~~~~~~~~~~~~~ + + Tests util.utils functions. + + :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from docutils import nodes + +from sphinx.util.docutils import docutils_namespace, register_node + + +def test_register_node(): + class custom_node(nodes.Element): + pass + + with docutils_namespace(): + register_node(custom_node) + + # check registered + assert hasattr(nodes.GenericNodeVisitor, 'visit_custom_node') + assert hasattr(nodes.GenericNodeVisitor, 'depart_custom_node') + assert hasattr(nodes.SparseNodeVisitor, 'visit_custom_node') + assert hasattr(nodes.SparseNodeVisitor, 'depart_custom_node') + + # check unregistered outside namespace + assert not hasattr(nodes.GenericNodeVisitor, 'visit_custom_node') + assert not hasattr(nodes.GenericNodeVisitor, 'depart_custom_node') + assert not hasattr(nodes.SparseNodeVisitor, 'visit_custom_node') + assert not hasattr(nodes.SparseNodeVisitor, 'depart_custom_node') diff --git a/utils/release-checklist b/utils/release-checklist index 7d9263dec..6b2ad74c8 100644 --- a/utils/release-checklist +++ b/utils/release-checklist @@ -51,6 +51,7 @@ for first beta releases * Check diff by ``git diff`` * ``git commit -am 'Bump version'`` * ``git push origin master`` +* open https://github.com/sphinx-doc/sphinx/settings/branches and make ``X.Y`` branch protected * Update `sphinx-doc-translations <https://github.com/sphinx-doc/sphinx-doc-translations>`_ * Add new version/milestone to tracker categories * Write announcement and send to sphinx-dev, sphinx-users and python-announce @@ -103,6 +104,7 @@ for major releases * ``git checkout master`` * ``git merge X.Y`` * ``git push origin master`` +* open https://github.com/sphinx-doc/sphinx/settings/branches and make ``A.B`` branch *not* protected * ``git checkout A.B`` (checkout old stable) * Run ``git tag A.B`` to paste a tag instead branch * Run ``git push origin :A.B --tags`` to remove old stable branch |
