diff options
Diffstat (limited to 'documentation3.html')
| -rw-r--r-- | documentation3.html | 1102 |
1 files changed, 0 insertions, 1102 deletions
diff --git a/documentation3.html b/documentation3.html deleted file mode 100644 index c5e2fac..0000000 --- a/documentation3.html +++ /dev/null @@ -1,1102 +0,0 @@ -<?xml version="1.0" encoding="utf-8" ?> -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> -<head> -<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<meta name="generator" content="Docutils 0.8.1: http://docutils.sourceforge.net/" /> -<title>The decorator module</title> -<meta name="author" content="Michele Simionato" /> -<style type="text/css"> - -.highlight { background: #f8f8f8; } -.highlight .c { color: #408080; font-style: italic } /* Comment */ -.highlight .err { border: 1px solid #FF0000 } /* Error */ -.highlight .k { color: #008000; font-weight: bold } /* Keyword */ -.highlight .o { color: #666666 } /* Operator */ -.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */ -.highlight .cp { color: #BC7A00 } /* Comment.Preproc */ -.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */ -.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */ -.highlight .gd { color: #A00000 } /* Generic.Deleted */ -.highlight .ge { font-style: italic } /* Generic.Emph */ -.highlight .gr { color: #FF0000 } /* Generic.Error */ -.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */ -.highlight .gi { color: #00A000 } /* Generic.Inserted */ -.highlight .go { color: #808080 } /* Generic.Output */ -.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */ -.highlight .gs { font-weight: bold } /* Generic.Strong */ -.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */ -.highlight .gt { color: #0040D0 } /* Generic.Traceback */ -.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */ -.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */ -.highlight .kp { color: #008000 } /* Keyword.Pseudo */ -.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */ -.highlight .kt { color: #008000; font-weight: bold } /* Keyword.Type */ -.highlight .m { color: #666666 } /* Literal.Number */ -.highlight .s { color: #BA2121 } /* Literal.String */ -.highlight .na { color: #7D9029 } /* Name.Attribute */ -.highlight .nb { color: #008000 } /* Name.Builtin */ -.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */ -.highlight .no { color: #880000 } /* Name.Constant */ -.highlight .nd { color: #AA22FF } /* Name.Decorator */ -.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */ -.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */ -.highlight .nf { color: #0000FF } /* Name.Function */ -.highlight .nl { color: #A0A000 } /* Name.Label */ -.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */ -.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */ -.highlight .nv { color: #19177C } /* Name.Variable */ -.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */ -.highlight .w { color: #bbbbbb } /* Text.Whitespace */ -.highlight .mf { color: #666666 } /* Literal.Number.Float */ -.highlight .mh { color: #666666 } /* Literal.Number.Hex */ -.highlight .mi { color: #666666 } /* Literal.Number.Integer */ -.highlight .mo { color: #666666 } /* Literal.Number.Oct */ -.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */ -.highlight .sc { color: #BA2121 } /* Literal.String.Char */ -.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */ -.highlight .s2 { color: #BA2121 } /* Literal.String.Double */ -.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */ -.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */ -.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */ -.highlight .sx { color: #008000 } /* Literal.String.Other */ -.highlight .sr { color: #BB6688 } /* Literal.String.Regex */ -.highlight .s1 { color: #BA2121 } /* Literal.String.Single */ -.highlight .ss { color: #19177C } /* Literal.String.Symbol */ -.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */ -.highlight .vc { color: #19177C } /* Name.Variable.Class */ -.highlight .vg { color: #19177C } /* Name.Variable.Global */ -.highlight .vi { color: #19177C } /* Name.Variable.Instance */ -.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */ - -</style> -</head> -<body> -<div class="document" id="the-decorator-module"> -<h1 class="title">The <tt class="docutils literal">decorator</tt> module</h1> -<table class="docinfo" frame="void" rules="none"> -<col class="docinfo-name" /> -<col class="docinfo-content" /> -<tbody valign="top"> -<tr><th class="docinfo-name">Author:</th> -<td>Michele Simionato</td></tr> -<tr class="field"><th class="docinfo-name">E-mail:</th><td class="field-body"><a class="reference external" href="mailto:michele.simionato@gmail.com">michele.simionato@gmail.com</a></td> -</tr> -<tr><th class="docinfo-name">Version:</th> -<td>3.4.0 (2012-10-18)</td></tr> -<tr class="field"><th class="docinfo-name">Requires:</th><td class="field-body">Python 2.4+</td> -</tr> -<tr class="field"><th class="docinfo-name">Download page:</th><td class="field-body"><a class="reference external" href="http://pypi.python.org/pypi/decorator/3.4.0">http://pypi.python.org/pypi/decorator/3.4.0</a></td> -</tr> -<tr class="field"><th class="docinfo-name">Installation:</th><td class="field-body"><tt class="docutils literal">easy_install decorator</tt></td> -</tr> -<tr class="field"><th class="docinfo-name">License:</th><td class="field-body">BSD license</td> -</tr> -</tbody> -</table> -<div class="contents topic" id="contents"> -<p class="topic-title first">Contents</p> -<ul class="simple"> -<li><a class="reference internal" href="#introduction" id="id4">Introduction</a></li> -<li><a class="reference internal" href="#definitions" id="id5">Definitions</a></li> -<li><a class="reference internal" href="#statement-of-the-problem" id="id6">Statement of the problem</a></li> -<li><a class="reference internal" href="#the-solution" id="id7">The solution</a></li> -<li><a class="reference internal" href="#a-trace-decorator" id="id8">A <tt class="docutils literal">trace</tt> decorator</a></li> -<li><a class="reference internal" href="#function-annotations" id="id9">Function annotations</a></li> -<li><a class="reference internal" href="#decorator-is-a-decorator" id="id10"><tt class="docutils literal">decorator</tt> is a decorator</a></li> -<li><a class="reference internal" href="#blocking" id="id11"><tt class="docutils literal">blocking</tt></a></li> -<li><a class="reference internal" href="#async" id="id12"><tt class="docutils literal">async</tt></a></li> -<li><a class="reference internal" href="#contextmanager" id="id13">contextmanager</a></li> -<li><a class="reference internal" href="#the-functionmaker-class" id="id14">The <tt class="docutils literal">FunctionMaker</tt> class</a></li> -<li><a class="reference internal" href="#getting-the-source-code" id="id15">Getting the source code</a></li> -<li><a class="reference internal" href="#dealing-with-third-party-decorators" id="id16">Dealing with third party decorators</a></li> -<li><a class="reference internal" href="#caveats-and-limitations" id="id17">Caveats and limitations</a></li> -<li><a class="reference internal" href="#compatibility-notes" id="id18">Compatibility notes</a></li> -<li><a class="reference internal" href="#licence" id="id19">LICENCE</a></li> -</ul> -</div> -<div class="section" id="introduction"> -<h1><a class="toc-backref" href="#id4">Introduction</a></h1> -<p>Python decorators are an interesting example of why syntactic sugar -matters. In principle, their introduction in Python 2.4 changed -nothing, since they do not provide any new functionality which was not -already present in the language. In practice, their introduction has -significantly changed the way we structure our programs in Python. I -believe the change is for the best, and that decorators are a great -idea since:</p> -<ul class="simple"> -<li>decorators help reducing boilerplate code;</li> -<li>decorators help separation of concerns;</li> -<li>decorators enhance readability and maintenability;</li> -<li>decorators are explicit.</li> -</ul> -<p>Still, as of now, writing custom decorators correctly requires -some experience and it is not as easy as it could be. For instance, -typical implementations of decorators involve nested functions, and -we all know that flat is better than nested.</p> -<p>The aim of the <tt class="docutils literal">decorator</tt> module it to simplify the usage of -decorators for the average programmer, and to popularize decorators by -showing various non-trivial examples. Of course, as all techniques, -decorators can be abused (I have seen that) and you should not try to -solve every problem with a decorator, just because you can.</p> -<p>You may find the source code for all the examples -discussed here in the <tt class="docutils literal">documentation.py</tt> file, which contains -this documentation in the form of doctests.</p> -</div> -<div class="section" id="definitions"> -<h1><a class="toc-backref" href="#id5">Definitions</a></h1> -<p>Technically speaking, any Python object which can be called with one argument -can be used as a decorator. However, this definition is somewhat too large -to be really useful. It is more convenient to split the generic class of -decorators in two subclasses:</p> -<ul class="simple"> -<li><em>signature-preserving</em> decorators, i.e. callable objects taking a -function as input and returning a function <em>with the same -signature</em> as output;</li> -<li><em>signature-changing</em> decorators, i.e. decorators that change -the signature of their input function, or decorators returning -non-callable objects.</li> -</ul> -<p>Signature-changing decorators have their use: for instance the -builtin classes <tt class="docutils literal">staticmethod</tt> and <tt class="docutils literal">classmethod</tt> are in this -group, since they take functions and return descriptor objects which -are not functions, nor callables.</p> -<p>However, signature-preserving decorators are more common and easier to -reason about; in particular signature-preserving decorators can be -composed together whereas other decorators in general cannot.</p> -<p>Writing signature-preserving decorators from scratch is not that -obvious, especially if one wants to define proper decorators that -can accept functions with any signature. A simple example will clarify -the issue.</p> -</div> -<div class="section" id="statement-of-the-problem"> -<h1><a class="toc-backref" href="#id6">Statement of the problem</a></h1> -<p>A very common use case for decorators is the memoization of functions. -A <tt class="docutils literal">memoize</tt> decorator works by caching -the result of the function call in a dictionary, so that the next time -the function is called with the same input parameters the result is retrieved -from the cache and not recomputed. There are many implementations of -<tt class="docutils literal">memoize</tt> in <a class="reference external" href="http://www.python.org/moin/PythonDecoratorLibrary">http://www.python.org/moin/PythonDecoratorLibrary</a>, -but they do not preserve the signature. -A simple implementation could be the following (notice -that in general it is impossible to memoize correctly something -that depends on non-hashable arguments):</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">memoize_uw</span><span class="p">(</span><span class="n">func</span><span class="p">):</span> - <span class="n">func</span><span class="o">.</span><span class="n">cache</span> <span class="o">=</span> <span class="p">{}</span> - <span class="k">def</span> <span class="nf">memoize</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="k">if</span> <span class="n">kw</span><span class="p">:</span> <span class="c"># frozenset is used to ensure hashability</span> - <span class="n">key</span> <span class="o">=</span> <span class="n">args</span><span class="p">,</span> <span class="nb">frozenset</span><span class="p">(</span><span class="n">kw</span><span class="o">.</span><span class="n">iteritems</span><span class="p">())</span> - <span class="k">else</span><span class="p">:</span> - <span class="n">key</span> <span class="o">=</span> <span class="n">args</span> - <span class="n">cache</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="n">cache</span> - <span class="k">if</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">cache</span><span class="p">:</span> - <span class="k">return</span> <span class="n">cache</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> - <span class="k">else</span><span class="p">:</span> - <span class="n">cache</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">result</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> - <span class="k">return</span> <span class="n">result</span> - <span class="k">return</span> <span class="n">functools</span><span class="o">.</span><span class="n">update_wrapper</span><span class="p">(</span><span class="n">memoize</span><span class="p">,</span> <span class="n">func</span><span class="p">)</span> -</pre></div> - -</div> -<p>Here we used the <a class="reference external" href="http://www.python.org/doc/2.5.2/lib/module-functools.html">functools.update_wrapper</a> utility, which has -been added in Python 2.5 expressly to simplify the definition of decorators -(in older versions of Python you need to copy the function attributes -<tt class="docutils literal">__name__</tt>, <tt class="docutils literal">__doc__</tt>, <tt class="docutils literal">__module__</tt> and <tt class="docutils literal">__dict__</tt> -from the original function to the decorated function by hand).</p> -<p>The implementation above works in the sense that the decorator -can accept functions with generic signatures; unfortunately this -implementation does <em>not</em> define a signature-preserving decorator, since in -general <tt class="docutils literal">memoize_uw</tt> returns a function with a -<em>different signature</em> from the original function.</p> -<p>Consider for instance the following case:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@memoize_uw</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">f1</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> -<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="c"># simulate some long computation</span> -<span class="o">...</span> <span class="k">return</span> <span class="n">x</span> -</pre></div> - -</div> -<p>Here the original function takes a single argument named <tt class="docutils literal">x</tt>, -but the decorated function takes any number of arguments and -keyword arguments:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">inspect</span> <span class="kn">import</span> <span class="n">getargspec</span> -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">getargspec</span><span class="p">(</span><span class="n">f1</span><span class="p">))</span> -<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[],</span> <span class="n">varargs</span><span class="o">=</span><span class="s">'args'</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="s">'kw'</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> -</pre></div> - -</div> -<p>This means that introspection tools such as pydoc will give -wrong informations about the signature of <tt class="docutils literal">f1</tt>. This is pretty bad: -pydoc will tell you that the function accepts a generic signature -<tt class="docutils literal">*args</tt>, <tt class="docutils literal">**kw</tt>, but when you try to call the function with more than an -argument, you will get an error:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">f1</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> -<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> - <span class="o">...</span> -<span class="ne">TypeError</span><span class="p">:</span> <span class="n">f1</span><span class="p">()</span> <span class="n">takes</span> <span class="n">exactly</span> <span class="mi">1</span> <span class="n">positional</span> <span class="n">argument</span> <span class="p">(</span><span class="mi">2</span> <span class="n">given</span><span class="p">)</span> -</pre></div> - -</div> -</div> -<div class="section" id="the-solution"> -<h1><a class="toc-backref" href="#id7">The solution</a></h1> -<p>The solution is to provide a generic factory of generators, which -hides the complexity of making signature-preserving decorators -from the application programmer. The <tt class="docutils literal">decorator</tt> function in -the <tt class="docutils literal">decorator</tt> module is such a factory:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">decorator</span> <span class="kn">import</span> <span class="n">decorator</span> -</pre></div> - -</div> -<p><tt class="docutils literal">decorator</tt> takes two arguments, a caller function describing the -functionality of the decorator and a function to be decorated; it -returns the decorated function. The caller function must have -signature <tt class="docutils literal">(f, *args, **kw)</tt> and it must call the original function <tt class="docutils literal">f</tt> -with arguments <tt class="docutils literal">args</tt> and <tt class="docutils literal">kw</tt>, implementing the wanted capability, -i.e. memoization in this case:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">_memoize</span><span class="p">(</span><span class="n">func</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="k">if</span> <span class="n">kw</span><span class="p">:</span> <span class="c"># frozenset is used to ensure hashability</span> - <span class="n">key</span> <span class="o">=</span> <span class="n">args</span><span class="p">,</span> <span class="nb">frozenset</span><span class="p">(</span><span class="n">kw</span><span class="o">.</span><span class="n">iteritems</span><span class="p">())</span> - <span class="k">else</span><span class="p">:</span> - <span class="n">key</span> <span class="o">=</span> <span class="n">args</span> - <span class="n">cache</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="n">cache</span> <span class="c"># attributed added by memoize</span> - <span class="k">if</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">cache</span><span class="p">:</span> - <span class="k">return</span> <span class="n">cache</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> - <span class="k">else</span><span class="p">:</span> - <span class="n">cache</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">result</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> - <span class="k">return</span> <span class="n">result</span> -</pre></div> - -</div> -<p>At this point you can define your decorator as follows:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">memoize</span><span class="p">(</span><span class="n">f</span><span class="p">):</span> - <span class="n">f</span><span class="o">.</span><span class="n">cache</span> <span class="o">=</span> <span class="p">{}</span> - <span class="k">return</span> <span class="n">decorator</span><span class="p">(</span><span class="n">_memoize</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span> -</pre></div> - -</div> -<p>The difference with respect to the <tt class="docutils literal">memoize_uw</tt> approach, which is based -on nested functions, is that the decorator module forces you to lift -the inner function at the outer level (<em>flat is better than nested</em>). -Moreover, you are forced to pass explicitly the function you want to -decorate to the caller function.</p> -<p>Here is a test of usage:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@memoize</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">heavy_computation</span><span class="p">():</span> -<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> -<span class="o">...</span> <span class="k">return</span> <span class="s">"done"</span> - -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">heavy_computation</span><span class="p">())</span> <span class="c"># the first time it will take 2 seconds</span> -<span class="n">done</span> - -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">heavy_computation</span><span class="p">())</span> <span class="c"># the second time it will be instantaneous</span> -<span class="n">done</span> -</pre></div> - -</div> -<p>The signature of <tt class="docutils literal">heavy_computation</tt> is the one you would expect:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">getargspec</span><span class="p">(</span><span class="n">heavy_computation</span><span class="p">))</span> -<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[],</span> <span class="n">varargs</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> -</pre></div> - -</div> -</div> -<div class="section" id="a-trace-decorator"> -<h1><a class="toc-backref" href="#id8">A <tt class="docutils literal">trace</tt> decorator</a></h1> -<p>As an additional example, here is how you can define a trivial -<tt class="docutils literal">trace</tt> decorator, which prints a message everytime the traced -function is called:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">_trace</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="n">kwstr</span> <span class="o">=</span> <span class="s">', '</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s">'</span><span class="si">%r</span><span class="s">: </span><span class="si">%r</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">kw</span><span class="p">[</span><span class="n">k</span><span class="p">])</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">kw</span><span class="p">))</span> - <span class="k">print</span><span class="p">(</span><span class="s">"calling </span><span class="si">%s</span><span class="s"> with args </span><span class="si">%s</span><span class="s">, {</span><span class="si">%s</span><span class="s">}"</span> <span class="o">%</span> <span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwstr</span><span class="p">))</span> - <span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> -</pre></div> - -</div> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">trace</span><span class="p">(</span><span class="n">f</span><span class="p">):</span> - <span class="k">return</span> <span class="n">decorator</span><span class="p">(</span><span class="n">_trace</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span> -</pre></div> - -</div> -<p>Here is an example of usage:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">f1</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> -<span class="o">...</span> <span class="k">pass</span> -</pre></div> - -</div> -<p>It is immediate to verify that <tt class="docutils literal">f1</tt> works</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">f1</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> -<span class="n">calling</span> <span class="n">f1</span> <span class="k">with</span> <span class="n">args</span> <span class="p">(</span><span class="mi">0</span><span class="p">,),</span> <span class="p">{}</span> -</pre></div> - -</div> -<p>and it that it has the correct signature:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">getargspec</span><span class="p">(</span><span class="n">f1</span><span class="p">))</span> -<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[</span><span class="s">'x'</span><span class="p">],</span> <span class="n">varargs</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> -</pre></div> - -</div> -<p>The same decorator works with functions of any signature:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">z</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> -<span class="o">...</span> <span class="k">pass</span> - -<span class="o">>>></span> <span class="n">f</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> -<span class="n">calling</span> <span class="n">f</span> <span class="k">with</span> <span class="n">args</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="p">{}</span> - -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">getargspec</span><span class="p">(</span><span class="n">f</span><span class="p">))</span> -<span class="n">ArgSpec</span><span class="p">(</span><span class="n">args</span><span class="o">=</span><span class="p">[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">,</span> <span class="s">'z'</span><span class="p">],</span> <span class="n">varargs</span><span class="o">=</span><span class="s">'args'</span><span class="p">,</span> <span class="n">keywords</span><span class="o">=</span><span class="s">'kw'</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span> -</pre></div> - -</div> -</div> -<div class="section" id="function-annotations"> -<h1><a class="toc-backref" href="#id9">Function annotations</a></h1> -<p>Python 3 introduced the concept of <a class="reference external" href="http://www.python.org/dev/peps/pep-3107/">function annotations</a>,i.e. the ability -to annotate the signature of a function with additional information, -stored in a dictionary named <tt class="docutils literal">__annotations__</tt>. The decorator module, -starting from release 3.3, is able to understand and to preserve the -annotations. Here is an example:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">:</span> <span class="s">'the first argument'</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="s">'default argument'</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">z</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> -<span class="o">...</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="s">'varargs'</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">:</span> <span class="s">'kwargs'</span><span class="p">):</span> -<span class="o">...</span> <span class="k">pass</span> -</pre></div> - -</div> -<p>In order to introspect functions with annotations, one needs the -utility <tt class="docutils literal">inspect.getfullargspec</tt>, new in Python 3:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">inspect</span> <span class="kn">import</span> <span class="n">getfullargspec</span> -<span class="o">>>></span> <span class="n">argspec</span> <span class="o">=</span> <span class="n">getfullargspec</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> -<span class="o">>>></span> <span class="n">argspec</span><span class="o">.</span><span class="n">args</span> -<span class="p">[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">,</span> <span class="s">'z'</span><span class="p">]</span> -<span class="o">>>></span> <span class="n">argspec</span><span class="o">.</span><span class="n">varargs</span> -<span class="s">'args'</span> -<span class="o">>>></span> <span class="n">argspec</span><span class="o">.</span><span class="n">varkw</span> -<span class="s">'kw'</span> -<span class="o">>>></span> <span class="n">argspec</span><span class="o">.</span><span class="n">defaults</span> -<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> -<span class="o">>>></span> <span class="n">argspec</span><span class="o">.</span><span class="n">kwonlyargs</span> -<span class="p">[]</span> -<span class="o">>>></span> <span class="n">argspec</span><span class="o">.</span><span class="n">kwonlydefaults</span> -</pre></div> - -</div> -<p>You can also check that the <tt class="docutils literal">__annotations__</tt> dictionary is preserved:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">__annotations__</span> <span class="o">==</span> <span class="n">f</span><span class="o">.</span><span class="n">__wrapped__</span><span class="o">.</span><span class="n">__annotations__</span> -<span class="bp">True</span> -</pre></div> - -</div> -<p>Depending on the version of the decorator module, the two dictionaries can -be the same object or not: you cannot rely on object identity, but you can -rely on the content being the same.</p> -</div> -<div class="section" id="decorator-is-a-decorator"> -<h1><a class="toc-backref" href="#id10"><tt class="docutils literal">decorator</tt> is a decorator</a></h1> -<p>It may be annoying to write a caller function (like the <tt class="docutils literal">_trace</tt> -function above) and then a trivial wrapper -(<tt class="docutils literal">def trace(f): return decorator(_trace, f)</tt>) every time. For this reason, -the <tt class="docutils literal">decorator</tt> module provides an easy shortcut to convert -the caller function into a signature-preserving decorator: -you can just call <tt class="docutils literal">decorator</tt> with a single argument. -In our example you can just write <tt class="docutils literal">trace = decorator(_trace)</tt>. -The <tt class="docutils literal">decorator</tt> function can also be used as a signature-changing -decorator, just as <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt>. -However, <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt> return generic -objects which are not callable, while <tt class="docutils literal">decorator</tt> returns -signature-preserving decorators, i.e. functions of a single argument. -For instance, you can write directly</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@decorator</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">trace</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> -<span class="o">...</span> <span class="n">kwstr</span> <span class="o">=</span> <span class="s">', '</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s">'</span><span class="si">%r</span><span class="s">: </span><span class="si">%r</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">kw</span><span class="p">[</span><span class="n">k</span><span class="p">])</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">kw</span><span class="p">))</span> -<span class="o">...</span> <span class="k">print</span><span class="p">(</span><span class="s">"calling </span><span class="si">%s</span><span class="s"> with args </span><span class="si">%s</span><span class="s">, {</span><span class="si">%s</span><span class="s">}"</span> <span class="o">%</span> <span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwstr</span><span class="p">))</span> -<span class="o">...</span> <span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> -</pre></div> - -</div> -<p>and now <tt class="docutils literal">trace</tt> will be a decorator. Actually <tt class="docutils literal">trace</tt> is a <tt class="docutils literal">partial</tt> -object which can be used as a decorator:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">trace</span> -<span class="o"><</span><span class="n">function</span> <span class="n">trace</span> <span class="n">at</span> <span class="mi">0</span><span class="n">x</span><span class="o">...></span> -</pre></div> - -</div> -<p>Here is an example of usage:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">func</span><span class="p">():</span> <span class="k">pass</span> - -<span class="o">>>></span> <span class="n">func</span><span class="p">()</span> -<span class="n">calling</span> <span class="n">func</span> <span class="k">with</span> <span class="n">args</span> <span class="p">(),</span> <span class="p">{}</span> -</pre></div> - -</div> -<p>If you are using an old Python version (Python 2.4) the -<tt class="docutils literal">decorator</tt> module provides a poor man replacement for -<tt class="docutils literal">functools.partial</tt>.</p> -</div> -<div class="section" id="blocking"> -<h1><a class="toc-backref" href="#id11"><tt class="docutils literal">blocking</tt></a></h1> -<p>Sometimes one has to deal with blocking resources, such as <tt class="docutils literal">stdin</tt>, and -sometimes it is best to have back a "busy" message than to block everything. -This behavior can be implemented with a suitable family of decorators, -where the parameter is the busy message:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">blocking</span><span class="p">(</span><span class="n">not_avail</span><span class="p">):</span> - <span class="k">def</span> <span class="nf">blocking</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="k">if</span> <span class="ow">not</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="s">"thread"</span><span class="p">):</span> <span class="c"># no thread running</span> - <span class="k">def</span> <span class="nf">set_result</span><span class="p">():</span> <span class="n">f</span><span class="o">.</span><span class="n">result</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> - <span class="n">f</span><span class="o">.</span><span class="n">thread</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="n">set_result</span><span class="p">)</span> - <span class="n">f</span><span class="o">.</span><span class="n">thread</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> - <span class="k">return</span> <span class="n">not_avail</span> - <span class="k">elif</span> <span class="n">f</span><span class="o">.</span><span class="n">thread</span><span class="o">.</span><span class="n">isAlive</span><span class="p">():</span> - <span class="k">return</span> <span class="n">not_avail</span> - <span class="k">else</span><span class="p">:</span> <span class="c"># the thread is ended, return the stored result</span> - <span class="k">del</span> <span class="n">f</span><span class="o">.</span><span class="n">thread</span> - <span class="k">return</span> <span class="n">f</span><span class="o">.</span><span class="n">result</span> - <span class="k">return</span> <span class="n">decorator</span><span class="p">(</span><span class="n">blocking</span><span class="p">)</span> -</pre></div> - -</div> -<p>Functions decorated with <tt class="docutils literal">blocking</tt> will return a busy message if -the resource is unavailable, and the intended result if the resource is -available. For instance:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@blocking</span><span class="p">(</span><span class="s">"Please wait ..."</span><span class="p">)</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">read_data</span><span class="p">():</span> -<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c"># simulate a blocking resource</span> -<span class="o">...</span> <span class="k">return</span> <span class="s">"some data"</span> - -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">read_data</span><span class="p">())</span> <span class="c"># data is not available yet</span> -<span class="n">Please</span> <span class="n">wait</span> <span class="o">...</span> - -<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">read_data</span><span class="p">())</span> <span class="c"># data is not available yet</span> -<span class="n">Please</span> <span class="n">wait</span> <span class="o">...</span> - -<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">read_data</span><span class="p">())</span> <span class="c"># data is not available yet</span> -<span class="n">Please</span> <span class="n">wait</span> <span class="o">...</span> - -<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">1.1</span><span class="p">)</span> <span class="c"># after 3.1 seconds, data is available</span> -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">read_data</span><span class="p">())</span> -<span class="n">some</span> <span class="n">data</span> -</pre></div> - -</div> -</div> -<div class="section" id="async"> -<h1><a class="toc-backref" href="#id12"><tt class="docutils literal">async</tt></a></h1> -<p>We have just seen an examples of a simple decorator factory, -implemented as a function returning a decorator. -For more complex situations, it is more -convenient to implement decorator factories as classes returning -callable objects that can be converted into decorators.</p> -<p>As an example, here will I show a decorator -which is able to convert a blocking function into an asynchronous -function. The function, when called, -is executed in a separate thread. Moreover, it is possible to set -three callbacks <tt class="docutils literal">on_success</tt>, <tt class="docutils literal">on_failure</tt> and <tt class="docutils literal">on_closing</tt>, -to specify how to manage the function call (of course the code here -is just an example, it is not a recommended way of doing multi-threaded -programming). The implementation is the following:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">on_success</span><span class="p">(</span><span class="n">result</span><span class="p">):</span> <span class="c"># default implementation</span> - <span class="s">"Called on the result of the function"</span> - <span class="k">return</span> <span class="n">result</span> -</pre></div> - -</div> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">on_failure</span><span class="p">(</span><span class="n">exc_info</span><span class="p">):</span> <span class="c"># default implementation</span> - <span class="s">"Called if the function fails"</span> - <span class="k">pass</span> -</pre></div> - -</div> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">on_closing</span><span class="p">():</span> <span class="c"># default implementation</span> - <span class="s">"Called at the end, both in case of success and failure"</span> - <span class="k">pass</span> -</pre></div> - -</div> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">class</span> <span class="nc">Async</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> - <span class="sd">"""</span> -<span class="sd"> A decorator converting blocking functions into asynchronous</span> -<span class="sd"> functions, by using threads or processes. Examples:</span> - -<span class="sd"> async_with_threads = Async(threading.Thread)</span> -<span class="sd"> async_with_processes = Async(multiprocessing.Process)</span> -<span class="sd"> """</span> - - <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">threadfactory</span><span class="p">,</span> <span class="n">on_success</span><span class="o">=</span><span class="n">on_success</span><span class="p">,</span> - <span class="n">on_failure</span><span class="o">=</span><span class="n">on_failure</span><span class="p">,</span> <span class="n">on_closing</span><span class="o">=</span><span class="n">on_closing</span><span class="p">):</span> - <span class="bp">self</span><span class="o">.</span><span class="n">threadfactory</span> <span class="o">=</span> <span class="n">threadfactory</span> - <span class="bp">self</span><span class="o">.</span><span class="n">on_success</span> <span class="o">=</span> <span class="n">on_success</span> - <span class="bp">self</span><span class="o">.</span><span class="n">on_failure</span> <span class="o">=</span> <span class="n">on_failure</span> - <span class="bp">self</span><span class="o">.</span><span class="n">on_closing</span> <span class="o">=</span> <span class="n">on_closing</span> - - <span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="k">try</span><span class="p">:</span> - <span class="n">counter</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="n">counter</span> - <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span> <span class="c"># instantiate the counter at the first call</span> - <span class="n">counter</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="n">counter</span> <span class="o">=</span> <span class="n">itertools</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> - <span class="n">name</span> <span class="o">=</span> <span class="s">'</span><span class="si">%s</span><span class="s">-</span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">func</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="nb">next</span><span class="p">(</span><span class="n">counter</span><span class="p">))</span> - <span class="k">def</span> <span class="nf">func_wrapper</span><span class="p">():</span> - <span class="k">try</span><span class="p">:</span> - <span class="n">result</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> - <span class="k">except</span><span class="p">:</span> - <span class="bp">self</span><span class="o">.</span><span class="n">on_failure</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">())</span> - <span class="k">else</span><span class="p">:</span> - <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">on_success</span><span class="p">(</span><span class="n">result</span><span class="p">)</span> - <span class="k">finally</span><span class="p">:</span> - <span class="bp">self</span><span class="o">.</span><span class="n">on_closing</span><span class="p">()</span> - <span class="n">thread</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">threadfactory</span><span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="n">func_wrapper</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span> - <span class="n">thread</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> - <span class="k">return</span> <span class="n">thread</span> -</pre></div> - -</div> -<p>The decorated function returns -the current execution thread, which can be stored and checked later, for -instance to verify that the thread <tt class="docutils literal">.isAlive()</tt>.</p> -<p>Here is an example of usage. Suppose one wants to write some data to -an external resource which can be accessed by a single user at once -(for instance a printer). Then the access to the writing function must -be locked. Here is a minimalistic example:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">async</span> <span class="o">=</span> <span class="n">decorator</span><span class="p">(</span><span class="n">Async</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">))</span> - -<span class="o">>>></span> <span class="n">datalist</span> <span class="o">=</span> <span class="p">[]</span> <span class="c"># for simplicity the written data are stored into a list.</span> - -<span class="o">>>></span> <span class="nd">@async</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">write</span><span class="p">(</span><span class="n">data</span><span class="p">):</span> -<span class="o">...</span> <span class="c"># append data to the datalist by locking</span> -<span class="o">...</span> <span class="k">with</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">():</span> -<span class="o">...</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="c"># emulate some long running operation</span> -<span class="o">...</span> <span class="n">datalist</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> -<span class="o">...</span> <span class="c"># other operations not requiring a lock here</span> -</pre></div> - -</div> -<p>Each call to <tt class="docutils literal">write</tt> will create a new writer thread, but there will -be no synchronization problems since <tt class="docutils literal">write</tt> is locked.</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">write</span><span class="p">(</span><span class="s">"data1"</span><span class="p">)</span> -<span class="o"><</span><span class="n">Thread</span><span class="p">(</span><span class="n">write</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">started</span><span class="o">...</span><span class="p">)</span><span class="o">></span> - -<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="o">.</span><span class="mi">1</span><span class="p">)</span> <span class="c"># wait a bit, so we are sure data2 is written after data1</span> - -<span class="o">>>></span> <span class="n">write</span><span class="p">(</span><span class="s">"data2"</span><span class="p">)</span> -<span class="o"><</span><span class="n">Thread</span><span class="p">(</span><span class="n">write</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="n">started</span><span class="o">...</span><span class="p">)</span><span class="o">></span> - -<span class="o">>>></span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c"># wait for the writers to complete</span> - -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">datalist</span><span class="p">)</span> -<span class="p">[</span><span class="s">'data1'</span><span class="p">,</span> <span class="s">'data2'</span><span class="p">]</span> -</pre></div> - -</div> -</div> -<div class="section" id="contextmanager"> -<h1><a class="toc-backref" href="#id13">contextmanager</a></h1> -<p>For a long time Python had in its standard library a <tt class="docutils literal">contextmanager</tt> -decorator, able to convert generator functions into -<tt class="docutils literal">_GeneratorContextManager</tt> -factories. For instance if you write</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">contextlib</span> <span class="kn">import</span> <span class="n">contextmanager</span> -<span class="o">>>></span> <span class="nd">@contextmanager</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">before_after</span><span class="p">(</span><span class="n">before</span><span class="p">,</span> <span class="n">after</span><span class="p">):</span> -<span class="o">...</span> <span class="k">print</span><span class="p">(</span><span class="n">before</span><span class="p">)</span> -<span class="o">...</span> <span class="k">yield</span> -<span class="o">...</span> <span class="k">print</span><span class="p">(</span><span class="n">after</span><span class="p">)</span> -</pre></div> - -</div> -<p>then <tt class="docutils literal">before_after</tt> is a factory function returning -<tt class="docutils literal">_GeneratorContextManager</tt> objects which can be used with -the <tt class="docutils literal">with</tt> statement:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">ba</span> <span class="o">=</span> <span class="n">before_after</span><span class="p">(</span><span class="s">'BEFORE'</span><span class="p">,</span> <span class="s">'AFTER'</span><span class="p">)</span> -<span class="o">>>></span> <span class="nb">type</span><span class="p">(</span><span class="n">ba</span><span class="p">)</span> -<span class="o"><</span><span class="k">class</span> <span class="err">'</span><span class="nc">contextlib</span><span class="o">.</span><span class="n">_GeneratorContextManager</span><span class="s">'></span> -<span class="o">>>></span> <span class="k">with</span> <span class="n">ba</span><span class="p">:</span> -<span class="o">...</span> <span class="k">print</span><span class="p">(</span><span class="s">'hello'</span><span class="p">)</span> -<span class="n">BEFORE</span> -<span class="n">hello</span> -<span class="n">AFTER</span> -</pre></div> - -</div> -<p>Basically, it is as if the content of the <tt class="docutils literal">with</tt> block was executed -in the place of the <tt class="docutils literal">yield</tt> expression in the generator function. -In Python 3.2 <tt class="docutils literal">_GeneratorContextManager</tt> -objects were enhanced with a <tt class="docutils literal">__call__</tt> -method, so that they can be used as decorators as in this example:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@ba</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">hello</span><span class="p">():</span> -<span class="o">...</span> <span class="k">print</span><span class="p">(</span><span class="s">'hello'</span><span class="p">)</span> -<span class="o">...</span> -<span class="o">>>></span> <span class="n">hello</span><span class="p">()</span> -<span class="n">BEFORE</span> -<span class="n">hello</span> -<span class="n">AFTER</span> -</pre></div> - -</div> -<p>The <tt class="docutils literal">ba</tt> decorator is basically inserting a <tt class="docutils literal">with ba:</tt> -block inside the function. -However there two issues: the first is that <tt class="docutils literal">_GeneratorContextManager</tt> -objects are callable only in Python 3.2, so the previous example will break -in older versions of Python; the second is that -<tt class="docutils literal">_GeneratorContextManager</tt> objects do not preserve the signature -of the decorated functions: the decorated <tt class="docutils literal">hello</tt> function here will have -a generic signature <tt class="docutils literal"><span class="pre">hello(*args,</span> **kwargs)</tt> but will break when -called with more than zero arguments. For such reasons the decorator -module, starting with release 3.4, offers a <tt class="docutils literal">decorator.contextmanager</tt> -decorator that solves both problems and works even in Python 2.5. -The usage is the same and factories decorated with <tt class="docutils literal">decorator.contextmanager</tt> -will returns instances of <tt class="docutils literal">ContextManager</tt>, a subclass of -<tt class="docutils literal">contextlib._GeneratorContextManager</tt> with a <tt class="docutils literal">__call__</tt> method -acting as a signature-preserving decorator.</p> -</div> -<div class="section" id="the-functionmaker-class"> -<h1><a class="toc-backref" href="#id14">The <tt class="docutils literal">FunctionMaker</tt> class</a></h1> -<p>You may wonder about how the functionality of the <tt class="docutils literal">decorator</tt> module -is implemented. The basic building block is -a <tt class="docutils literal">FunctionMaker</tt> class which is able to generate on the fly -functions with a given name and signature from a function template -passed as a string. Generally speaking, you should not need to -resort to <tt class="docutils literal">FunctionMaker</tt> when writing ordinary decorators, but -it is handy in some circumstances. You will see an example shortly, in -the implementation of a cool decorator utility (<tt class="docutils literal">decorator_apply</tt>).</p> -<p><tt class="docutils literal">FunctionMaker</tt> provides a <tt class="docutils literal">.create</tt> classmethod which -takes as input the name, signature, and body of the function -we want to generate as well as the execution environment -were the function is generated by <tt class="docutils literal">exec</tt>. Here is an example:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> <span class="c"># a function with a generic signature</span> -<span class="o">...</span> <span class="k">print</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="n">kw</span><span class="p">)</span> - -<span class="o">>>></span> <span class="n">f1</span> <span class="o">=</span> <span class="n">FunctionMaker</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="s">'f1(a, b)'</span><span class="p">,</span> <span class="s">'f(a, b)'</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">f</span><span class="o">=</span><span class="n">f</span><span class="p">))</span> -<span class="o">>>></span> <span class="n">f1</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span> -<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> <span class="p">{}</span> -</pre></div> - -</div> -<p>It is important to notice that the function body is interpolated -before being executed, so be careful with the <tt class="docutils literal">%</tt> sign!</p> -<p><tt class="docutils literal">FunctionMaker.create</tt> also accepts keyword arguments and such -arguments are attached to the resulting function. This is useful -if you want to set some function attributes, for instance the -docstring <tt class="docutils literal">__doc__</tt>.</p> -<p>For debugging/introspection purposes it may be useful to see -the source code of the generated function; to do that, just -pass the flag <tt class="docutils literal">addsource=True</tt> and a <tt class="docutils literal">__source__</tt> attribute will -be added to the generated function:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">f1</span> <span class="o">=</span> <span class="n">FunctionMaker</span><span class="o">.</span><span class="n">create</span><span class="p">(</span> -<span class="o">...</span> <span class="s">'f1(a, b)'</span><span class="p">,</span> <span class="s">'f(a, b)'</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">f</span><span class="o">=</span><span class="n">f</span><span class="p">),</span> <span class="n">addsource</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">f1</span><span class="o">.</span><span class="n">__source__</span><span class="p">)</span> -<span class="k">def</span> <span class="nf">f1</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span> - <span class="n">f</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span> -<span class="o"><</span><span class="n">BLANKLINE</span><span class="o">></span> -</pre></div> - -</div> -<p><tt class="docutils literal">FunctionMaker.create</tt> can take as first argument a string, -as in the examples before, or a function. This is the most common -usage, since typically you want to decorate a pre-existing -function. A framework author may want to use directly <tt class="docutils literal">FunctionMaker.create</tt> -instead of <tt class="docutils literal">decorator</tt>, since it gives you direct access to the body -of the generated function. For instance, suppose you want to instrument -the <tt class="docutils literal">__init__</tt> methods of a set of classes, by preserving their -signature (such use case is not made up; this is done in SQAlchemy -and in other frameworks). When the first argument of <tt class="docutils literal">FunctionMaker.create</tt> -is a function, a <tt class="docutils literal">FunctionMaker</tt> object is instantiated internally, -with attributes <tt class="docutils literal">args</tt>, <tt class="docutils literal">varargs</tt>, -<tt class="docutils literal">keywords</tt> and <tt class="docutils literal">defaults</tt> which are the -the return values of the standard library function <tt class="docutils literal">inspect.getargspec</tt>. -For each argument in the <tt class="docutils literal">args</tt> (which is a list of strings containing -the names of the mandatory arguments) an attribute <tt class="docutils literal">arg0</tt>, <tt class="docutils literal">arg1</tt>, -..., <tt class="docutils literal">argN</tt> is also generated. Finally, there is a <tt class="docutils literal">signature</tt> -attribute, a string with the signature of the original function.</p> -<p>Notice that while I do not have plans -to change or remove the functionality provided in the -<tt class="docutils literal">FunctionMaker</tt> class, I do not guarantee that it will stay -unchanged forever. For instance, right now I am using the traditional -string interpolation syntax for function templates, but Python 2.6 -and Python 3.0 provide a newer interpolation syntax and I may use -the new syntax in the future. -On the other hand, the functionality provided by -<tt class="docutils literal">decorator</tt> has been there from version 0.1 and it is guaranteed to -stay there forever.</p> -</div> -<div class="section" id="getting-the-source-code"> -<h1><a class="toc-backref" href="#id15">Getting the source code</a></h1> -<p>Internally <tt class="docutils literal">FunctionMaker.create</tt> uses <tt class="docutils literal">exec</tt> to generate the -decorated function. Therefore -<tt class="docutils literal">inspect.getsource</tt> will not work for decorated functions. That -means that the usual '??' trick in IPython will give you the (right on -the spot) message <tt class="docutils literal">Dynamically generated function. No source code -available</tt>. In the past I have considered this acceptable, since -<tt class="docutils literal">inspect.getsource</tt> does not really work even with regular -decorators. In that case <tt class="docutils literal">inspect.getsource</tt> gives you the wrapper -source code which is probably not what you want:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">identity_dec</span><span class="p">(</span><span class="n">func</span><span class="p">):</span> - <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> - <span class="k">return</span> <span class="n">wrapper</span> -</pre></div> - -</div> -<div class="codeblock python"> -<div class="highlight"><pre><span class="nd">@identity_dec</span> -<span class="k">def</span> <span class="nf">example</span><span class="p">():</span> <span class="k">pass</span> - -<span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">inspect</span><span class="o">.</span><span class="n">getsource</span><span class="p">(</span><span class="n">example</span><span class="p">))</span> - <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span> - <span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> -<span class="o"><</span><span class="n">BLANKLINE</span><span class="o">></span> -</pre></div> - -</div> -<p>(see bug report <a class="reference external" href="http://bugs.python.org/issue1764286">1764286</a> for an explanation of what is happening). -Unfortunately the bug is still there, even in Python 2.7 and 3.1. -There is however a workaround. The decorator module adds an -attribute <tt class="docutils literal">.__wrapped__</tt> to the decorated function, containing -a reference to the original function. The easy way to get -the source code is to call <tt class="docutils literal">inspect.getsource</tt> on the -undecorated function:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">inspect</span><span class="o">.</span><span class="n">getsource</span><span class="p">(</span><span class="n">factorial</span><span class="o">.</span><span class="n">__wrapped__</span><span class="p">))</span> -<span class="nd">@tail_recursive</span> -<span class="k">def</span> <span class="nf">factorial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">acc</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span> - <span class="s">"The good old factorial"</span> - <span class="k">if</span> <span class="n">n</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="k">return</span> <span class="n">acc</span> - <span class="k">return</span> <span class="n">factorial</span><span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="o">*</span><span class="n">acc</span><span class="p">)</span> -<span class="o"><</span><span class="n">BLANKLINE</span><span class="o">></span> -</pre></div> - -</div> -</div> -<div class="section" id="dealing-with-third-party-decorators"> -<h1><a class="toc-backref" href="#id16">Dealing with third party decorators</a></h1> -<p>Sometimes you find on the net some cool decorator that you would -like to include in your code. However, more often than not the cool -decorator is not signature-preserving. Therefore you may want an easy way to -upgrade third party decorators to signature-preserving decorators without -having to rewrite them in terms of <tt class="docutils literal">decorator</tt>. You can use a -<tt class="docutils literal">FunctionMaker</tt> to implement that functionality as follows:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">decorator_apply</span><span class="p">(</span><span class="n">dec</span><span class="p">,</span> <span class="n">func</span><span class="p">):</span> - <span class="sd">"""</span> -<span class="sd"> Decorate a function by preserving the signature even if dec</span> -<span class="sd"> is not a signature-preserving decorator.</span> -<span class="sd"> """</span> - <span class="k">return</span> <span class="n">FunctionMaker</span><span class="o">.</span><span class="n">create</span><span class="p">(</span> - <span class="n">func</span><span class="p">,</span> <span class="s">'return decorated(</span><span class="si">%(signature)s</span><span class="s">)'</span><span class="p">,</span> - <span class="nb">dict</span><span class="p">(</span><span class="n">decorated</span><span class="o">=</span><span class="n">dec</span><span class="p">(</span><span class="n">func</span><span class="p">)),</span> <span class="n">__wrapped__</span><span class="o">=</span><span class="n">func</span><span class="p">)</span> -</pre></div> - -</div> -<p><tt class="docutils literal">decorator_apply</tt> sets the attribute <tt class="docutils literal">.__wrapped__</tt> of the generated -function to the original function, so that you can get the right -source code.</p> -<p>Notice that I am not providing this functionality in the <tt class="docutils literal">decorator</tt> -module directly since I think it is best to rewrite the decorator rather -than adding an additional level of indirection. However, practicality -beats purity, so you can add <tt class="docutils literal">decorator_apply</tt> to your toolbox and -use it if you need to.</p> -<p>In order to give an example of usage of <tt class="docutils literal">decorator_apply</tt>, I will show a -pretty slick decorator that converts a tail-recursive function in an iterative -function. I have shamelessly stolen the basic idea from Kay Schluehr's recipe -in the Python Cookbook, -<a class="reference external" href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691">http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691</a>.</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">class</span> <span class="nc">TailRecursive</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> - <span class="sd">"""</span> -<span class="sd"> tail_recursive decorator based on Kay Schluehr's recipe</span> -<span class="sd"> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691</span> -<span class="sd"> with improvements by me and George Sakkis.</span> -<span class="sd"> """</span> - - <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">func</span><span class="p">):</span> - <span class="bp">self</span><span class="o">.</span><span class="n">func</span> <span class="o">=</span> <span class="n">func</span> - <span class="bp">self</span><span class="o">.</span><span class="n">firstcall</span> <span class="o">=</span> <span class="bp">True</span> - <span class="bp">self</span><span class="o">.</span><span class="n">CONTINUE</span> <span class="o">=</span> <span class="nb">object</span><span class="p">()</span> <span class="c"># sentinel</span> - - <span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwd</span><span class="p">):</span> - <span class="n">CONTINUE</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">CONTINUE</span> - <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">firstcall</span><span class="p">:</span> - <span class="n">func</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">func</span> - <span class="bp">self</span><span class="o">.</span><span class="n">firstcall</span> <span class="o">=</span> <span class="bp">False</span> - <span class="k">try</span><span class="p">:</span> - <span class="k">while</span> <span class="bp">True</span><span class="p">:</span> - <span class="n">result</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwd</span><span class="p">)</span> - <span class="k">if</span> <span class="n">result</span> <span class="ow">is</span> <span class="n">CONTINUE</span><span class="p">:</span> <span class="c"># update arguments</span> - <span class="n">args</span><span class="p">,</span> <span class="n">kwd</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">argskwd</span> - <span class="k">else</span><span class="p">:</span> <span class="c"># last call</span> - <span class="k">return</span> <span class="n">result</span> - <span class="k">finally</span><span class="p">:</span> - <span class="bp">self</span><span class="o">.</span><span class="n">firstcall</span> <span class="o">=</span> <span class="bp">True</span> - <span class="k">else</span><span class="p">:</span> <span class="c"># return the arguments of the tail call</span> - <span class="bp">self</span><span class="o">.</span><span class="n">argskwd</span> <span class="o">=</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwd</span> - <span class="k">return</span> <span class="n">CONTINUE</span> -</pre></div> - -</div> -<p>Here the decorator is implemented as a class returning callable -objects.</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">tail_recursive</span><span class="p">(</span><span class="n">func</span><span class="p">):</span> - <span class="k">return</span> <span class="n">decorator_apply</span><span class="p">(</span><span class="n">TailRecursive</span><span class="p">,</span> <span class="n">func</span><span class="p">)</span> -</pre></div> - -</div> -<p>Here is how you apply the upgraded decorator to the good old factorial:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="nd">@tail_recursive</span> -<span class="k">def</span> <span class="nf">factorial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">acc</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span> - <span class="s">"The good old factorial"</span> - <span class="k">if</span> <span class="n">n</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="k">return</span> <span class="n">acc</span> - <span class="k">return</span> <span class="n">factorial</span><span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="o">*</span><span class="n">acc</span><span class="p">)</span> -</pre></div> - -</div> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="k">print</span><span class="p">(</span><span class="n">factorial</span><span class="p">(</span><span class="mi">4</span><span class="p">))</span> -<span class="mi">24</span> -</pre></div> - -</div> -<p>This decorator is pretty impressive, and should give you some food for -your mind ;) Notice that there is no recursion limit now, and you can -easily compute <tt class="docutils literal">factorial(1001)</tt> or larger without filling the stack -frame. Notice also that the decorator will not work on functions which -are not tail recursive, such as the following</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="k">def</span> <span class="nf">fact</span><span class="p">(</span><span class="n">n</span><span class="p">):</span> <span class="c"># this is not tail-recursive</span> - <span class="k">if</span> <span class="n">n</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="k">return</span> <span class="mi">1</span> - <span class="k">return</span> <span class="n">n</span> <span class="o">*</span> <span class="n">fact</span><span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> -</pre></div> - -</div> -<p>(reminder: a function is tail recursive if it either returns a value without -making a recursive call, or returns directly the result of a recursive -call).</p> -</div> -<div class="section" id="caveats-and-limitations"> -<h1><a class="toc-backref" href="#id17">Caveats and limitations</a></h1> -<p>The first thing you should be aware of, it the fact that decorators -have a performance penalty. -The worse case is shown by the following example:</p> -<pre class="literal-block"> -$ cat performance.sh -python3 -m timeit -s " -from decorator import decorator - -@decorator -def do_nothing(func, *args, **kw): - return func(*args, **kw) - -@do_nothing -def f(): - pass -" "f()" - -python3 -m timeit -s " -def f(): - pass -" "f()" -</pre> -<p>On my MacBook, using the <tt class="docutils literal">do_nothing</tt> decorator instead of the -plain function is more than three times slower:</p> -<pre class="literal-block"> -$ bash performance.sh -1000000 loops, best of 3: 0.669 usec per loop -1000000 loops, best of 3: 0.181 usec per loop -</pre> -<p>It should be noted that a real life function would probably do -something more useful than <tt class="docutils literal">f</tt> here, and therefore in real life the -performance penalty could be completely negligible. As always, the -only way to know if there is -a penalty in your specific use case is to measure it.</p> -<p>You should be aware that decorators will make your tracebacks -longer and more difficult to understand. Consider this example:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">():</span> -<span class="o">...</span> <span class="mi">1</span><span class="o">/</span><span class="mi">0</span> -</pre></div> - -</div> -<p>Calling <tt class="docutils literal">f()</tt> will give you a <tt class="docutils literal">ZeroDivisionError</tt>, but since the -function is decorated the traceback will be longer:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="n">f</span><span class="p">()</span> -<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> - <span class="o">...</span> - <span class="n">File</span> <span class="s">"<string>"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">2</span><span class="p">,</span> <span class="ow">in</span> <span class="n">f</span> - <span class="n">File</span> <span class="s">"<doctest __main__[22]>"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">4</span><span class="p">,</span> <span class="ow">in</span> <span class="n">trace</span> - <span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span> - <span class="n">File</span> <span class="s">"<doctest __main__[51]>"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">3</span><span class="p">,</span> <span class="ow">in</span> <span class="n">f</span> - <span class="mi">1</span><span class="o">/</span><span class="mi">0</span> -<span class="ne">ZeroDivisionError</span><span class="p">:</span> <span class="o">...</span> -</pre></div> - -</div> -<p>You see here the inner call to the decorator <tt class="docutils literal">trace</tt>, which calls -<tt class="docutils literal"><span class="pre">f(*args,</span> **kw)</tt>, and a reference to <tt class="docutils literal">File <span class="pre">"<string>",</span> line 2, in f</tt>. -This latter reference is due to the fact that internally the decorator -module uses <tt class="docutils literal">exec</tt> to generate the decorated function. Notice that -<tt class="docutils literal">exec</tt> is <em>not</em> responsibile for the performance penalty, since is the -called <em>only once</em> at function decoration time, and not every time -the decorated function is called.</p> -<p>At present, there is no clean way to avoid <tt class="docutils literal">exec</tt>. A clean solution -would require to change the CPython implementation of functions and -add an hook to make it possible to change their signature directly. -That could happen in future versions of Python (see PEP <a class="reference external" href="http://www.python.org/dev/peps/pep-0362">362</a>) and -then the decorator module would become obsolete. However, at present, -even in Python 3.2 it is impossible to change the function signature -directly, therefore the <tt class="docutils literal">decorator</tt> module is still useful. -Actually, this is one of the main reasons why I keep maintaining -the module and releasing new versions.</p> -<p>In the present implementation, decorators generated by <tt class="docutils literal">decorator</tt> -can only be used on user-defined Python functions or methods, not on generic -callable objects, nor on built-in functions, due to limitations of the -<tt class="docutils literal">inspect</tt> module in the standard library.</p> -<p>There is a restriction on the names of the arguments: for instance, -if try to call an argument <tt class="docutils literal">_call_</tt> or <tt class="docutils literal">_func_</tt> -you will get a <tt class="docutils literal">NameError</tt>:</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="nd">@trace</span> -<span class="o">...</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">_func_</span><span class="p">):</span> <span class="k">print</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> -<span class="o">...</span> -<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span> - <span class="o">...</span> -<span class="ne">NameError</span><span class="p">:</span> <span class="n">_func_</span> <span class="ow">is</span> <span class="n">overridden</span> <span class="ow">in</span> -<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">_func_</span><span class="p">):</span> - <span class="k">return</span> <span class="n">_call_</span><span class="p">(</span><span class="n">_func_</span><span class="p">,</span> <span class="n">_func_</span><span class="p">)</span> -</pre></div> - -</div> -<p>Finally, the implementation is such that the decorated function contains -a <em>copy</em> of the original function dictionary -(<tt class="docutils literal">vars(decorated_f) is not vars(f)</tt>):</p> -<div class="codeblock python"> -<div class="highlight"><pre><span class="o">>>></span> <span class="k">def</span> <span class="nf">f</span><span class="p">():</span> <span class="k">pass</span> <span class="c"># the original function</span> -<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">attr1</span> <span class="o">=</span> <span class="s">"something"</span> <span class="c"># setting an attribute</span> -<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">attr2</span> <span class="o">=</span> <span class="s">"something else"</span> <span class="c"># setting another attribute</span> - -<span class="o">>>></span> <span class="n">traced_f</span> <span class="o">=</span> <span class="n">trace</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="c"># the decorated function</span> - -<span class="o">>>></span> <span class="n">traced_f</span><span class="o">.</span><span class="n">attr1</span> -<span class="s">'something'</span> -<span class="o">>>></span> <span class="n">traced_f</span><span class="o">.</span><span class="n">attr2</span> <span class="o">=</span> <span class="s">"something different"</span> <span class="c"># setting attr</span> -<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">attr2</span> <span class="c"># the original attribute did not change</span> -<span class="s">'something else'</span> -</pre></div> - -</div> -</div> -<div class="section" id="compatibility-notes"> -<h1><a class="toc-backref" href="#id18">Compatibility notes</a></h1> -<p>Version 3.3 is the first version of the <tt class="docutils literal">decorator</tt> module to fully -support Python 3, including <a class="reference external" href="http://www.python.org/dev/peps/pep-3107/">function annotations</a>. Version 3.2 was the -first version to support Python 3 via the <tt class="docutils literal">2to3</tt> conversion tool -invoked in the build process by the <a class="reference external" href="http://packages.python.org/distribute/">distribute</a> project, the Python -3-compatible replacement of easy_install. The hard work (for me) has -been converting the documentation and the doctests. This has been -possible only after that <a class="reference external" href="http://docutils.sourceforge.net/">docutils</a> and <a class="reference external" href="http://pygments.org/">pygments</a> have been ported to -Python 3.</p> -<p>Version 3 of the <tt class="docutils literal">decorator</tt> module do not contain any backward -incompatible change, apart from the removal of the functions -<tt class="docutils literal">get_info</tt> and <tt class="docutils literal">new_wrapper</tt>, which have been deprecated for -years. <tt class="docutils literal">get_info</tt> has been removed since it was little used and -since it had to be changed anyway to work with Python 3.0; -<tt class="docutils literal">new_wrapper</tt> has been removed since it was useless: its major use -case (converting signature changing decorators to signature preserving -decorators) has been subsumed by <tt class="docutils literal">decorator_apply</tt>, whereas the other use -case can be managed with the <tt class="docutils literal">FunctionMaker</tt>.</p> -<p>There are a few changes in the documentation: I removed the -<tt class="docutils literal">decorator_factory</tt> example, which was confusing some of my users, -and I removed the part about exotic signatures in the Python 3 -documentation, since Python 3 does not support them.</p> -<p>Finally <tt class="docutils literal">decorator</tt> cannot be used as a class decorator and the -<a class="reference external" href="http://www.phyast.pitt.edu/~micheles/python/documentation.html#class-decorators-and-decorator-factories">functionality introduced in version 2.3</a> has been removed. That -means that in order to define decorator factories with classes you -need to define the <tt class="docutils literal">__call__</tt> method explicitly (no magic anymore). -All these changes should not cause any trouble, since they were -all rarely used features. Should you have any trouble, you can always -downgrade to the 2.3 version.</p> -<p>The examples shown here have been tested with Python 2.6. Python 2.4 -is also supported - of course the examples requiring the <tt class="docutils literal">with</tt> -statement will not work there. Python 2.5 works fine, but if you -run the examples in the interactive interpreter -you will notice a few differences since -<tt class="docutils literal">getargspec</tt> returns an <tt class="docutils literal">ArgSpec</tt> namedtuple instead of a regular -tuple. That means that running the file -<tt class="docutils literal">documentation.py</tt> under Python 2.5 will print a few errors, but -they are not serious.</p> -</div> -<div class="section" id="licence"> -<h1><a class="toc-backref" href="#id19">LICENCE</a></h1> -<p>Copyright (c) 2005-2012, Michele Simionato -All rights reserved.</p> -<p>Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are -met:</p> -<blockquote> -Redistributions of source code must retain the above copyright -notice, this list of conditions and the following disclaimer. -Redistributions in bytecode form must reproduce the above copyright -notice, this list of conditions and the following disclaimer in -the documentation and/or other materials provided with the -distribution.</blockquote> -<p>THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, -INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, -BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS -OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR -TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE -USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH -DAMAGE.</p> -<p>If you use this software and you are happy with it, consider sending me a -note, just to gratify my ego. On the other hand, if you use this software and -you are unhappy with it, send me a patch!</p> -</div> -</div> -</body> -</html> |