diff options
49 files changed, 679 insertions, 336 deletions
diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index b28aa5f793..b4efbf00f7 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -70,8 +70,7 @@ Unless otherwise stated, buffers are not NUL-terminated. as *converter*. ``s*`` (:class:`str`, :class:`bytes`, :class:`bytearray` or buffer compatible object) [Py_buffer] - This format accepts Unicode objects as well as objects supporting the - buffer protocol. + This format accepts Unicode objects as well as :term:`bytes-like object`\ s. It fills a :c:type:`Py_buffer` structure provided by the caller. In this case the resulting C string may contain embedded NUL bytes. Unicode objects are converted to C strings using ``'utf-8'`` encoding. @@ -101,14 +100,14 @@ Unless otherwise stated, buffers are not NUL-terminated. contain embedded NUL bytes; if it does, a :exc:`TypeError` exception is raised. -``y*`` (:class:`bytes`, :class:`bytearray` or buffer compatible object) [Py_buffer] - This variant on ``s*`` doesn't accept Unicode objects, only objects - supporting the buffer protocol. **This is the recommended way to accept +``y*`` (:class:`bytes`, :class:`bytearray` or :term:`bytes-like object`) [Py_buffer] + This variant on ``s*`` doesn't accept Unicode objects, only + :term:`bytes-like object`\ s. **This is the recommended way to accept binary data.** ``y#`` (:class:`bytes`) [const char \*, int] - This variant on ``s#`` doesn't accept Unicode objects, only bytes-like - objects. + This variant on ``s#`` doesn't accept Unicode objects, only :term:`bytes-like + object`\ s. ``S`` (:class:`bytes`) [PyBytesObject \*] Requires that the Python object is a :class:`bytes` object, without diff --git a/Doc/c-api/bytearray.rst b/Doc/c-api/bytearray.rst index 95ded96eb6..82022056df 100644 --- a/Doc/c-api/bytearray.rst +++ b/Doc/c-api/bytearray.rst @@ -40,7 +40,7 @@ Direct API functions .. c:function:: PyObject* PyByteArray_FromObject(PyObject *o) Return a new bytearray object from any object, *o*, that implements the - buffer protocol. + :ref:`buffer protocol <bufferobjects>`. .. XXX expand about the buffer protocol, at least somewhere diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 19419a3c36..63f1021afc 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -79,8 +79,12 @@ Glossary <http://www.python.org/~guido/>`_, Python's creator. bytes-like object - An object that supports the :ref:`bufferobjects`, like :class:`bytes` or - :class:`bytearray`. + An object that supports the :ref:`bufferobjects`, like :class:`bytes`, + :class:`bytearray` or :class:`memoryview`. Bytes-like objects can + be used for various operations that expect binary data, such as + compression, saving to a binary file or sending over a socket. + Some operations need the binary data to be mutable, in which case + not all bytes-like objects can apply. bytecode Python source code is compiled into bytecode, the internal representation @@ -248,6 +252,16 @@ Glossary the execution of the body. See also :term:`parameter`, :term:`method`, and the :ref:`function` section. + function annotation + An arbitrary metadata value associated with a function parameter or return + value. Its syntax is explained in section :ref:`function`. Annotations + may be accessed via the :attr:`__annotations__` special attribute of a + function object. + + Python itself does not assign any particular meaning to function + annotations. They are intended to be interpreted by third-party libraries + or tools. See :pep:`3107`, which describes some of their potential uses. + __future__ A pseudo-module which programmers can use to enable new language features which are not compatible with the current interpreter. diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index 1b14ceb6bd..ce1f4413cb 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -5,39 +5,43 @@ ********************************** :Author: A.M. Kuchling, Eric S. Raymond -:Release: 2.03 +:Release: 2.04 .. topic:: Abstract - This document describes how to write text-mode programs with Python 2.x, using - the :mod:`curses` extension module to control the display. + This document describes how to use the :mod:`curses` extension + module to control text-mode displays. What is curses? =============== The curses library supplies a terminal-independent screen-painting and -keyboard-handling facility for text-based terminals; such terminals include -VT100s, the Linux console, and the simulated terminal provided by X11 programs -such as xterm and rxvt. Display terminals support various control codes to -perform common operations such as moving the cursor, scrolling the screen, and -erasing areas. Different terminals use widely differing codes, and often have -their own minor quirks. - -In a world of X displays, one might ask "why bother"? It's true that -character-cell display terminals are an obsolete technology, but there are -niches in which being able to do fancy things with them are still valuable. One -is on small-footprint or embedded Unixes that don't carry an X server. Another -is for tools like OS installers and kernel configurators that may have to run -before X is available. - -The curses library hides all the details of different terminals, and provides -the programmer with an abstraction of a display, containing multiple -non-overlapping windows. The contents of a window can be changed in various -ways-- adding text, erasing it, changing its appearance--and the curses library -will automagically figure out what control codes need to be sent to the terminal -to produce the right output. +keyboard-handling facility for text-based terminals; such terminals +include VT100s, the Linux console, and the simulated terminal provided +by various programs. Display terminals support various control codes +to perform common operations such as moving the cursor, scrolling the +screen, and erasing areas. Different terminals use widely differing +codes, and often have their own minor quirks. + +In a world of graphical displays, one might ask "why bother"? It's +true that character-cell display terminals are an obsolete technology, +but there are niches in which being able to do fancy things with them +are still valuable. One niche is on small-footprint or embedded +Unixes that don't run an X server. Another is tools such as OS +installers and kernel configurators that may have to run before any +graphical support is available. + +The curses library provides fairly basic functionality, providing the +programmer with an abstraction of a display containing multiple +non-overlapping windows of text. The contents of a window can be +changed in various ways---adding text, erasing it, changing its +appearance---and the curses library will figure out what control codes +need to be sent to the terminal to produce the right output. curses +doesn't provide many user-interface concepts such as buttons, checkboxes, +or dialogs; if you need such features, consider a user interface library such as +`Urwid <https://pypi.python.org/pypi/urwid/>`_. The curses library was originally written for BSD Unix; the later System V versions of Unix from AT&T added many enhancements and new functions. BSD curses @@ -49,10 +53,13 @@ code, all the functions described here will probably be available. The older versions of curses carried by some proprietary Unixes may not support everything, though. -No one has made a Windows port of the curses module. On a Windows platform, try -the Console module written by Fredrik Lundh. The Console module provides -cursor-addressable text output, plus full support for mouse and keyboard input, -and is available from http://effbot.org/zone/console-index.htm. +The Windows version of Python doesn't include the :mod:`curses` +module. A ported version called `UniCurses +<https://pypi.python.org/pypi/UniCurses>`_ is available. You could +also try `the Console module <http://effbot.org/zone/console-index.htm>`_ +written by Fredrik Lundh, which doesn't +use the same API as curses but provides cursor-addressable text output +and full support for mouse and keyboard input. The Python curses module @@ -61,11 +68,12 @@ The Python curses module Thy Python module is a fairly simple wrapper over the C functions provided by curses; if you're already familiar with curses programming in C, it's really easy to transfer that knowledge to Python. The biggest difference is that the -Python interface makes things simpler, by merging different C functions such as -:func:`addstr`, :func:`mvaddstr`, :func:`mvwaddstr`, into a single -:meth:`addstr` method. You'll see this covered in more detail later. +Python interface makes things simpler by merging different C functions such as +:c:func:`addstr`, :c:func:`mvaddstr`, and :c:func:`mvwaddstr` into a single +:meth:`~curses.window.addstr` method. You'll see this covered in more +detail later. -This HOWTO is simply an introduction to writing text-mode programs with curses +This HOWTO is an introduction to writing text-mode programs with curses and Python. It doesn't attempt to be a complete guide to the curses API; for that, see the Python library guide's section on ncurses, and the C manual pages for ncurses. It will, however, give you the basic ideas. @@ -74,25 +82,27 @@ for ncurses. It will, however, give you the basic ideas. Starting and ending a curses application ======================================== -Before doing anything, curses must be initialized. This is done by calling the -:func:`initscr` function, which will determine the terminal type, send any -required setup codes to the terminal, and create various internal data -structures. If successful, :func:`initscr` returns a window object representing -the entire screen; this is usually called ``stdscr``, after the name of the +Before doing anything, curses must be initialized. This is done by +calling the :func:`~curses.initscr` function, which will determine the +terminal type, send any required setup codes to the terminal, and +create various internal data structures. If successful, +:func:`initscr` returns a window object representing the entire +screen; this is usually called ``stdscr`` after the name of the corresponding C variable. :: import curses stdscr = curses.initscr() -Usually curses applications turn off automatic echoing of keys to the screen, in -order to be able to read keys and only display them under certain circumstances. -This requires calling the :func:`noecho` function. :: +Usually curses applications turn off automatic echoing of keys to the +screen, in order to be able to read keys and only display them under +certain circumstances. This requires calling the +:func:`~curses.noecho` function. :: curses.noecho() -Applications will also commonly need to react to keys instantly, without -requiring the Enter key to be pressed; this is called cbreak mode, as opposed to -the usual buffered input mode. :: +Applications will also commonly need to react to keys instantly, +without requiring the Enter key to be pressed; this is called cbreak +mode, as opposed to the usual buffered input mode. :: curses.cbreak() @@ -103,12 +113,14 @@ curses can do it for you, returning a special value such as :const:`curses.KEY_LEFT`. To get curses to do the job, you'll have to enable keypad mode. :: - stdscr.keypad(1) + stdscr.keypad(True) Terminating a curses application is much easier than starting one. You'll need -to call :: +to call:: - curses.nocbreak(); stdscr.keypad(0); curses.echo() + curses.nocbreak() + stdscr.keypad(False) + curses.echo() to reverse the curses-friendly terminal settings. Then call the :func:`endwin` function to restore the terminal to its original operating mode. :: @@ -122,102 +134,147 @@ raises an uncaught exception. Keys are no longer echoed to the screen when you type them, for example, which makes using the shell difficult. In Python you can avoid these complications and make debugging much easier by -importing the module :mod:`curses.wrapper`. It supplies a :func:`wrapper` -function that takes a callable. It does the initializations described above, -and also initializes colors if color support is present. It then runs your -provided callable and finally deinitializes appropriately. The callable is -called inside a try-catch clause which catches exceptions, performs curses -deinitialization, and then passes the exception upwards. Thus, your terminal -won't be left in a funny state on exception. +importing the :func:`curses.wrapper` function and using it like this:: + + from curses import wrapper + + def main(stdscr): + # Clear screen + stdscr.clear() + + # This raises ZeroDivisionError when i == 10. + for i in range(0, 10): + v = i-10 + stdscr.addstr(i, 0, '10 divided by {} is {}'.format(v, 10/v)) + + stdscr.refresh() + stdscr.getkey() + + wrapper(main) + +The :func:`wrapper` function takes a callable object and does the +initializations described above, also initializing colors if color +support is present. :func:`wrapper` then runs your provided callable. +Once the callable returns, :func:`wrapper` will restore the original +state of the terminal. The callable is called inside a +:keyword:`try`...\ :keyword:`except` that catches exceptions, restores +the state of the terminal, and then re-raises the exception. Therefore +your terminal won't be left in a funny state on exception and you'll be +able to read the exception's message and traceback. Windows and Pads ================ Windows are the basic abstraction in curses. A window object represents a -rectangular area of the screen, and supports various methods to display text, +rectangular area of the screen, and supports methods to display text, erase it, allow the user to input strings, and so forth. -The ``stdscr`` object returned by the :func:`initscr` function is a window -object that covers the entire screen. Many programs may need only this single -window, but you might wish to divide the screen into smaller windows, in order -to redraw or clear them separately. The :func:`newwin` function creates a new -window of a given size, returning the new window object. :: +The ``stdscr`` object returned by the :func:`initscr` function is a +window object that covers the entire screen. Many programs may need +only this single window, but you might wish to divide the screen into +smaller windows, in order to redraw or clear them separately. The +:func:`~curses.newwin` function creates a new window of a given size, +returning the new window object. :: begin_x = 20 ; begin_y = 7 height = 5 ; width = 40 win = curses.newwin(height, width, begin_y, begin_x) -A word about the coordinate system used in curses: coordinates are always passed -in the order *y,x*, and the top-left corner of a window is coordinate (0,0). -This breaks a common convention for handling coordinates, where the *x* -coordinate usually comes first. This is an unfortunate difference from most -other computer applications, but it's been part of curses since it was first -written, and it's too late to change things now. - -When you call a method to display or erase text, the effect doesn't immediately -show up on the display. This is because curses was originally written with slow -300-baud terminal connections in mind; with these terminals, minimizing the time -required to redraw the screen is very important. This lets curses accumulate -changes to the screen, and display them in the most efficient manner. For -example, if your program displays some characters in a window, and then clears -the window, there's no need to send the original characters because they'd never -be visible. - -Accordingly, curses requires that you explicitly tell it to redraw windows, -using the :func:`refresh` method of window objects. In practice, this doesn't +Note that the coordinate system used in curses is unusual. +Coordinates are always passed in the order *y,x*, and the top-left +corner of a window is coordinate (0,0). This breaks the normal +convention for handling coordinates where the *x* coordinate comes +first. This is an unfortunate difference from most other computer +applications, but it's been part of curses since it was first written, +and it's too late to change things now. + +Your application can determine the size of the screen by using the +:data:`curses.LINES` and :data:`curses.COLS` variables to obtain the *y* and +*x* sizes. Legal coordinates will then extend from ``(0,0)`` to +``(curses.LINES - 1, curses.COLS - 1)``. + +When you call a method to display or erase text, the effect doesn't +immediately show up on the display. Instead you must call the +:meth:`~curses.window.refresh` method of window objects to update the +screen. + +This is because curses was originally written with slow 300-baud +terminal connections in mind; with these terminals, minimizing the +time required to redraw the screen was very important. Instead curses +accumulates changes to the screen and displays them in the most +efficient manner when you call :meth:`refresh`. For example, if your +program displays some text in a window and then clears the window, +there's no need to send the original text because they're never +visible. + +In practice, explicitly telling curses to redraw a window doesn't really complicate programming with curses much. Most programs go into a flurry of activity, and then pause waiting for a keypress or some other action on the part of the user. All you have to do is to be sure that the screen has been -redrawn before pausing to wait for user input, by simply calling -``stdscr.refresh()`` or the :func:`refresh` method of some other relevant +redrawn before pausing to wait for user input, by first calling +``stdscr.refresh()`` or the :meth:`refresh` method of some other relevant window. A pad is a special case of a window; it can be larger than the actual display -screen, and only a portion of it displayed at a time. Creating a pad simply +screen, and only a portion of the pad displayed at a time. Creating a pad requires the pad's height and width, while refreshing a pad requires giving the coordinates of the on-screen area where a subsection of the pad will be -displayed. :: +displayed. :: pad = curses.newpad(100, 100) - # These loops fill the pad with letters; this is + # These loops fill the pad with letters; addch() is # explained in the next section - for y in range(0, 100): - for x in range(0, 100): - try: pad.addch(y,x, ord('a') + (x*x+y*y) % 26 ) - except curses.error: pass - - # Displays a section of the pad in the middle of the screen + for y in range(0, 99): + for x in range(0, 99): + pad.addch(y,x, ord('a') + (x*x+y*y) % 26 ) + + # Displays a section of the pad in the middle of the screen. + # (0,0) : coordinate of upper-left corner of pad area to display. + # (5,5) : coordinate of upper-left corner of window area to be filled + # with pad content. + # (20, 75) : coordinate of lower-right corner of window area to be + # : filled with pad content. pad.refresh( 0,0, 5,5, 20,75) -The :func:`refresh` call displays a section of the pad in the rectangle +The :meth:`refresh` call displays a section of the pad in the rectangle extending from coordinate (5,5) to coordinate (20,75) on the screen; the upper left corner of the displayed section is coordinate (0,0) on the pad. Beyond that difference, pads are exactly like ordinary windows and support the same methods. -If you have multiple windows and pads on screen there is a more efficient way to -go, which will prevent annoying screen flicker at refresh time. Use the -:meth:`noutrefresh` method of each window to update the data structure -representing the desired state of the screen; then change the physical screen to -match the desired state in one go with the function :func:`doupdate`. The -normal :meth:`refresh` method calls :func:`doupdate` as its last act. +If you have multiple windows and pads on screen there is a more +efficient way to update the screen and prevent annoying screen flicker +as each part of the screen gets updated. :meth:`refresh` actually +does two things: + +1) Calls the :meth:`~curses.window.noutrefresh` method of each window + to update an underlying data structure representing the desired + state of the screen. +2) Calls the function :func:`~curses.doupdate` function to change the + physical screen to match the desired state recorded in the data structure. + +Instead you can call :meth:`noutrefresh` on a number of windows to +update the data structure, and then call :func:`doupdate` to update +the screen. Displaying Text =============== -From a C programmer's point of view, curses may sometimes look like a twisty -maze of functions, all subtly different. For example, :func:`addstr` displays a -string at the current cursor location in the ``stdscr`` window, while -:func:`mvaddstr` moves to a given y,x coordinate first before displaying the -string. :func:`waddstr` is just like :func:`addstr`, but allows specifying a -window to use, instead of using ``stdscr`` by default. :func:`mvwaddstr` follows -similarly. +From a C programmer's point of view, curses may sometimes look like a +twisty maze of functions, all subtly different. For example, +:c:func:`addstr` displays a string at the current cursor location in +the ``stdscr`` window, while :c:func:`mvaddstr` moves to a given y,x +coordinate first before displaying the string. :c:func:`waddstr` is just +like :func:`addstr`, but allows specifying a window to use instead of +using ``stdscr`` by default. :c:func:`mvwaddstr` allows specifying both +a window and a coordinate. -Fortunately the Python interface hides all these details; ``stdscr`` is a window -object like any other, and methods like :func:`addstr` accept multiple argument -forms. Usually there are four different forms. +Fortunately the Python interface hides all these details. ``stdscr`` +is a window object like any other, and methods such as :meth:`addstr` +accept multiple argument forms. Usually there are four different +forms. +---------------------------------+-----------------------------------------------+ | Form | Description | @@ -236,17 +293,26 @@ forms. Usually there are four different forms. | | display *str* or *ch*, using attribute *attr* | +---------------------------------+-----------------------------------------------+ -Attributes allow displaying text in highlighted forms, such as in boldface, +Attributes allow displaying text in highlighted forms such as boldface, underline, reverse code, or in color. They'll be explained in more detail in the next subsection. -The :func:`addstr` function takes a Python string as the value to be displayed, -while the :func:`addch` functions take a character, which can be either a Python -string of length 1 or an integer. If it's a string, you're limited to -displaying characters between 0 and 255. SVr4 curses provides constants for -extension characters; these constants are integers greater than 255. For -example, :const:`ACS_PLMINUS` is a +/- symbol, and :const:`ACS_ULCORNER` is the -upper left corner of a box (handy for drawing borders). + +The :meth:`~curses.window.addstr` method takes a Python string or +bytestring as the value to be displayed. The contents of bytestrings +are sent to the terminal as-is. Strings are encoded to bytes using +the value of the window's :attr:`encoding` attribute; this defaults to +the default system encoding as returned by +:func:`locale.getpreferredencoding`. + +The :meth:`~curses.window.addch` methods take a character, which can be +either a string of length 1, a bytestring of length 1, or an integer. + +Constants are provided for extension characters; these constants are +integers greater than 255. For example, :const:`ACS_PLMINUS` is a +/- +symbol, and :const:`ACS_ULCORNER` is the upper left corner of a box +(handy for drawing borders). You can also use the appropriate Unicode +character. Windows remember where the cursor was left after the last operation, so if you leave out the *y,x* coordinates, the string or character will be displayed @@ -256,10 +322,11 @@ you may want to ensure that the cursor is positioned in some location where it won't be distracting; it can be confusing to have the cursor blinking at some apparently random location. -If your application doesn't need a blinking cursor at all, you can call -``curs_set(0)`` to make it invisible. Equivalently, and for compatibility with -older curses versions, there's a ``leaveok(bool)`` function. When *bool* is -true, the curses library will attempt to suppress the flashing cursor, and you +If your application doesn't need a blinking cursor at all, you can +call ``curs_set(False)`` to make it invisible. For compatibility +with older curses versions, there's a ``leaveok(bool)`` function +that's a synonym for :func:`curs_set`. When *bool* is true, the +curses library will attempt to suppress the flashing cursor, and you won't need to worry about leaving it in odd locations. @@ -267,15 +334,16 @@ Attributes and Color -------------------- Characters can be displayed in different ways. Status lines in a text-based -application are commonly shown in reverse video; a text viewer may need to +application are commonly shown in reverse video, or a text viewer may need to highlight certain words. curses supports this by allowing you to specify an attribute for each cell on the screen. -An attribute is an integer, each bit representing a different attribute. You can -try to display text with multiple attribute bits set, but curses doesn't -guarantee that all the possible combinations are available, or that they're all -visually distinct. That depends on the ability of the terminal being used, so -it's safest to stick to the most commonly available attributes, listed here. +An attribute is an integer, each bit representing a different +attribute. You can try to display text with multiple attribute bits +set, but curses doesn't guarantee that all the possible combinations +are available, or that they're all visually distinct. That depends on +the ability of the terminal being used, so it's safest to stick to the +most commonly available attributes, listed here. +----------------------+--------------------------------------+ | Attribute | Description | @@ -306,7 +374,7 @@ xterms. To use color, you must call the :func:`start_color` function soon after calling :func:`initscr`, to initialize the default color set (the -:func:`curses.wrapper.wrapper` function does this automatically). Once that's +:func:`curses.wrapper` function does this automatically). Once that's done, the :func:`has_colors` function returns TRUE if the terminal in use can actually display color. (Note: curses uses the American spelling 'color', instead of the Canadian/British spelling 'colour'. If you're used to the @@ -325,15 +393,16 @@ An example, which displays a line of text using color pair 1:: stdscr.refresh() As I said before, a color pair consists of a foreground and background color. -:func:`start_color` initializes 8 basic colors when it activates color mode. -They are: 0:black, 1:red, 2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and -7:white. The curses module defines named constants for each of these colors: -:const:`curses.COLOR_BLACK`, :const:`curses.COLOR_RED`, and so forth. - The ``init_pair(n, f, b)`` function changes the definition of color pair *n*, to foreground color f and background color b. Color pair 0 is hard-wired to white on black, and cannot be changed. +Colors are numbered, and :func:`start_color` initializes 8 basic +colors when it activates color mode. They are: 0:black, 1:red, +2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The :mod:`curses` +module defines named constants for each of these colors: +:const:`curses.COLOR_BLACK`, :const:`curses.COLOR_RED`, and so forth. + Let's put all this together. To change color 1 to red text on a white background, you would call:: @@ -350,87 +419,130 @@ RGB value. This lets you change color 1, which is usually red, to purple or blue or any other color you like. Unfortunately, the Linux console doesn't support this, so I'm unable to try it out, and can't provide any examples. You can check if your terminal can do this by calling :func:`can_change_color`, -which returns TRUE if the capability is there. If you're lucky enough to have +which returns True if the capability is there. If you're lucky enough to have such a talented terminal, consult your system's man pages for more information. User Input ========== -The curses library itself offers only very simple input mechanisms. Python's -support adds a text-input widget that makes up some of the lack. - -The most common way to get input to a window is to use its :meth:`getch` method. -:meth:`getch` pauses and waits for the user to hit a key, displaying it if -:func:`echo` has been called earlier. You can optionally specify a coordinate -to which the cursor should be moved before pausing. - -It's possible to change this behavior with the method :meth:`nodelay`. After -``nodelay(1)``, :meth:`getch` for the window becomes non-blocking and returns -``curses.ERR`` (a value of -1) when no input is ready. There's also a -:func:`halfdelay` function, which can be used to (in effect) set a timer on each -:meth:`getch`; if no input becomes available within a specified -delay (measured in tenths of a second), curses raises an exception. +The C curses library offers only very simple input mechanisms. Python's +:mod:`curses` module adds a basic text-input widget. (Other libraries +such as `Urwid <https://pypi.python.org/pypi/urwid/>`_ have more extensive +collections of widgets.) + +There are two methods for getting input from a window: + +* :meth:`~curses.window.getch` refreshes the screen and then waits for + the user to hit a key, displaying the key if :func:`echo` has been + called earlier. You can optionally specify a coordinate to which + the cursor should be moved before pausing. + +* :meth:`~curses.window.getkey` does the same thing but converts the + integer to a string. Individual characters are returned as + 1-character strings, and special keys such as function keys return + longer strings containing a key name such as ``KEY_UP`` or ``^G``. + +It's possible to not wait for the user using the +:meth:`~curses.window.nodelay` window method. After ``nodelay(True)``, +:meth:`getch` and :meth:`getkey` for the window become +non-blocking. To signal that no input is ready, :meth:`getch` returns +``curses.ERR`` (a value of -1) and :meth:`getkey` raises an exception. +There's also a :func:`~curses.halfdelay` function, which can be used to (in +effect) set a timer on each :meth:`getch`; if no input becomes +available within a specified delay (measured in tenths of a second), +curses raises an exception. The :meth:`getch` method returns an integer; if it's between 0 and 255, it represents the ASCII code of the key pressed. Values greater than 255 are special keys such as Page Up, Home, or the cursor keys. You can compare the value returned to constants such as :const:`curses.KEY_PPAGE`, -:const:`curses.KEY_HOME`, or :const:`curses.KEY_LEFT`. Usually the main loop of -your program will look something like this:: +:const:`curses.KEY_HOME`, or :const:`curses.KEY_LEFT`. The main loop of +your program may look something like this:: while True: c = stdscr.getch() - if c == ord('p'): PrintDocument() - elif c == ord('q'): break # Exit the while() - elif c == curses.KEY_HOME: x = y = 0 + if c == ord('p'): + PrintDocument() + elif c == ord('q'): + break # Exit the while loop + elif c == curses.KEY_HOME: + x = y = 0 The :mod:`curses.ascii` module supplies ASCII class membership functions that -take either integer or 1-character-string arguments; these may be useful in -writing more readable tests for your command interpreters. It also supplies +take either integer or 1-character string arguments; these may be useful in +writing more readable tests for such loops. It also supplies conversion functions that take either integer or 1-character-string arguments and return the same type. For example, :func:`curses.ascii.ctrl` returns the control character corresponding to its argument. -There's also a method to retrieve an entire string, :const:`getstr()`. It isn't -used very often, because its functionality is quite limited; the only editing -keys available are the backspace key and the Enter key, which terminates the -string. It can optionally be limited to a fixed number of characters. :: +There's also a method to retrieve an entire string, +:meth:`~curses.window.getstr`. It isn't used very often, because its +functionality is quite limited; the only editing keys available are +the backspace key and the Enter key, which terminates the string. It +can optionally be limited to a fixed number of characters. :: curses.echo() # Enable echoing of characters # Get a 15-character string, with the cursor on the top line s = stdscr.getstr(0,0, 15) -The Python :mod:`curses.textpad` module supplies something better. With it, you -can turn a window into a text box that supports an Emacs-like set of -keybindings. Various methods of :class:`Textbox` class support editing with -input validation and gathering the edit results either with or without trailing -spaces. See the library documentation on :mod:`curses.textpad` for the -details. +The :mod:`curses.textpad` module supplies a text box that supports an +Emacs-like set of keybindings. Various methods of the +:class:`~curses.textpad.Textbox` class support editing with input +validation and gathering the edit results either with or without +trailing spaces. Here's an example:: + + import curses + from curses.textpad import Textbox, rectangle + def main(stdscr): + stdscr.addstr(0, 0, "Enter IM message: (hit Ctrl-G to send)") -For More Information -==================== + editwin = curses.newwin(5,30, 2,1) + rectangle(stdscr, 1,0, 1+5+1, 1+30+1) + stdscr.refresh() -This HOWTO didn't cover some advanced topics, such as screen-scraping or -capturing mouse events from an xterm instance. But the Python library page for -the curses modules is now pretty complete. You should browse it next. + box = Textbox(editwin) -If you're in doubt about the detailed behavior of any of the ncurses entry -points, consult the manual pages for your curses implementation, whether it's -ncurses or a proprietary Unix vendor's. The manual pages will document any -quirks, and provide complete lists of all the functions, attributes, and -:const:`ACS_\*` characters available to you. + # Let the user edit until Ctrl-G is struck. + box.edit() -Because the curses API is so large, some functions aren't supported in the -Python interface, not because they're difficult to implement, but because no one -has needed them yet. Feel free to add them and then submit a patch. Also, we -don't yet have support for the menu library associated with -ncurses; feel free to add that. + # Get resulting contents + message = box.gather() -If you write an interesting little program, feel free to contribute it as -another demo. We can always use more of them! +See the library documentation on :mod:`curses.textpad` for more details. -The ncurses FAQ: http://invisible-island.net/ncurses/ncurses.faq.html +For More Information +==================== + +This HOWTO doesn't cover some advanced topics, such as reading the +contents of the screen or capturing mouse events from an xterm +instance, but the Python library page for the :mod:`curses` module is now +reasonably complete. You should browse it next. + +If you're in doubt about the detailed behavior of the curses +functions, consult the manual pages for your curses implementation, +whether it's ncurses or a proprietary Unix vendor's. The manual pages +will document any quirks, and provide complete lists of all the +functions, attributes, and :const:`ACS_\*` characters available to +you. + +Because the curses API is so large, some functions aren't supported in +the Python interface. Often this isn't because they're difficult to +implement, but because no one has needed them yet. Also, Python +doesn't yet support the menu library associated with ncurses. +Patches adding support for these would be welcome; see +`the Python Developer's Guide <http://docs.python.org/devguide/>`_ to +learn more about submitting patches to Python. + +* `Writing Programs with NCURSES <http://invisible-island.net/ncurses/ncurses-intro.html>`_: + a lengthy tutorial for C programmers. +* `The ncurses man page <http://www.linuxmanpages.com/man3/ncurses.3x.php>`_ +* `The ncurses FAQ <http://invisible-island.net/ncurses/ncurses.faq.html>`_ +* `"Use curses... don't swear" <http://www.youtube.com/watch?v=eN1eZtjLEnU>`_: + video of a PyCon 2013 talk on controlling terminals using curses or Urwid. +* `"Console Applications with Urwid" <http://www.pyvideo.org/video/1568/console-applications-with-urwid>`_: + video of a PyCon CA 2012 talk demonstrating some applications written using + Urwid. diff --git a/Doc/library/array.rst b/Doc/library/array.rst index 8f6943a240..752bad5773 100644 --- a/Doc/library/array.rst +++ b/Doc/library/array.rst @@ -73,8 +73,8 @@ The module defines the following type: .. class:: array(typecode[, initializer]) A new array whose items are restricted by *typecode*, and initialized - from the optional *initializer* value, which must be a list, object - supporting the buffer interface, or iterable over elements of the + from the optional *initializer* value, which must be a list, a + :term:`bytes-like object`, or iterable over elements of the appropriate type. If given a list or string, the initializer is passed to the new array's @@ -91,7 +91,7 @@ Array objects support the ordinary sequence operations of indexing, slicing, concatenation, and multiplication. When using slice assignment, the assigned value must be an array object with the same type code; in all other cases, :exc:`TypeError` is raised. Array objects also implement the buffer interface, -and may be used wherever buffer objects are supported. +and may be used wherever :term:`bytes-like object`\ s are supported. The following data items and methods are also supported: diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst index baf430db89..02ec5d854a 100644 --- a/Doc/library/binascii.rst +++ b/Doc/library/binascii.rst @@ -21,8 +21,9 @@ higher-level modules. .. note:: ``a2b_*`` functions accept Unicode strings containing only ASCII characters. - Other functions only accept bytes and bytes-compatible objects (such as - bytearray objects and other objects implementing the buffer API). + Other functions only accept :term:`bytes-like object`\ s (such as + :class:`bytes`, :class:`bytearray` and other objects that support the buffer + protocol). .. versionchanged:: 3.3 ASCII-only unicode strings are now accepted by the ``a2b_*`` functions. diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index 9e1a9c744a..27a02cdacf 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1188,42 +1188,44 @@ particular, the following variants typically exist: The following codecs provide bytes-to-bytes mappings. -.. tabularcolumns:: |l|p{0.3\linewidth}|p{0.3\linewidth}| +.. tabularcolumns:: |l|L| -+--------------------+---------------------------+---------------------------+ -| Codec | Aliases | Purpose | -+====================+===========================+===========================+ -| base64_codec | base64, base-64 | Convert operand to MIME | -| | | base64 | -+--------------------+---------------------------+---------------------------+ -| bz2_codec | bz2 | Compress the operand | -| | | using bz2 | -+--------------------+---------------------------+---------------------------+ -| hex_codec | hex | Convert operand to | -| | | hexadecimal | -| | | representation, with two | -| | | digits per byte | -+--------------------+---------------------------+---------------------------+ -| quopri_codec | quopri, quoted-printable, | Convert operand to MIME | -| | quotedprintable | quoted printable | -+--------------------+---------------------------+---------------------------+ -| uu_codec | uu | Convert the operand using | -| | | uuencode | -+--------------------+---------------------------+---------------------------+ -| zlib_codec | zip, zlib | Compress the operand | -| | | using gzip | -+--------------------+---------------------------+---------------------------+ ++--------------------+---------------------------+ +| Codec | Purpose | ++====================+===========================+ +| base64_codec | Convert operand to MIME | +| | base64 (the result always | +| | includes a trailing | +| | ``'\n'``) | ++--------------------+---------------------------+ +| bz2_codec | Compress the operand | +| | using bz2 | ++--------------------+---------------------------+ +| hex_codec | Convert operand to | +| | hexadecimal | +| | representation, with two | +| | digits per byte | ++--------------------+---------------------------+ +| quopri_codec | Convert operand to MIME | +| | quoted printable | ++--------------------+---------------------------+ +| uu_codec | Convert the operand using | +| | uuencode | ++--------------------+---------------------------+ +| zlib_codec | Compress the operand | +| | using gzip | ++--------------------+---------------------------+ The following codecs provide string-to-string mappings. -.. tabularcolumns:: |l|p{0.3\linewidth}|p{0.3\linewidth}| +.. tabularcolumns:: |l|L| -+--------------------+---------------------------+---------------------------+ -| Codec | Aliases | Purpose | -+====================+===========================+===========================+ -| rot_13 | rot13 | Returns the Caesar-cypher | -| | | encryption of the operand | -+--------------------+---------------------------+---------------------------+ ++--------------------+---------------------------+ +| Codec | Purpose | ++====================+===========================+ +| rot_13 | Returns the Caesar-cypher | +| | encryption of the operand | ++--------------------+---------------------------+ .. versionadded:: 3.2 bytes-to-bytes and string-to-string codecs. diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst index fee5067bd0..fba48f47d5 100644 --- a/Doc/library/contextlib.rst +++ b/Doc/library/contextlib.rst @@ -259,11 +259,12 @@ Functions and classes provided: with ExitStack() as stack: files = [stack.enter_context(open(fname)) for fname in filenames] - close_files = stack.pop_all().close() + # Hold onto the close method, but don't call it yet. + close_files = stack.pop_all().close # If opening any file fails, all previously opened files will be # closed automatically. If all files are opened successfully, # they will remain open even after the with statement ends. - # close_files() can then be invoked explicitly to close them all + # close_files() can then be invoked explicitly to close them all. .. method:: close() diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index 929d41b439..0a1b208d96 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -32,9 +32,9 @@ digests. The modern term is secure hash. There is one constructor method named for each type of :dfn:`hash`. All return a hash object with the same simple interface. For example: use :func:`sha1` to -create a SHA1 hash object. You can now feed this object with objects conforming -to the buffer interface (normally :class:`bytes` objects) using the -:meth:`update` method. At any point you can ask it for the :dfn:`digest` of the +create a SHA1 hash object. You can now feed this object with :term:`bytes-like +object`\ s (normally :class:`bytes`) using the :meth:`update` method. +At any point you can ask it for the :dfn:`digest` of the concatenation of the data fed to it so far using the :meth:`digest` or :meth:`hexdigest` methods. diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst index 0706ff4378..c2066a7bbd 100644 --- a/Doc/library/hmac.rst +++ b/Doc/library/hmac.rst @@ -74,8 +74,7 @@ This module also provides the following helper function: timing analysis by avoiding content-based short circuiting behaviour, making it appropriate for cryptography. *a* and *b* must both be of the same type: either :class:`str` (ASCII only, as e.g. returned by - :meth:`HMAC.hexdigest`), or any type that supports the buffer protocol - (e.g. :class:`bytes`). + :meth:`HMAC.hexdigest`), or a :term:`bytes-like object`. .. note:: diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 1eb554a40a..7099fa031b 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -705,9 +705,9 @@ which incur interpreter overhead. next(b, None) return zip(a, b) - def grouper(n, iterable, fillvalue=None): + def grouper(iterable, n, fillvalue=None): "Collect data into fixed-length chunks or blocks" - # grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx" + # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx" args = [iter(iterable)] * n return zip_longest(*args, fillvalue=fillvalue) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index fe38d23ea6..1c18062f8d 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -800,8 +800,7 @@ Connection objects are usually created using :func:`Pipe` -- see also .. method:: send_bytes(buffer[, offset[, size]]) - Send byte data from an object supporting the buffer interface as a - complete message. + Send byte data from a :term:`bytes-like object` as a complete message. If *offset* is given then data is read from that position in *buffer*. If *size* is given then that many bytes will be read from buffer. Very large @@ -832,7 +831,7 @@ Connection objects are usually created using :func:`Pipe` -- see also :exc:`EOFError` if there is nothing left to receive and the other end was closed. - *buffer* must be an object satisfying the writable buffer interface. If + *buffer* must be a writable :term:`bytes-like object`. If *offset* is given then the message will be written into the buffer from that position. Offset must be a non-negative integer less than the length of *buffer* (in bytes). diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index 3860880bf0..de7a542474 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -241,13 +241,22 @@ lookups. These are useful for making fast field extractors as arguments for expect a function argument. -.. function:: attrgetter(attr[, args...]) +.. function:: attrgetter(attr) + attrgetter(*attrs) - Return a callable object that fetches *attr* from its operand. If more than one - attribute is requested, returns a tuple of attributes. After, - ``f = attrgetter('name')``, the call ``f(b)`` returns ``b.name``. After, - ``f = attrgetter('name', 'date')``, the call ``f(b)`` returns ``(b.name, - b.date)``. Equivalent to:: + Return a callable object that fetches *attr* from its operand. + If more than one attribute is requested, returns a tuple of attributes. + The attribute names can also contain dots. For example: + + * After ``f = attrgetter('name')``, the call ``f(b)`` returns ``b.name``. + + * After ``f = attrgetter('name', 'date')``, the call ``f(b)`` returns + ``(b.name, b.date)``. + + * After ``f = attrgetter('name.first', 'name.last')``, the call ``f(b)`` + returns ``(r.name.first, r.name.last)``. + + Equivalent to:: def attrgetter(*items): if any(not isinstance(item, str) for item in items): @@ -267,14 +276,19 @@ expect a function argument. return obj - The attribute names can also contain dots; after ``f = attrgetter('date.month')``, - the call ``f(b)`` returns ``b.date.month``. - -.. function:: itemgetter(item[, args...]) +.. function:: itemgetter(item) + itemgetter(*items) Return a callable object that fetches *item* from its operand using the operand's :meth:`__getitem__` method. If multiple items are specified, - returns a tuple of lookup values. Equivalent to:: + returns a tuple of lookup values. For example: + + * After ``f = itemgetter(2)``, the call ``f(r)`` returns ``r[2]``. + + * After ``g = itemgetter(2, 5, 3)``, the call ``g(r)`` returns + ``(r[2], r[5], r[3])``. + + Equivalent to:: def itemgetter(*items): if len(items) == 1: @@ -313,9 +327,14 @@ expect a function argument. Return a callable object that calls the method *name* on its operand. If additional arguments and/or keyword arguments are given, they will be given - to the method as well. After ``f = methodcaller('name')``, the call ``f(b)`` - returns ``b.name()``. After ``f = methodcaller('name', 'foo', bar=1)``, the - call ``f(b)`` returns ``b.name('foo', bar=1)``. Equivalent to:: + to the method as well. For example: + + * After ``f = methodcaller('name')``, the call ``f(b)`` returns ``b.name()``. + + * After ``f = methodcaller('name', 'foo', bar=1)``, the call ``f(b)`` + returns ``b.name('foo', bar=1)``. + + Equivalent to:: def methodcaller(name, *args, **kwargs): def caller(obj): diff --git a/Doc/library/os.rst b/Doc/library/os.rst index e70c8869ce..6d4fb04108 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -999,9 +999,8 @@ as internal buffering of data. On Mac OS X and FreeBSD, a value of 0 for *nbytes* specifies to send until the end of *in* is reached. - On Solaris, *out* may be the file descriptor of a regular file or the file - descriptor of a socket. On all other platforms, *out* must be the file - descriptor of an open socket. + All platforms support sockets as *out* file descriptor, and some platforms + allow other types (e.g. regular file, pipe) as well. Availability: Unix. diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 03ab5db38e..e7c777bc9d 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -519,9 +519,8 @@ class`. In addition, it provides one more method: >>> int.from_bytes([255, 0, 0], byteorder='big') 16711680 - The argument *bytes* must either support the buffer protocol or be an - iterable producing bytes. :class:`bytes` and :class:`bytearray` are - examples of built-in objects that support the buffer protocol. + The argument *bytes* must either be a :term:`bytes-like object` or an + iterable producing bytes. The *byteorder* argument determines the byte order used to represent the integer. If *byteorder* is ``"big"``, the most significant byte is at the @@ -1417,10 +1416,9 @@ multiple fragments. single: bytes; str (built-in class) If at least one of *encoding* or *errors* is given, *object* should be a - :class:`bytes` or :class:`bytearray` object, or more generally any object - that supports the :ref:`buffer protocol <bufferobjects>`. In this case, if - *object* is a :class:`bytes` (or :class:`bytearray`) object, then - ``str(bytes, encoding, errors)`` is equivalent to + :term:`bytes-like object` (e.g. :class:`bytes` or :class:`bytearray`). In + this case, if *object* is a :class:`bytes` (or :class:`bytearray`) object, + then ``str(bytes, encoding, errors)`` is equivalent to :meth:`bytes.decode(encoding, errors) <bytes.decode>`. Otherwise, the bytes object underlying the buffer object is obtained before calling :meth:`bytes.decode`. See :ref:`binaryseq` and @@ -2911,8 +2909,8 @@ The constructors for both classes work the same: based on their members. For example, ``set('abc') == frozenset('abc')`` returns ``True`` and so does ``set('abc') in set([frozenset('abc')])``. - The subset and equality comparisons do not generalize to a complete ordering - function. For example, any two disjoint sets are not equal and are not + The subset and equality comparisons do not generalize to a total ordering + function. For example, any two nonempty disjoint sets are not equal and are not subsets of each other, so *all* of the following return ``False``: ``a<b``, ``a==b``, or ``a>b``. diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index c44ab23b31..85d1ce6190 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1746,7 +1746,8 @@ Loading and running tests instead of repeatedly creating new instances. -.. class:: TextTestRunner(stream=None, descriptions=True, verbosity=1, runnerclass=None, warnings=None) +.. class:: TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, \ + buffer=False, resultclass=None, warnings=None) A basic test runner implementation that outputs results to a stream. If *stream* is ``None``, the default, :data:`sys.stderr` is used as the output stream. This class diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 81dc748242..b9ebaaa554 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -737,22 +737,6 @@ in the module's namespace which do not begin with an underscore character to avoid accidentally exporting items that are not part of the API (such as library modules which were imported and used within the module). -The :keyword:`from` form with ``*`` may only occur in a module scope. -Attempting to use it in class or function definitions will raise a -:exc:`SyntaxError`. - -.. index:: single: __all__ (optional module attribute) - -The *public names* defined by a module are determined by checking the module's -namespace for a variable named ``__all__``; if defined, it must be a sequence -of strings which are names defined or imported by that module. The names -given in ``__all__`` are all considered public and are required to exist. If -``__all__`` is not defined, the set of public names includes all names found -in the module's namespace which do not begin with an underscore character -(``'_'``). ``__all__`` should contain the entire public API. It is intended -to avoid accidentally exporting items that are not part of the API (such as -library modules which were imported and used within the module). - The :keyword:`from` form with ``*`` may only occur in a module scope. The wild card form of import --- ``import *`` --- is only allowed at the module level. Attempting to use it in class or function definitions will raise a diff --git a/Include/pythonrun.h b/Include/pythonrun.h index 4d24b2df7d..e8a582d50a 100644 --- a/Include/pythonrun.h +++ b/Include/pythonrun.h @@ -219,6 +219,7 @@ PyAPI_FUNC(void) PyFloat_Fini(void); PyAPI_FUNC(void) PyOS_FiniInterrupts(void); PyAPI_FUNC(void) _PyGC_Fini(void); PyAPI_FUNC(void) PySlice_Fini(void); +PyAPI_FUNC(void) _PyType_Fini(void); PyAPI_DATA(PyThreadState *) _Py_Finalizing; #endif diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index 707c53b3e1..9f55a3e7a3 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -281,6 +281,10 @@ class {typename}(tuple): 'Return self as a plain tuple. Used by copy and pickle.' return tuple(self) + def __getstate__(self): + 'Exclude the OrderedDict from pickling' + return None + {field_defs} ''' diff --git a/Lib/ctypes/test/__init__.py b/Lib/ctypes/test/__init__.py index 82dc9e3e9e..cc5fe02d1b 100644 --- a/Lib/ctypes/test/__init__.py +++ b/Lib/ctypes/test/__init__.py @@ -62,7 +62,7 @@ def get_tests(package, mask, verbosity, exclude=()): continue try: mod = __import__(modname, globals(), locals(), ['*']) - except ResourceDenied as detail: + except (ResourceDenied, unittest.SkipTest) as detail: skipped.append(modname) if verbosity > 1: print("Skipped %s: %s" % (modname, detail), file=sys.stderr) diff --git a/Lib/ctypes/test/test_wintypes.py b/Lib/ctypes/test/test_wintypes.py new file mode 100644 index 0000000000..806fccef81 --- /dev/null +++ b/Lib/ctypes/test/test_wintypes.py @@ -0,0 +1,43 @@ +import sys +import unittest + +if not sys.platform.startswith('win'): + raise unittest.SkipTest('Windows-only test') + +from ctypes import * +from ctypes import wintypes + +class WinTypesTest(unittest.TestCase): + def test_variant_bool(self): + # reads 16-bits from memory, anything non-zero is True + for true_value in (1, 32767, 32768, 65535, 65537): + true = POINTER(c_int16)(c_int16(true_value)) + value = cast(true, POINTER(wintypes.VARIANT_BOOL)) + self.assertEqual(repr(value.contents), 'VARIANT_BOOL(True)') + + vb = wintypes.VARIANT_BOOL() + self.assertIs(vb.value, False) + vb.value = True + self.assertIs(vb.value, True) + vb.value = true_value + self.assertIs(vb.value, True) + + for false_value in (0, 65536, 262144, 2**33): + false = POINTER(c_int16)(c_int16(false_value)) + value = cast(false, POINTER(wintypes.VARIANT_BOOL)) + self.assertEqual(repr(value.contents), 'VARIANT_BOOL(False)') + + # allow any bool conversion on assignment to value + for set_value in (65536, 262144, 2**33): + vb = wintypes.VARIANT_BOOL() + vb.value = set_value + self.assertIs(vb.value, True) + + vb = wintypes.VARIANT_BOOL() + vb.value = [2, 3] + self.assertIs(vb.value, True) + vb.value = [] + self.assertIs(vb.value, False) + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/html/parser.py b/Lib/html/parser.py index f8ac82834a..60a322a949 100644 --- a/Lib/html/parser.py +++ b/Lib/html/parser.py @@ -249,6 +249,7 @@ class HTMLParser(_markupbase.ParserBase): if self.strict: self.error("EOF in middle of entity or char ref") else: + k = match.end() if k <= i: k = n i = self.updatepos(i, i + 1) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py index 343b6e4230..27c989cb11 100644 --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -316,11 +316,10 @@ class EditorWindow(object): self.good_load = True is_py_src = self.ispythonsource(filename) self.set_indentation_params(is_py_src) - if is_py_src: - self.color = color = self.ColorDelegator() - per.insertfilter(color) else: io.set_filename(filename) + self.good_load = True + self.ResetColorizer() self.saved_change_hook() self.update_recent_files_list() diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py index 177e49e55e..1805644b6e 100644 --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -858,8 +858,6 @@ class PyShell(OutputWindow): text.bind("<<open-stack-viewer>>", self.open_stack_viewer) text.bind("<<toggle-debugger>>", self.toggle_debugger) text.bind("<<toggle-jit-stack-viewer>>", self.toggle_jit_stack_viewer) - self.color = color = self.ColorDelegator() - self.per.insertfilter(color) if use_subprocess: text.bind("<<view-restart>>", self.view_restart_mark) text.bind("<<restart-shell>>", self.restart_shell) diff --git a/Lib/imp.py b/Lib/imp.py index 6d156d3115..40869f5ac0 100644 --- a/Lib/imp.py +++ b/Lib/imp.py @@ -168,7 +168,7 @@ def load_module(name, file, filename, details): warnings.simplefilter('ignore') if mode and (not mode.startswith(('r', 'U')) or '+' in mode): raise ValueError('invalid file open mode {!r}'.format(mode)) - elif file is None and type_ in {PY_SOURCE, PY_COMPILED, C_EXTENSION}: + elif file is None and type_ in {PY_SOURCE, PY_COMPILED}: msg = 'file object required for import (type code {})'.format(type_) raise ValueError(msg) elif type_ == PY_SOURCE: @@ -176,7 +176,11 @@ def load_module(name, file, filename, details): elif type_ == PY_COMPILED: return load_compiled(name, filename, file) elif type_ == C_EXTENSION and load_dynamic is not None: - return load_dynamic(name, filename, file) + if file is None: + with open(filename, 'rb') as opened_file: + return load_dynamic(name, filename, opened_file) + else: + return load_dynamic(name, filename, file) elif type_ == PKG_DIRECTORY: return load_package(name, filename) elif type_ == C_BUILTIN: diff --git a/Lib/mimetypes.py b/Lib/mimetypes.py index 3f0bd0e719..2872ee4245 100644 --- a/Lib/mimetypes.py +++ b/Lib/mimetypes.py @@ -378,12 +378,14 @@ def _default_mime_types(): '.taz': '.tar.gz', '.tz': '.tar.gz', '.tbz2': '.tar.bz2', + '.txz': '.tar.xz', } encodings_map = { '.gz': 'gzip', '.Z': 'compress', '.bz2': 'bzip2', + '.xz': 'xz', } # Before adding new types, make sure they are either registered with IANA, diff --git a/Lib/multiprocessing/pool.py b/Lib/multiprocessing/pool.py index 7f73b441c2..fc9d90402b 100644 --- a/Lib/multiprocessing/pool.py +++ b/Lib/multiprocessing/pool.py @@ -572,6 +572,8 @@ class ApplyResult(object): self._event.set() del self._cache[self._job] +AsyncResult = ApplyResult # create alias -- see #17805 + # # Class whose instances are returned by `Pool.map_async()` # diff --git a/Lib/tarfile.py b/Lib/tarfile.py index 11b4b68146..6693840c22 100644 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -2398,16 +2398,18 @@ class TarIter: # Fix for SF #1100429: Under rare circumstances it can # happen that getmembers() is called during iteration, # which will cause TarIter to stop prematurely. - if not self.tarfile._loaded: + + if self.index == 0 and self.tarfile.firstmember is not None: + tarinfo = self.tarfile.next() + elif self.index < len(self.tarfile.members): + tarinfo = self.tarfile.members[self.index] + elif not self.tarfile._loaded: tarinfo = self.tarfile.next() if not tarinfo: self.tarfile._loaded = True raise StopIteration else: - try: - tarinfo = self.tarfile.members[self.index] - except IndexError: - raise StopIteration + raise StopIteration self.index += 1 return tarinfo diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py index 8850e8bc13..af27d22b4e 100644 --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -273,6 +273,7 @@ class TestNamedTuple(unittest.TestCase): q = loads(dumps(p, protocol)) self.assertEqual(p, q) self.assertEqual(p._fields, q._fields) + self.assertNotIn(b'OrderedDict', dumps(p, protocol)) def test_copy(self): p = TestNT(x=10, y=20, z=30) diff --git a/Lib/test/test_email/test_utils.py b/Lib/test/test_email/test_utils.py index 622ef3a9c9..e507dd2c03 100644 --- a/Lib/test/test_email/test_utils.py +++ b/Lib/test/test_email/test_utils.py @@ -4,6 +4,7 @@ import test.support import time import unittest import sys +import os.path class DateTimeTests(unittest.TestCase): @@ -123,6 +124,9 @@ class LocaltimeTests(unittest.TestCase): # XXX: Need a more robust test for Olson's tzdata @unittest.skipIf(sys.platform.startswith('win'), "Windows does not use Olson's TZ database") + @unittest.skipUnless(os.path.exists('/usr/share/zoneinfo') or + os.path.exists('/usr/lib/zoneinfo'), + "Can't find the Olson's TZ database") @test.support.run_with_tz('Europe/Kiev') def test_variable_tzname(self): t0 = datetime.datetime(1984, 1, 1, tzinfo=datetime.timezone.utc) diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py index c5d878dca5..b15b6fd4c6 100644 --- a/Lib/test/test_htmlparser.py +++ b/Lib/test/test_htmlparser.py @@ -535,6 +535,20 @@ class HTMLParserTolerantTestCase(HTMLParserStrictTestCase): ] self._run_check(html, expected) + def test_EOF_in_charref(self): + # see #17802 + # This test checks that the UnboundLocalError reported in the issue + # is not raised, however I'm not sure the returned values are correct. + # Maybe HTMLParser should use self.unescape for these + data = [ + ('a&', [('data', 'a&')]), + ('a&b', [('data', 'ab')]), + ('a&b ', [('data', 'a'), ('entityref', 'b'), ('data', ' ')]), + ('a&b;', [('data', 'a'), ('entityref', 'b')]), + ] + for html, expected in data: + self._run_check(html, expected) + def test_unescape_function(self): p = self.get_collector() self.assertEqual(p.unescape('&#bad;'),'&#bad;') diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py index 72ae145f68..fe436f3985 100644 --- a/Lib/test/test_imp.py +++ b/Lib/test/test_imp.py @@ -208,6 +208,8 @@ class ImportTests(unittest.TestCase): self.assertIsNot(orig_getenv, new_os.getenv) @support.cpython_only + @unittest.skipIf(not hasattr(imp, 'load_dynamic'), + 'imp.load_dynamic() required') def test_issue15828_load_extensions(self): # Issue 15828 picked up that the adapter between the old imp API # and importlib couldn't handle C extensions @@ -230,6 +232,21 @@ class ImportTests(unittest.TestCase): self.assertIn(path, err.exception.path) self.assertEqual(name, err.exception.name) + @support.cpython_only + @unittest.skipIf(not hasattr(imp, 'load_dynamic'), + 'imp.load_dynamic() required') + def test_load_module_extension_file_is_None(self): + # When loading an extension module and the file is None, open one + # on the behalf of imp.load_dynamic(). + # Issue #15902 + name = '_heapq' + found = imp.find_module(name) + if found[0] is not None: + found[0].close() + if found[2][2] != imp.C_EXTENSION: + return + imp.load_module(name, None, *found[1:]) + class ReloadTests(unittest.TestCase): diff --git a/Lib/test/test_mimetypes.py b/Lib/test/test_mimetypes.py index 91da28927d..593fdb0a42 100644 --- a/Lib/test/test_mimetypes.py +++ b/Lib/test/test_mimetypes.py @@ -22,6 +22,8 @@ class MimeTypesTestCase(unittest.TestCase): eq(self.db.guess_type("foo.tgz"), ("application/x-tar", "gzip")) eq(self.db.guess_type("foo.tar.gz"), ("application/x-tar", "gzip")) eq(self.db.guess_type("foo.tar.Z"), ("application/x-tar", "compress")) + eq(self.db.guess_type("foo.tar.bz2"), ("application/x-tar", "bzip2")) + eq(self.db.guess_type("foo.tar.xz"), ("application/x-tar", "xz")) def test_data_urls(self): eq = self.assertEqual diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py index 724c530261..0e7d866485 100644 --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -318,7 +318,8 @@ class PosixPathTest(unittest.TestCase): # expanduser should fall back to using the password database del env['HOME'] home = pwd.getpwuid(os.getuid()).pw_dir - self.assertEqual(posixpath.expanduser("~"), home) + # $HOME can end with a trailing /, so strip it (see #17809) + self.assertEqual(posixpath.expanduser("~"), home.rstrip("/")) def test_normpath(self): self.assertEqual(posixpath.normpath(""), ".") diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index b224bf093f..9b98df215d 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -415,6 +415,14 @@ class MiscReadTest(CommonReadTest): finally: support.unlink(empty) + def test_parallel_iteration(self): + # Issue #16601: Restarting iteration over tarfile continued + # from where it left off. + with tarfile.open(self.tarname) as tar: + for m1, m2 in zip(tar, tar): + self.assertEqual(m1.offset, m2.offset) + self.assertEqual(m1.get_info(), m2.get_info()) + class StreamReadTest(CommonReadTest): diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py index 354826c350..cb4cde9bc6 100644 --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -457,6 +457,9 @@ class Win64WinregTests(BaseWinregTests): DeleteKeyEx(HKEY_CURRENT_USER, test_reflect_key_name, KEY_WOW64_32KEY, 0) + def test_exception_numbers(self): + with self.assertRaises(FileNotFoundError) as ctx: + QueryValue(HKEY_CLASSES_ROOT, 'some_value_that_does_not_exist') def test_main(): support.run_unittest(LocalWinregTests, RemoteWinregTests, @@ -117,6 +117,7 @@ Adrian von Bidder David Binger Dominic Binks Philippe Biondi +Michael Birtwell Stuart Bishop Roy Bixler Jonathan Black @@ -204,6 +205,7 @@ Godefroid Chapelle Brad Chapman Greg Chapman Mitch Chapman +Yogesh Chaudhari David Chaum Nicolas Chauvat Jerry Chen @@ -363,6 +365,7 @@ Michael Farrell Troy J. Farrell Mark Favas Boris Feld +Thomas Fenzl Niels Ferguson Sebastian Fernandez Florian Festi @@ -646,6 +649,7 @@ Kim Knapp Lenny Kneler Pat Knight Jeff Knupp +Kubilay Kocak Greg Kochanski Damon Kohler Marko Kohtala @@ -811,6 +815,7 @@ Trent Mick Jason Michalski Franck Michea Tom Middleton +Thomas Miedema Stan Mihai Stefan Mihaila Aristotelis Mikropoulos @@ -1258,6 +1263,7 @@ Nikita Vetoshkin Al Vezza Jacques A. Vidrine John Viega +Dino Viehland Kannan Vijayan Kurt Vile Norman Vine @@ -12,6 +12,11 @@ What's New in Python 3.3.2? Core and Builtins ----------------- +- Issue #17237: Fix crash in the ASCII decoder on m68k. + +- Issue #17408: Avoid using an obsolete instance of the copyreg module when + the interpreter is shutdown and then started again. + - Issue #17863: In the interactive console, don't loop forever if the encoding can't be fetched from stdin. @@ -44,6 +49,30 @@ Core and Builtins Library ------- +- Issue #16601: Restarting iteration over tarfile no more continues from where + it left off. Patch by Michael Birtwell. + +- Issue #17289: The readline module now plays nicer with external modules + or applications changing the rl_completer_word_break_characters global + variable. Initial patch by Bradley Froehle. + +- Issue #12181: select module: Fix struct kevent definition on OpenBSD 64-bit + platforms. Patch by Federico Schwindt. + +- Issue #14173: Avoid crashing when reading a signal handler during + interpreter shutdown. + +- Issue #16316: mimetypes now recognizes the .xz and .txz (.tar.xz) extensions. + +- Issue #15902: Fix imp.load_module() accepting None as a file when loading an + extension module. + +- Issue #17802: Fix an UnboundLocalError in html.parser. Initial tests by + Thomas Barlow. + +- Issue #15535: Fix namedtuple pickles which were picking up the OrderedDict + instead of just the underlying tuple. + - Issue #17192: Restore the patch for Issue #11729 which was ommitted in 3.3.1 when updating the bundled version of libffi used by ctypes. Update many libffi files that were missed in 3.3.1's update to libffi-3.0.13. @@ -120,6 +149,10 @@ IDLE - Issue #17838: Allow sys.stdin to be reassigned. +- Issue #13495: Avoid loading the color delegator twice in IDLE. + +- Issue #17798: Allow IDLE to edit new files when specified on command line. + - Issue #14735: Update IDLE docs to omit "Control-z on Windows". - Issue #17585: Fixed IDLE regression. Now closes when using exit() or quit(). @@ -149,6 +182,12 @@ IDLE Tests ----- +- Issue #17833: Fix test_gdb failures seen on machines where debug symbols + for glibc are available (seen on PPC64 Linux). + +- Issue #7855: Add tests for ctypes/winreg for issues found in IronPython. + Initial patch by Dino Viehland. + - Issue #17712: Fix test_gdb failures on Ubuntu 13.04. - Issue #17835: Fix test_io when the default OS pipe buffer size is larger @@ -186,6 +225,12 @@ Documentation - Issue #6696: add documentation for the Profile objects, and improve profile/cProfile docs. Patch by Tom Pinckney. +Build +----- + +- Issue #17547: In configure, explicitly pass -Wformat for the benefit for GCC + 4.8. + What's New in Python 3.3.1? =========================== diff --git a/Modules/operator.c b/Modules/operator.c index 12fdad54d6..5156b6b32d 100644 --- a/Modules/operator.c +++ b/Modules/operator.c @@ -460,8 +460,8 @@ PyDoc_STRVAR(itemgetter_doc, "itemgetter(item, ...) --> itemgetter object\n\ \n\ Return a callable object that fetches the given item(s) from its operand.\n\ -After, f=itemgetter(2), the call f(r) returns r[2].\n\ -After, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])"); +After f = itemgetter(2), the call f(r) returns r[2].\n\ +After g = itemgetter(2, 5, 3), the call g(r) returns (r[2], r[5], r[3])"); static PyTypeObject itemgetter_type = { PyVarObject_HEAD_INIT(NULL, 0) @@ -712,9 +712,9 @@ PyDoc_STRVAR(attrgetter_doc, "attrgetter(attr, ...) --> attrgetter object\n\ \n\ Return a callable object that fetches the given attribute(s) from its operand.\n\ -After, f=attrgetter('name'), the call f(r) returns r.name.\n\ -After, g=attrgetter('name', 'date'), the call g(r) returns (r.name, r.date).\n\ -After, h=attrgetter('name.first', 'name.last'), the call h(r) returns\n\ +After f = attrgetter('name'), the call f(r) returns r.name.\n\ +After g = attrgetter('name', 'date'), the call g(r) returns (r.name, r.date).\n\ +After h = attrgetter('name.first', 'name.last'), the call h(r) returns\n\ (r.name.first, r.name.last)."); static PyTypeObject attrgetter_type = { @@ -844,8 +844,8 @@ PyDoc_STRVAR(methodcaller_doc, "methodcaller(name, ...) --> methodcaller object\n\ \n\ Return a callable object that calls the given method on its operand.\n\ -After, f = methodcaller('name'), the call f(r) returns r.name().\n\ -After, g = methodcaller('name', 'date', foo=1), the call g(r) returns\n\ +After f = methodcaller('name'), the call f(r) returns r.name().\n\ +After g = methodcaller('name', 'date', foo=1), the call g(r) returns\n\ r.name('date', foo=1)."); static PyTypeObject methodcaller_type = { diff --git a/Modules/readline.c b/Modules/readline.c index a710652e14..71c24231c8 100644 --- a/Modules/readline.c +++ b/Modules/readline.c @@ -70,6 +70,10 @@ on_completion_display_matches_hook(char **matches, int num_matches, int max_length); #endif +/* Memory allocated for rl_completer_word_break_characters + (see issue #17289 for the motivation). */ +static char *completer_word_break_characters; + /* Exported function to send one line to readline's init file parser */ static PyObject * @@ -368,12 +372,20 @@ set_completer_delims(PyObject *self, PyObject *args) { char *break_chars; - if(!PyArg_ParseTuple(args, "s:set_completer_delims", &break_chars)) { + if (!PyArg_ParseTuple(args, "s:set_completer_delims", &break_chars)) { return NULL; } - free((void*)rl_completer_word_break_characters); - rl_completer_word_break_characters = strdup(break_chars); - Py_RETURN_NONE; + /* Keep a reference to the allocated memory in the module state in case + some other module modifies rl_completer_word_break_characters + (see issue #17289). */ + free(completer_word_break_characters); + completer_word_break_characters = strdup(break_chars); + if (completer_word_break_characters) { + rl_completer_word_break_characters = completer_word_break_characters; + Py_RETURN_NONE; + } + else + return PyErr_NoMemory(); } PyDoc_STRVAR(doc_set_completer_delims, @@ -918,7 +930,8 @@ setup_readline(void) /* Set our completion function */ rl_attempted_completion_function = (CPPFunction *)flex_complete; /* Set Python word break characters */ - rl_completer_word_break_characters = + completer_word_break_characters = + rl_completer_word_break_characters = strdup(" \t\n`~!@#$%^&*()-=+[{]}\\|;:'\",<>/?"); /* All nonalphanums except '.' */ @@ -1174,8 +1187,6 @@ PyInit_readline(void) if (m == NULL) return NULL; - - PyOS_ReadlineFunctionPointer = call_readline; setup_readline(); return m; diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c index 06abcf1ac3..2ca2d41a15 100644 --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -1571,6 +1571,23 @@ static PyTypeObject kqueue_queue_Type; # error uintptr_t does not match int, long, or long long! #endif +/* + * kevent is not standard and its members vary across BSDs. + */ +#if !defined(__OpenBSD__) +# define IDENT_TYPE T_UINTPTRT +# define IDENT_CAST Py_intptr_t +# define DATA_TYPE T_INTPTRT +# define DATA_FMT_UNIT INTPTRT_FMT_UNIT +# define IDENT_AsType PyLong_AsUintptr_t +#else +# define IDENT_TYPE T_UINT +# define IDENT_CAST int +# define DATA_TYPE T_INT +# define DATA_FMT_UNIT "i" +# define IDENT_AsType PyLong_AsUnsignedLong +#endif + /* Unfortunately, we can't store python objects in udata, because * kevents in the kernel can be removed without warning, which would * forever lose the refcount on the object stored with it. @@ -1578,11 +1595,11 @@ static PyTypeObject kqueue_queue_Type; #define KQ_OFF(x) offsetof(kqueue_event_Object, x) static struct PyMemberDef kqueue_event_members[] = { - {"ident", T_UINTPTRT, KQ_OFF(e.ident)}, + {"ident", IDENT_TYPE, KQ_OFF(e.ident)}, {"filter", T_SHORT, KQ_OFF(e.filter)}, {"flags", T_USHORT, KQ_OFF(e.flags)}, {"fflags", T_UINT, KQ_OFF(e.fflags)}, - {"data", T_INTPTRT, KQ_OFF(e.data)}, + {"data", DATA_TYPE, KQ_OFF(e.data)}, {"udata", T_UINTPTRT, KQ_OFF(e.udata)}, {NULL} /* Sentinel */ }; @@ -1608,7 +1625,7 @@ kqueue_event_init(kqueue_event_Object *self, PyObject *args, PyObject *kwds) PyObject *pfd; static char *kwlist[] = {"ident", "filter", "flags", "fflags", "data", "udata", NULL}; - static char *fmt = "O|hhi" INTPTRT_FMT_UNIT UINTPTRT_FMT_UNIT ":kevent"; + static char *fmt = "O|hhi" DATA_FMT_UNIT UINTPTRT_FMT_UNIT ":kevent"; EV_SET(&(self->e), 0, EVFILT_READ, EV_ADD, 0, 0, 0); /* defaults */ @@ -1618,8 +1635,12 @@ kqueue_event_init(kqueue_event_Object *self, PyObject *args, PyObject *kwds) return -1; } - if (PyLong_Check(pfd)) { - self->e.ident = PyLong_AsUintptr_t(pfd); + if (PyLong_Check(pfd) +#if IDENT_TYPE == T_UINT + && PyLong_AsUnsignedLong(pfd) <= UINT_MAX +#endif + ) { + self->e.ident = IDENT_AsType(pfd); } else { self->e.ident = PyObject_AsFileDescriptor(pfd); @@ -1647,10 +1668,10 @@ kqueue_event_richcompare(kqueue_event_Object *s, kqueue_event_Object *o, Py_TYPE(s)->tp_name, Py_TYPE(o)->tp_name); return NULL; } - if (((result = s->e.ident - o->e.ident) == 0) && + if (((result = (IDENT_CAST)(s->e.ident - o->e.ident)) == 0) && ((result = s->e.filter - o->e.filter) == 0) && ((result = s->e.flags - o->e.flags) == 0) && - ((result = s->e.fflags - o->e.fflags) == 0) && + ((result = (int)(s->e.fflags - o->e.fflags)) == 0) && ((result = s->e.data - o->e.data) == 0) && ((result = s->e.udata - o->e.udata) == 0) ) { diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index 0cc7237d2e..fbe1bb7a8c 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -344,7 +344,10 @@ signal_signal(PyObject *self, PyObject *args) Handlers[sig_num].tripped = 0; Py_INCREF(obj); Handlers[sig_num].func = obj; - return old_handler; + if (old_handler != NULL) + return old_handler; + else + Py_RETURN_NONE; } PyDoc_STRVAR(signal_doc, @@ -372,8 +375,13 @@ signal_getsignal(PyObject *self, PyObject *args) return NULL; } old_handler = Handlers[sig_num].func; - Py_INCREF(old_handler); - return old_handler; + if (old_handler != NULL) { + Py_INCREF(old_handler); + return old_handler; + } + else { + Py_RETURN_NONE; + } } PyDoc_STRVAR(getsignal_doc, diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 6ece741833..f40dd10a34 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -7,6 +7,10 @@ #include <ctype.h> +/* Cached lookup of the copyreg module, for faster __reduce__ calls */ + +static PyObject *cached_copyreg_module = NULL; + /* Support type attribute cache */ /* The cache can keep references to the names alive for longer than @@ -69,6 +73,15 @@ PyType_ClearCache(void) } void +_PyType_Fini(void) +{ + PyType_ClearCache(); + /* Need to forget our obsolete instance of the copyreg module at + * interpreter shutdown (issue #17408). */ + Py_CLEAR(cached_copyreg_module); +} + +void PyType_Modified(PyTypeObject *type) { /* Invalidate any cached data for the specified type and all @@ -3339,19 +3352,18 @@ static PyObject * import_copyreg(void) { static PyObject *copyreg_str; - static PyObject *mod_copyreg = NULL; if (!copyreg_str) { copyreg_str = PyUnicode_InternFromString("copyreg"); if (copyreg_str == NULL) return NULL; } - if (!mod_copyreg) { - mod_copyreg = PyImport_Import(copyreg_str); + if (!cached_copyreg_module) { + cached_copyreg_module = PyImport_Import(copyreg_str); } - Py_XINCREF(mod_copyreg); - return mod_copyreg; + Py_XINCREF(cached_copyreg_module); + return cached_copyreg_module; } static PyObject * diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index c21e80c99d..8d6cda50ba 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -4661,6 +4661,14 @@ ascii_decode(const char *start, const char *end, Py_UCS1 *dest) const char *p = start; const char *aligned_end = (const char *) _Py_ALIGN_DOWN(end, SIZEOF_LONG); + /* + * Issue #17237: m68k is a bit different from most architectures in + * that objects do not use "natural alignment" - for example, int and + * long are only aligned at 2-byte boundaries. Therefore the assert() + * won't work; also, tests have shown that skipping the "optimised + * version" will even speed up m68k. + */ +#if !defined(__m68k__) #if SIZEOF_LONG <= SIZEOF_VOID_P assert(_Py_IS_ALIGNED(dest, SIZEOF_LONG)); if (_Py_IS_ALIGNED(p, SIZEOF_LONG)) { @@ -4686,6 +4694,7 @@ ascii_decode(const char *start, const char *end, Py_UCS1 *dest) return p - start; } #endif +#endif while (p < end) { /* Fast path, see in STRINGLIB(utf8_decode) in stringlib/codecs.h for an explanation. */ diff --git a/Python/pythonrun.c b/Python/pythonrun.c index ee6071e631..9ef653b57a 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -506,9 +506,6 @@ Py_Finalize(void) /* Disable signal handling */ PyOS_FiniInterrupts(); - /* Clear type lookup cache */ - PyType_ClearCache(); - /* Collect garbage. This may call finalizers; it's nice to call these * before all modules are destroyed. * XXX If a __del__ or weakref callback is triggered here, and tries to @@ -561,6 +558,9 @@ Py_Finalize(void) /* Destroy the database used by _PyImport_{Fixup,Find}Extension */ _PyImport_Fini(); + /* Cleanup typeobject.c's internal caches. */ + _PyType_Fini(); + /* unload faulthandler module */ _PyFaulthandler_Fini(); @@ -581,7 +581,7 @@ Py_Finalize(void) _Py_PrintReferences(stderr); #endif /* Py_TRACE_REFS */ - /* Clear interpreter state */ + /* Clear interpreter state and all thread states. */ PyInterpreterState_Clear(interp); /* Now we decref the exception classes. After this point nothing @@ -597,10 +597,6 @@ Py_Finalize(void) _PyGILState_Fini(); #endif /* WITH_THREAD */ - /* Delete current thread */ - PyThreadState_Swap(NULL); - PyInterpreterState_Delete(interp); - /* Sundry finalizers */ PyMethod_Fini(); PyFrame_Fini(); @@ -618,6 +614,10 @@ Py_Finalize(void) /* Cleanup Unicode implementation */ _PyUnicode_Fini(); + /* Delete current thread. After this, many C API calls become crashy. */ + PyThreadState_Swap(NULL); + PyInterpreterState_Delete(interp); + /* reset file system default encoding */ if (!Py_HasFileSystemDefaultEncoding && Py_FileSystemDefaultEncoding) { free((char*)Py_FileSystemDefaultEncoding); diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py index cab226e5d0..84d4fa31c4 100644 --- a/Tools/gdb/libpython.py +++ b/Tools/gdb/libpython.py @@ -1460,7 +1460,7 @@ class Frame(object): # This assumes the _POSIX_THREADS version of Python/ceval_gil.h: name = self._gdbframe.name() if name: - return name.startswith('pthread_cond_timedwait') + return 'pthread_cond_timedwait' in name def is_gc_collect(self): '''Is this frame "collect" within the garbage-collector?''' @@ -6525,7 +6525,7 @@ then { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether gcc supports ParseTuple __format__" >&5 $as_echo_n "checking whether gcc supports ParseTuple __format__... " >&6; } save_CFLAGS=$CFLAGS - CFLAGS="$CFLAGS -Werror" + CFLAGS="$CFLAGS -Werror -Wformat" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ diff --git a/configure.ac b/configure.ac index 6d54b162c9..9179fe64de 100644 --- a/configure.ac +++ b/configure.ac @@ -1331,7 +1331,7 @@ if test "$GCC" = "yes" then AC_MSG_CHECKING(whether gcc supports ParseTuple __format__) save_CFLAGS=$CFLAGS - CFLAGS="$CFLAGS -Werror" + CFLAGS="$CFLAGS -Werror -Wformat" AC_COMPILE_IFELSE([ AC_LANG_PROGRAM([[void f(char*,...)__attribute((format(PyArg_ParseTuple, 1, 2)));]], [[]]) ],[ diff --git a/pyconfig.h.in b/pyconfig.h.in index 10e2c91dc5..cbbc42d99f 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1187,9 +1187,6 @@ /* Define if setpgrp() must be called as setpgrp(0, 0). */ #undef SETPGRP_HAVE_ARG -/* Define this to be extension of shared libraries (including the dot!). */ -#undef SHLIB_EXT - /* Define if i>>j for signed int i does not extend the sign bit when i < 0 */ #undef SIGNED_RIGHT_SHIFT_ZERO_FILLS |