summaryrefslogtreecommitdiff
path: root/Lib
Commit message (Collapse)AuthorAgeFilesLines
...
* bpo-32689: Updates shutil.move to allow for Path objects to be used as ↵Maxwell A McKinnon2019-09-302-2/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | source arg (GH-15326) Important work originally done by @emilyemorehouse two years ago and nearly ready to go in. This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwise wonderful pathlib and PEP519. https://stackoverflow.com/questions/33625931/copy-file-with-pathlib-in-python. This adds the outstanding test request from that PR @vstinner (https://github.com/python/cpython/pull/5393). Test fails without the change, passes with it, along with every other test in test_shutil. Some variants were experimented with to make the one line change and the most performant one was picked. # Added Test for PathLike directory destination, the current fail case ``` Lib/test/test_shutil.py::TestMove::test_move_file_pathlike FAILED [100%] ============================================================== FAILURES =============================================================== __________________________________________________ TestMove.test_move_file_pathlike ___________________________________________________ self = <test.test_shutil.TestMove testMethod=test_move_file_pathlike> def test_move_file_pathlike(self): # Move a file to another location on the same filesystem. src = pathlib.Path(self.src_file) > self._check_move_file(src, self.dst_dir, self.dst_file) Lib/test/test_shutil.py:1563: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Lib/test/test_shutil.py:1545: in _check_move_file shutil.move(src, dst) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:562: in move real_dst = os.path.join(dst, _basename(src)) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ path = PosixPath('/var/folders/r2/psq74t5x3nbfzlph8bh2pvdw0000gn/T/tmp9ie0wh9_/foo') def _basename(path): # A basename() variant which first strips the trailing slash, if present. # Thus we always get the last component of the path, even for directories. sep = os.path.sep + (os.path.altsep or '') > return os.path.basename(path.rstrip(sep)) E AttributeError: 'PosixPath' object has no attribute 'rstrip' /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:526: AttributeError ============================================== 1 failed, 102 deselected in 0.30 seconds =============================================== ``` After change: ``` ========================================================= test session starts ========================================================= platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7 cachedir: .pytest_cache rootdir: /Users/maxwellmckinnon/dev/cpython plugins: cov-2.7.1, mock-1.10.4 collected 103 items / 102 deselected / 1 selected Lib/test/test_shutil.py::TestMove::test_move_file_pathlike PASSED [100%] ============================================== 1 passed, 102 deselected in 0.06 seconds =============================================== ``` Running all the tests in test_shutil.py ``` ╰─ pytest Lib/test/test_shutil.py -v ========================================================= test session starts ========================================================= platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7 cachedir: .pytest_cache rootdir: /Users/maxwellmckinnon/dev/cpython plugins: cov-2.7.1, mock-1.10.4 collected 103 items Lib/test/test_shutil.py::TestShutil::test_chown PASSED [ 0%] Lib/test/test_shutil.py::TestShutil::test_copy PASSED [ 1%] ... Lib/test/test_shutil.py::TermsizeTests::test_stty_match SKIPPED [ 99%] Lib/test/test_shutil.py::PublicAPITests::test_module_all_attribute PASSED [100%] ================================================ 96 passed, 7 skipped in 1.25 seconds ================================================= ``` # Performance Considerations Is it considered poor form to get rid of _basename altogether and make use of pathlib in the move function? I'm not sure if the idea is for all these modules to strictly avoid circular dependencies. They are already using os.path which is just as much a citizen in 3.8 as pathlib right? e.g. `real_dst = os.path.join(dst, _basename(src))` becomes `real_dst = Path(dst) / Path(src).name` I've looked around and familiarized myself, and I now think importing pathlib here is fine. My only remaining concern is that of performance. Here's the performance difference for this step. ``` In [46]: %timeit real_dst = os.path.join("a/b/c", _basename('b/')) 2.71 µs ± 62.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) In [47]: %timeit real_dst = Path("a/b/c") / Path('b/').name 12.4 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) ``` Is 10us significant or insignificant compared to the least expensive operation this function will do? I don't know. Let's find out. ``` In [55]: %timeit os.rename('/tmp/a/a.txt', '/tmp/a/b.txt'); os.rename('/tmp/a/b.txt', '/tmp/a/a.txt') 124 µs ± 2.18 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each) ``` 62us to rename. 10us seems significant enough that we wouldn't want to favor the Path sugar suggestion. 16% speed decrease from adding the 10us. What do people think? I was hoping to get to use pathlib.Path here, but I suspect for this low level move, it should be as fast as possible, and 16% is not worth one line of sugary code to me. https://bugs.python.org/issue32689 Automerge-Triggered-By: @gvanrossum
* Fix and improve `asyncio.run()` docs (GH-16403)Kyle Stanley2019-09-301-5/+1
|
* bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel ↵Yury Selivanov2019-09-291-62/+16
| | | | (#7468)
* bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)Yury Selivanov2019-09-298-2221/+304
| | | See https://bugs.python.org/issue38242 for more details
* bpo-38163: Child mocks detect their type as sync or async (GH-16471)Lisa Roach2019-09-292-27/+45
|
* bpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676)Giampaolo Rodola2019-09-301-6/+85
|
* bpo-38161: Removes _AwaitEvent from AsyncMock. (GH-16443)Lisa Roach2019-09-292-39/+1
|
* bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)Victor Stinner2019-09-301-2/+26
| | | | | | | | | Fix warnings options priority: PyConfig.warnoptions has the highest priority, as stated in the PEP 587. * Document options order in PyConfig.warnoptions documentation. * Make PyWideStringList_INIT macro private: replace "Py" prefix with "_Py". * test_embed: add test_init_warnoptions().
* bpo-38019: correctly handle pause/resume reading of closed asyncio unix pipe ↵Andrew Svetlov2019-09-292-0/+38
| | | | (GH-16472)
* bpo-38108: Makes mock objects inherit from Base (GH-16060)Lisa Roach2019-09-283-57/+56
|
* bpo-38115: Deal with invalid bytecode offsets in lnotab (GH-16079)T. Wouters2019-09-282-1/+73
| | | | | | | | | | Document that lnotab can contain invalid bytecode offsets (because of terrible reasons that are difficult to fix). Make dis.findlinestarts() ignore invalid offsets in lnotab. All other uses of lnotab in CPython (various reimplementations of addr2line or line2addr in Python, C and gdb) already ignore this, because they take an address to look for, instead. Add tests for the result of dis.findlinestarts() on wacky constructs in test_peepholer.py, because it's the easiest place to add them.
* bpo-38216, bpo-36274: Allow subclasses to separately override validation and ↵Jason R. Coombs2019-09-282-9/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | encoding behavior (GH-16448) * bpo-38216: Allow bypassing input validation * bpo-36274: Also allow the URL encoding to be overridden. * bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL. * Call with skip_host to avoid tripping on the host checking in the URL. * Remove obsolete comment. * Make _prepare_path_encoding its own attr. This makes overriding just that simpler. Also, don't use the := operator to make backporting easier. * Add a news entry. * _prepare_path_encoding -> _encode_prepared_path() * Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
* bpo-38243, xmlrpc.server: Escape the server_title (GH-16373)Dong-hee Na2019-09-272-1/+18
| | | | Escape the server title of xmlrpc.server.DocXMLRPCServer when rendering the document page as HTML.
* bpo-32820: Simplify __format__ implementation for ipaddress. (GH-16378)Serhiy Storchaka2019-09-271-40/+19
| | | | Also cache the compiled RE for parsing the format specifier.
* bpo-38187: Fix a refleak in Tools/c-analyzer. (gh-16304)Eric Snow2019-09-277-40/+100
| | | | | The "Slot" helper (descriptor) is leaking references due to its caching mechanism. The change includes a partial fix to Slot, but also adds Variable.storage to replace the problematic use of Slot. https://bugs.python.org/issue38187
* bpo-38270: More fixes for strict crypto policy (GH-16418)Christian Heimes2019-09-273-11/+35
| | | | | | | | | | | | | | | test_hmac and test_hashlib test built-in hashing implementations and OpenSSL-based hashing implementations. Add more checks to skip OpenSSL implementations when a strict crypto policy is active. Use EVP_DigestInit_ex() instead of EVP_DigestInit() to initialize the EVP context. The EVP_DigestInit() function clears alls flags and breaks usedforsecurity flag again. Signed-off-by: Christian Heimes <christian@python.org> https://bugs.python.org/issue38270
* cleanup ababstractproperty in typing.py (GH-16432)HongWeipeng2019-09-271-8/+15
|
* bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672)Michael Felt2019-09-262-93/+173
|
* bpo-38275: Fix test_ssl issue caused by GH-16386 (#16428)Christian Heimes2019-09-261-0/+2
| | | | | | Check presence of SSLContext.minimum_version to make tests pass with old versions of OpenSSL. Signed-off-by: Christian Heimes <christian@python.org>
* bpo-38275: Skip ssl tests for disabled versions (GH-16386)Christian Heimes2019-09-261-51/+143
| | | | | | | | | | | test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto policy and run-time settings are recognized and tests for disabled versions are skipped. Signed-off-by: Christian Heimes <christian@python.org> https://bugs.python.org/issue38275
* bpo-38239: Fix test_gdb for Link Time Optimization (LTO) (GH-16422)Victor Stinner2019-09-261-2/+9
|
* bpo-38234: Py_Initialize() sets global path configuration (GH-16421)Victor Stinner2019-09-261-0/+38
| | | | | | * Py_InitializeFromConfig() now writes PyConfig path configuration to the global path configuration (_Py_path_config). * Add test_embed.test_get_pathconfig(). * Fix typo in _PyWideStringList_Join().
* compileall tests: Use shorter name for long_path test (GH-16419)Petr Viktorin2019-09-261-2/+6
| | | | Apparently, the path needs to be limited to 260 characters on (some versions of) Windows.
* bpo-38223: Reorganize test_shutil. (GH-16281)Serhiy Storchaka2019-09-261-487/+478
| | | | | | * Group tests for specific functions and groups of related functions into separate classes. * Clean up creating and cleaning up temporary directories. * Simplify and make more robust monkey patching of shutil.open.
* bpo-38112: compileall: Skip long path path on Windows if the path can't be ↵Petr Viktorin2019-09-261-12/+51
| | | | | | | | | | | | created (GH-16414) This avoids the buildbot failure on Windows: ``` FileNotFoundError: [WinError 206] The filename or extension is too long: 'd:\\temp\\tmp5r3z438t\\long\\1\\2\\3\\4\\5\\6\\7\\8\\9\\10\\11\\12\\13\\14\\15\\16\\17\\18\\19\\20\\21\\22\\23\\24\\25\\26\\27\\28\\29\\30\\31\\32\\33\\34\\35\\36\\37\\38\\39\\40\\41\\42\\43\\44\\45\\46\\47\\48\\49\\50\\51\\52\\53\\54\\55\\56\\57\\58\\59\\60\\61\\62\\63\\64\\65\\66\\67\\68\\69\\70\\71\\72\\73\\74\\75\\76\\77\\78' ``` Creates a path that's long but avoids OS restrictions. https://bugs.python.org/issue38112
* bpo-38112: Compileall improvements (GH-16012)Lumír 'Frenzy' Balhar2019-09-262-38/+345
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * Raise the limit of maximum path depth to actual recursion limit * Add posibilities to adjust a path compiled in .pyc file. Now, you can: - Strip a part of path from a beggining of path into compiled file example "-s /test /test/build/real/test.py" → "build/real/test.py" - Append some new path to a beggining of path into compiled file example "-p /boo real/test.py" → "/boo/real/test.py" You can also use both options in the same time. In that case, striping is done before appending. * Add a possibility to specify multiple optimization levels Each optimization level then leads to separated compiled file. Use `action='append'` instead of `nargs='+'` for the -o option. Instead of `-o 0 1 2`, specify `-o 0 -o 1 -o 2`. It's more to type, but much more explicit. * Add a symlinks limitation feature This feature allows us to limit byte-compilation of symbolic links if they are pointing outside specified dir (build root for example).
* bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)Victor Stinner2019-09-261-7/+6
| | | | | | | | | Explicitly preinitializes with a Python preconfiguration to avoid Py_SetPath() implicit preinitialization with a compat preconfiguration. Fix also test_init_setpath() and test_init_setpythonhome() on macOS: use self.test_exe as the executable (and base_executable), rather than shutil.which('python3').
* bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)Victor Stinner2019-09-261-8/+51
| | | | | | | | | | | | * Add test_embed.test_init_setpath_config(): test Py_SetPath() with PyConfig. * test_init_setpath() and test_init_setpythonhome() no longer call Py_SetProgramName(), but use the default program name. * _PyPathConfig: isolated, site_import and base_executable fields are now only available on Windows. * If executable is set explicitly in the configuration, ignore calculated base_executable: _PyConfig_InitPathConfig() copies executable to base_executable. * Complete path config documentation.
* bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)Christian Heimes2019-09-251-10/+0
| | | | * Updated _hashopenssl.c to be PEP 384 compliant * Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
* bpo-22273: Removed temporary test skipping on PPC platforms. (GH-16399)Vinay Sajip2019-09-251-2/+3
|
* bpo-38271: encrypt private key test files with AES256 (GH-16385)Christian Heimes2019-09-253-86/+87
| | | | | | | | | | | | | | | | The private keys for test_ssl were encrypted with 3DES in traditional PKCS#5 format. 3DES and the digest algorithm of PKCS#5 are blocked by some strict crypto policies. Use PKCS#8 format with AES256 encryption instead. Signed-off-by: Christian Heimes <christian@python.org> https://bugs.python.org/issue38271 Automerge-Triggered-By: @tiran
* bpo-38005: Remove support of string argument in InterpreterID(). (GH-16227)Serhiy Storchaka2019-09-251-9/+6
| | | Make negative interpreter id to raise ValueError instead of RuntimeError.
* bpo-38234: Fix test_embed pathconfig tests (GH-16390)Victor Stinner2019-09-251-0/+5
| | | | | bpo-38234: On macOS and FreeBSD, the temporary directory can be symbolic link. For example, /tmp can be a symbolic link to /var/tmp. Call realpath() to resolve all symbolic links.
* bpo-38270: Check for hash digest algorithms and avoid MD5 (GH-16382)Christian Heimes2019-09-257-45/+115
| | | | | | | | | | | Make it easier to run and test Python on systems with restrict crypto policies: * add requires_hashdigest to test.support to check if a hash digest algorithm is available and working * avoid MD5 in test_hmac * replace MD5 with SHA256 in test_tarfile * mark network tests that require MD5 for MD5-based digest auth or CRAM-MD5 https://bugs.python.org/issue38270
* bpo-22273: Re-enabled ctypes test on ARM machines. (GH-16388)Vinay Sajip2019-09-251-1/+1
|
* bpo-36046: posix_spawn() doesn't support uid/gid (GH-16384)Victor Stinner2019-09-252-34/+42
| | | | | * subprocess.Popen now longer uses posix_spawn() if uid, gid or gids are set. * test_subprocess: add "nobody" and "nfsnobody" group names for test_group(). * test_subprocess: test_user() and test_group() are now also tested with close_fds=False.
* bpo-37064: Add option -a to pathfix.py tool (GH-15717)PatrikKopkan2019-09-251-17/+56
| | | Add option -a to Tools/Scripts/pathfix.py script: add flags.
* bpo-38248: Fix inconsistent immediate asyncio.Task cancellation (GH-16330)Yury Selivanov2019-09-252-1/+5
|
* Changed conditions for ctypes array-in-struct handling. (GH-16381)Vinay Sajip2019-09-251-1/+1
|
* bpo-38260: Add Docs on asyncio.run (GH-16337)Emmanuel Arias2019-09-251-0/+4
| | | | | | | | | | | | | Add docs about return and raise exception on asyncio.run https://bugs.python.org/issue38260 Automerge-Triggered-By: @asvetlov
* bpo-22273: Disabled tests while investigating buildbot failures on ↵Vinay Sajip2019-09-251-0/+5
| | | | ARM7L/PPC64. (GH-16377)
* bpo-22273: Update ctypes to correctly handle arrays in small structur… ↵Vinay Sajip2019-09-251-0/+41
| | | | (GH-15839)
* bpo-38234: Cleanup getpath.c (GH-16367)Victor Stinner2019-09-251-0/+7
| | | | | | | | | * search_for_prefix() directly calls reduce() if found is greater than 0. * Add calculate_pybuilddir() subfunction. * search_for_prefix(): add path string buffer for readability. * Fix some error handling code paths: release resources on error. * calculate_read_pyenv(): rename tmpbuffer to filename. * test.pythoninfo now also logs windows.dll_path
* bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt (GH-16366)Victor Stinner2019-09-252-16/+166
| | | | | | | Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed to test pyvenv.cfg and pybuilddir.txt configuration files. Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8 encoding, not ASCII.
* bpo-36871: Avoid duplicated 'Actual:' in assertion message (GH-16361)Samuel Freilich2019-09-243-18/+32
| | | | | Fixes an issue caught after merge of PR 16005. Tightened test assertions to check the entire assertion message.
* bpo-36871: Handle spec errors in assert_has_calls (GH-16005)Samuel Freilich2019-09-243-5/+61
| | | | | | | | | | | | The fix in PR 13261 handled the underlying issue about the spec for specific methods not being applied correctly, but it didn't fix the issue that was causing the misleading error message. The code currently grabs a list of responses from _call_matcher (which may include exceptions). But it doesn't reach inside the list when checking if the result is an exception. This results in a misleading error message when one of the provided calls does not match the spec. https://bugs.python.org/issue36871 Automerge-Triggered-By: @gpshead
* bpo-38234: Add tests for Python init path config (GH-16358)Victor Stinner2019-09-241-11/+78
|
* bpo-37123: multiprocessing test_mymanager() accepts SIGTERM (GH-16349)Victor Stinner2019-09-241-5/+6
| | | | | | | | Multiprocessing test test_mymanager() now also expects -SIGTERM, not only exitcode 0. bpo-30356: BaseManager._finalize_manager() sends SIGTERM to the manager process if it takes longer than 1 second to stop, which happens on slow buildbots.
* bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)Victor Stinner2019-09-241-2/+1
| | | | Multiprocessing tests: increase test_queue_feeder_donot_stop_onexc() timeout from 1 to 60 seconds.
* bpo-38136: Updates await_count and call_count to be different things (GH-16192)Lisa Roach2019-09-233-18/+191
|
View Lorry Controller Status