summaryrefslogtreecommitdiff
path: root/Python/pystate.c
Commit message (Collapse)AuthorAgeFilesLines
* bpo-35081: Rename internal headers (GH-10275)Victor Stinner2018-11-121-2/+2
| | | | | | | | | | | | | | Rename Include/internal/ headers: * pycore_hash.h -> pycore_pyhash.h * pycore_lifecycle.h -> pycore_pylifecycle.h * pycore_mem.h -> pycore_pymem.h * pycore_state.h -> pycore_pystate.h Add missing headers to Makefile.pre.in and PCbuild: * pycore_condvar.h. * pycore_hamt.h * pycore_pyhash.h
* bpo-35081: Add _PyThreadState_GET() internal macro (GH-10266)Victor Stinner2018-11-011-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | If Py_BUILD_CORE is defined, the PyThreadState_GET() macro access _PyRuntime which comes from the internal pycore_state.h header. Public headers must not require internal headers. Move PyThreadState_GET() and _PyInterpreterState_GET_UNSAFE() from Include/pystate.h to Include/internal/pycore_state.h, and rename PyThreadState_GET() to _PyThreadState_GET() there. The PyThreadState_GET() macro of pystate.h is now redefined when pycore_state.h is included, to use the fast _PyThreadState_GET(). Changes: * Add _PyThreadState_GET() macro * Replace "PyThreadState_GET()->interp" with _PyInterpreterState_GET_UNSAFE() * Replace PyThreadState_GET() with _PyThreadState_GET() in internal C files (compiled with Py_BUILD_CORE defined), but keep PyThreadState_GET() in the public header files. * _testcapimodule.c: replace PyThreadState_GET() with PyThreadState_Get(); the module is not compiled with Py_BUILD_CORE defined. * pycore_state.h now requires Py_BUILD_CORE to be defined.
* bpo-35081: Add pycore_ prefix to internal header files (GH-10263)Victor Stinner2018-11-011-2/+2
| | | | | | | | | | | | | | | | | | | | * Rename Include/internal/ header files: * pyatomic.h -> pycore_atomic.h * ceval.h -> pycore_ceval.h * condvar.h -> pycore_condvar.h * context.h -> pycore_context.h * pygetopt.h -> pycore_getopt.h * gil.h -> pycore_gil.h * hamt.h -> pycore_hamt.h * hash.h -> pycore_hash.h * mem.h -> pycore_mem.h * pystate.h -> pycore_state.h * warnings.h -> pycore_warnings.h * PCbuild project, Makefile.pre.in, Modules/Setup: add the Include/internal/ directory to the search paths of header files. * Update includes. For example, replace #include "internal/mem.h" with #include "pycore_mem.h".
* bpo-35081: Move Py_BUILD_CORE code to internal/mem.h (GH-10249)Victor Stinner2018-10-311-0/+1
| | | | | * Add #include "internal/mem.h" to C files using _PyMem_SetDefaultAllocator(). * Include/internal/mem.h now requires Py_BUILD_CORE to be defined.
* bpo-35081: Cleanup pystate.c and pystate.h (GH-10240)Victor Stinner2018-10-301-23/+20
| | | | | | | | | | | | * Remove _PyThreadState_Current * Replace GET_TSTATE() with PyThreadState_GET() * Replace GET_INTERP_STATE() with _PyInterpreterState_GET_UNSAFE() * Replace direct access to _PyThreadState_Current with PyThreadState_GET() * Replace _PyThreadState_Current with _PyRuntime.gilstate.tstate_current * Rename SET_TSTATE() to _PyThreadState_SET(), name more consistent with _PyThreadState_GET() * Update outdated comments
* bpo-34651: Only allow the main interpreter to fork. (gh-9279)Eric Snow2018-09-141-0/+38
| | | | | When os.fork() is called (on platforms that support it) all threads but the current one are destroyed in the child process. Consequently we must ensure that all but the associated interpreter are likewise destroyed. The main interpreter is critical for runtime operation, so we must ensure that fork only happens in the main interpreter. https://bugs.python.org/issue34651
* bpo-34408: Prevent a null pointer dereference and resource leakage in ↵Pablo Galindo2018-08-311-6/+10
| | | | | | | `PyInterpreterState_New()` (GH-8767) * A pointer in `PyInterpreterState_New()` could have been `NULL` when being dereferenced. * Memory was leaked in `PyInterpreterState_New()` when taking some error-handling code path.
* bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)Victor Stinner2018-08-031-4/+18
| | | | sys_setcheckinterval() now uses a local variable to parse arguments, before writing into interp->check_interval.
* bpo-34170: Add _PyCoreConfig.bytes_warning (GH-8447)Victor Stinner2018-07-251-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add more fields to _PyCoreConfig: * _check_hash_pycs_mode * bytes_warning * debug * inspect * interactive * legacy_windows_fs_encoding * legacy_windows_stdio * optimization_level * quiet * unbuffered_stdio * user_site_directory * verbose * write_bytecode Changes: * Remove pymain_get_global_config() and pymain_set_global_config() which became useless. These functions have been replaced by _PyCoreConfig_GetGlobalConfig() and _PyCoreConfig_SetGlobalConfig(). * sys.flags.dont_write_bytecode value is now restricted to 1 even if -B option is specified multiple times on the command line. * PyThreadState_Clear() now uses the config from the current interpreter rather than using global Py_VerboseFlag
* bpo-33615: Re-enable a subinterpreter test. (gh-7251)Eric Snow2018-06-011-18/+37
| | | For bpo-32604 I added extra subinterpreter-related tests (see #6914), which caused a few buildbots to crash. This patch fixes the crash by ensuring that refcounts in channels are handled properly.
* bpo-33612: Remove PyThreadState_Clear() assertion (#7069)Victor Stinner2018-05-231-1/+0
| | | | | bpo-25612, bpo-33612: Remove an assertion from PyThreadState_Clear() which failed at Python shutdown or on fork if a thread was running a generator.
* bpo-32604: Improve subinterpreter tests. (#6914)Eric Snow2018-05-161-3/+77
| | | Add more tests for subinterpreters. This patch also fixes a few small defects in the channel implementation.
* bpo-33005: Fix _PyGILState_Reinit() (#6001)Victor Stinner2018-03-061-4/+23
| | | | | | | | Fix a crash on fork when using a custom memory allocator (ex: using PYTHONMALLOC env var). _PyGILState_Reinit() and _PyInterpreterState_Enable() now use the default RAW memory allocator to allocate a new interpreters mutex on fork.
* bpo-32604: Swap threads only if the interpreter is different. (gh-5778)Eric Snow2018-02-201-6/+14
| | | The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
* bpo-32604: Clean up created subinterpreters before runtime finalization. ↵Eric Snow2018-02-161-1/+57
| | | | | | (gh-5709)
* bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (#5507)Eric Snow2018-02-021-0/+1
|
* bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. ↵Eric Snow2018-01-291-1/+274
| | | | | (gh-1748) The module is primarily intended for internal use in the test suite. Building the module under Windows will come in a follow-up PR.
* bpo-32436: Implement PEP 567 (#5027)Yury Selivanov2018-01-221-0/+9
|
* bpo-32591: Add native coroutine origin tracking (#5250)Nathaniel J. Smith2018-01-211-0/+2
| | | | | | * Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth * Use coroutine origin information in the unawaited coroutine warning * Stop using set_coroutine_wrapper in asyncio debug mode * In BaseEventLoop.set_debug, enable debugging in the correct thread
* bpo-31901: atexit callbacks should be run at subinterpreter shutdown (#4611)Marcel Plch2017-12-201-0/+2
| | | | Change atexit behavior and PEP-489 multiphase init support.
* bpo-32030: Add _PyCoreConfig_Copy() (#4874)Victor Stinner2017-12-151-0/+2
| | | | | | | | | | Each interpreter now has its core_config and main_config copy: * Add _PyCoreConfig_Copy() and _PyMainInterpreterConfig_Copy() * Move _PyCoreConfig_Read(), _PyCoreConfig_Clear() and _PyMainInterpreterConfig_Clear() from Python/pylifecycle.c to Modules/main.c * Fix _Py_InitializeEx_Private(): call _PyCoreConfig_ReadEnv() before _Py_InitializeCore()
* bpo-20891: Fix PyGILState_Ensure() (#4650)Victor Stinner2017-11-301-6/+18
| | | | | | | When PyGILState_Ensure() is called in a non-Python thread before PyEval_InitThreads(), only call PyEval_InitThreads() after calling PyThreadState_New() to fix a crash. Add an unit test in test_embed.
* bpo-32030: Rework memory allocators (#4625)Victor Stinner2017-11-291-7/+19
| | | | | | | | | | | | | | | | | | | | * Fix _PyMem_SetupAllocators("debug"): always restore allocators to the defaults, rather than only caling _PyMem_SetupDebugHooks(). * Add _PyMem_SetDefaultAllocator() helper to set the "default" allocator. * Add _PyMem_GetAllocatorsName(): get the name of the allocators * main() now uses debug hooks on memory allocators if Py_DEBUG is defined, rather than calling directly malloc() * Document default memory allocators in C API documentation * _Py_InitializeCore() now fails with a fatal user error if PYTHONMALLOC value is an unknown memory allocator, instead of failing with a fatal internal error. * Add new tests on the PYTHONMALLOC environment variable * Add support.with_pymalloc() * Add the _testcapi.WITH_PYMALLOC constant and expose it as support.with_pymalloc(). * sysconfig.get_config_var('WITH_PYMALLOC') doesn't work on Windows, so replace it with support.with_pymalloc(). * pythoninfo: add _testcapi collector for pymem
* bpo-32096: Remove obj and mem from _PyRuntime (#4532)Victor Stinner2017-11-241-2/+0
| | | | | | | | | | | | | | | | | bpo-32096, bpo-30860: Partially revert the commit 2ebc5ce42a8a9e047e790aefbf9a94811569b2b6: * Move structures back from Include/internal/mem.h to Objects/obmalloc.c * Remove _PyObject_Initialize() and _PyMem_Initialize() * Remove Include/internal/pymalloc.h * Add test_capi.test_pre_initialization_api(): Make sure that it's possible to call Py_DecodeLocale(), and then call Py_SetProgramName() with the decoded string, before Py_Initialize(). PyMem_RawMalloc() and Py_DecodeLocale() can be called again before _PyRuntimeState_Init(). Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
* bpo-32030: Add _PyCoreConfig.module_search_path_env (#4504)Victor Stinner2017-11-231-40/+45
| | | | | | | | | | | Changes: * Py_Main() initializes _PyCoreConfig.module_search_path_env from the PYTHONPATH environment variable. * PyInterpreterState_New() now initializes core_config and config fields * Compute sys.path a little bit ealier in _Py_InitializeMainInterpreter() and new_interpreter() * Add _Py_GetPathWithConfig() private function.
* bpo-32043: New "developer mode": "-X dev" option (#4413)Victor Stinner2017-11-161-0/+8
| | | | | | | | | | | | | Add a new "developer mode": new "-X dev" command line option to enable debug checks at runtime. Changes: * Add unit tests for -X dev * test_cmd_line: replace test.support with support. * Fix _PyRuntimeState_Fini(): Use the same memory allocator than _PyRuntimeState_Init(). * Fix _PyMem_GetDefaultRawAllocator()
* bpo-32030: Enhance Py_Main() (#4412)Victor Stinner2017-11-151-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Parse more env vars in Py_Main(): * Add more options to _PyCoreConfig: * faulthandler * tracemalloc * importtime * Move code to parse environment variables from _Py_InitializeCore() to Py_Main(). This change fixes a regression from Python 3.6: PYTHONUNBUFFERED is now read before calling pymain_init_stdio(). * _PyFaulthandler_Init() and _PyTraceMalloc_Init() now take an argument to decide if the module has to be enabled at startup. * tracemalloc_start() is now responsible to check the maximum number of frames. Other changes: * Cleanup Py_Main(): * Rename some pymain_xxx() subfunctions * Add pymain_run_python() subfunction * Cleanup Py_NewInterpreter() * _PyInterpreterState_Enable() now reports failure * init_hash_secret() now considers pyurandom() failure as an "user error": don't fail with abort(). * pymain_optlist_append() and pymain_strdup() now sets err on memory allocation failure.
* bpo-32030: Split Py_Main() into subfunctions (#4399)Victor Stinner2017-11-151-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Don't use "Python runtime" anymore to parse command line options or to get environment variables: pymain_init() is now a strict separation. * Use an error message rather than "crashing" directly with Py_FatalError(). Limit the number of calls to Py_FatalError(). It prepares the code to handle errors more nicely later. * Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now only added to the sys module once Python core is properly initialized. * _PyMain is now the well identified owner of some important strings like: warnings options, XOptions, and the "program name". The program name string is now properly freed at exit. pymain_free() is now responsible to free the "command" string. * Rename most methods in Modules/main.c to use a "pymain_" prefix to avoid conflits and ease debug. * Replace _Py_CommandLineDetails_INIT with memset(0) * Reorder a lot of code to fix the initialization ordering. For example, initializing standard streams now comes before parsing PYTHONWARNINGS. * Py_Main() now handles errors when adding warnings options and XOptions. * Add _PyMem_GetDefaultRawAllocator() private function. * Cleanup _PyMem_Initialize(): remove useless global constants: move them into _PyMem_Initialize(). * Call _PyRuntime_Initialize() as soon as possible: _PyRuntime_Initialize() now returns an error message on failure. * Add _PyInitError structure and following macros: * _Py_INIT_OK() * _Py_INIT_ERR(msg) * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case * _Py_INIT_FAILED(err)
* Fix trailing whitespaces in C files. (#4130)Serhiy Storchaka2017-10-261-1/+1
|
* bpo-31857: Make the behavior of USE_STACKCHECK deterministic (#4098)pdox2017-10-251-0/+1
|
* Move exc state to generator. Fixes bpo-25612 (#1773)Mark Shannon2017-10-221-6/+15
| | | Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
* bpo-25658: Implement PEP 539 for Thread Specific Storage (TSS) API (GH-1362)Masayuki Yamamoto2017-10-061-33/+38
| | | | | | | | | See PEP 539 for details. Highlights of changes: - Add Thread Specific Storage (TSS) API - Document the Thread Local Storage (TLS) API as deprecated - Update code that used TLS API to use TSS API
* bpo-30860: Fix a refleak. (#3567)Eric Snow2017-09-141-2/+0
| | | | | Resolves bpo-31420. (This was accidentally reverted when in #3565.)
* bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)Eric Snow2017-09-131-0/+4
| | | PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
* bpo-30860: Fix a refleak. (#3506)Eric Snow2017-09-111-2/+0
| | | | | | | | | | | | | | * Drop warnoptions from PyInterpreterState. * Drop xoptions from PyInterpreterState. * Don't set warnoptions and _xoptions again. * Decref after adding to sys.__dict__. * Drop an unused macro. * Check sys.xoptions *before* we delete it.
* bpo-30860: Consolidate stateful runtime globals. (#3397)Eric Snow2017-09-071-81/+100
| | | | | | | * group the (stateful) runtime globals into various topical structs * consolidate the topical structs under a single top-level _PyRuntimeState struct * add a check-c-globals.py script that helps identify runtime globals Other globals are excluded (see globals.txt and check-c-globals.py).
* bpo-31370: Remove support for threads-less builds (#3385)Antoine Pitrou2017-09-071-29/+1
| | | | | | * Remove Setup.config * Always define WITH_THREAD for compatibility.
* Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)Eric Snow2017-09-051-107/+88
| | | Windows buildbots started failing due to include-related errors.
* correct initialization code (#3376)Benjamin Peterson2017-09-051-2/+1
| | | Explicitly initialize struct members rather than relying on compiler extensions.
* bpo-30860: Consolidate stateful runtime globals. (#2594)Eric Snow2017-09-051-88/+108
| | | | | | | | | * group the (stateful) runtime globals into various topical structs * consolidate the topical structs under a single top-level _PyRuntimeState struct * add a check-c-globals.py script that helps identify runtime globals Other globals are excluded (see globals.txt and check-c-globals.py).
* bpo-28411: Remove "modules" field from Py_InterpreterState. (#1638)Eric Snow2017-09-041-2/+0
| | | sys.modules is the one true source.
* bpo-30604: clean up co_extra support (#2144)Dino Viehland2017-06-211-1/+1
| | | bpo-30604: port fix from 3.6 dropping binary compatibility tweaks
* Doc nits for bpo-16500 (#1841)Antoine Pitrou2017-05-281-1/+1
| | | | | | * Doc nits for bpo-16500 * Fix more references
* bpo-16500: Allow registering at-fork handlers (#1715)Antoine Pitrou2017-05-271-0/+10
| | | | | | | | | | | | * bpo-16500: Allow registering at-fork handlers * Address Serhiy's comments * Add doc for new C API * Add doc for new Python-facing function * Add NEWS entry + doc nit
* bpo-22257: Small changes for PEP 432. (#1728)Eric Snow2017-05-221-0/+16
| | | PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
* bpo-29102: Add a unique ID to PyInterpreterState. (#1639)Eric Snow2017-05-221-0/+37
|
* bpo-30395 _PyGILState_Reinit deadlock fix (#1734)Jason Fried2017-05-221-0/+4
| | | | head_lock could be held by another thread when fork happened. We should reset it to avoid deadlock.
* bpo-6532: Make the thread id an unsigned integer. (#781)Serhiy Storchaka2017-03-231-2/+3
| | | | | | | | | | | * bpo-6532: Make the thread id an unsigned integer. From C API side the type of results of PyThread_start_new_thread() and PyThread_get_thread_ident(), the id parameter of PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState changed from "long" to "unsigned long". * Restore a check in thread_get_ident().
* remove ceval timestamp supportBenjamin Peterson2016-09-091-3/+0
|
* Issue #28003: Implement PEP 525 -- Asynchronous Generators.Yury Selivanov2016-09-081-0/+5
|
View Lorry Controller Status