summaryrefslogtreecommitdiff
path: root/scripts/internal/winmake.py
Commit message (Collapse)AuthorAgeFilesLines
* assert scripts have +x bitGiampaolo Rodola2021-12-151-5/+5
| | | | Signed-off-by: Giampaolo Rodola <g.rodola@gmail.com>
* Automatically sort imports (isort CLI tool) (#2033)Giampaolo Rodola2021-12-141-2/+3
|
* Remove Travis and Cirrus, use GH also for FreeBSD (#1880)Giampaolo Rodola2020-11-151-21/+0
|
* Fix py 3.9 [WinError 998] Invalid access to memory location (#1866)Giampaolo Rodola2020-10-311-3/+26
|
* Wheels 3 (#1764)Giampaolo Rodola2020-05-231-8/+6
|
* rename memleaks scriptGiampaolo Rodola2020-05-131-1/+1
|
* Memory leak test: take fluctuations into account (#1757)Giampaolo Rodola2020-05-131-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Preamble ======= We have a [memory leak test suite](https://github.com/giampaolo/psutil/blob/e1ea2bccf8aea404dca0f79398f36f37217c45f6/psutil/tests/__init__.py#L897), which calls a function many times and fails if the process memory increased. We do this in order to detect missing `free()` or `Py_DECREF` calls in the C modules. When we do, then we have a memory leak. The problem ========== A problem we've been having for probably over 10 years, is the false positives. That's because the memory fluctuates. Sometimes it may increase (or even decrease!) due to how the OS handles memory, the Python's garbage collector, the fact that RSS is an approximation and who knows what else. So thus far we tried to compensate that by using the following logic: - warmup (call fun 10 times) - call the function many times (1000) - if memory increased before/after calling function 1000 times, then keep calling it for another 3 secs - if it still increased at all (> 0) then fail This logic didn't really solve the problem, as we still had occasional false positives, especially lately on FreeBSD. The solution ========= This PR changes the internal algorithm so that in case of failure (mem > 0 after calling fun() N times) we retry the test for up to 5 times, increasing N (repetitions) each time, so we consider it a failure only if the memory **keeps increasing** between runs. So for instance, here's a legitimate failure: ``` psutil.tests.test_memory_leaks.TestModuleFunctionsLeaks.test_disk_partitions ... Run #1: extra-mem=696.0K, per-call=3.5K, calls=200 Run #2: extra-mem=1.4M, per-call=3.5K, calls=400 Run #3: extra-mem=2.1M, per-call=3.5K, calls=600 Run #4: extra-mem=2.7M, per-call=3.5K, calls=800 Run #5: extra-mem=3.4M, per-call=3.5K, calls=1000 FAIL ``` If, on the other hand, the memory increased on one run (say 200 calls) but decreased on the next run (say 400 calls), then it clearly means it's a false positive, because memory consumption may be > 0 on second run, but if it's lower than the previous run with less repetitions, then it cannot possibly represent a leak (just a fluctuation): ``` psutil.tests.test_memory_leaks.TestModuleFunctionsLeaks.test_net_connections ... Run #1: extra-mem=568.0K, per-call=2.8K, calls=200 Run #2: extra-mem=24.0K, per-call=61.4B, calls=400 OK ``` Note about mallinfo() ================ Aka #1275. `mallinfo()` on Linux is supposed to provide memory metrics about how many bytes gets allocated on the heap by `malloc()`, so it's supposed to be way more precise than RSS and also [USS](http://grodola.blogspot.com/2016/02/psutil-4-real-process-memory-and-environ.html). In another branch were I exposed it, I verified that fluctuations still occur even when using `mallinfo()` though, despite less often. So that means even `mallinfo()` would not grant 100% stability.
* Add new TestFdsLeak test class (#1752)Giampaolo Rodola2020-05-061-0/+8
|
* fix some memleak tests on winGiampaolo Rodola2020-05-051-2/+2
|
* Drastically improve "make test/build" speed.Giampaolo Rodola2020-05-011-14/+14
| | | | | | | Doing "make install" before any test is slow and not really necessary. Instead do "make build", and remove the part import setuptools and test psutil can be imported (do that in make install instead). This way I went down from 0.8 secs (install phase before starting the test) to 0.3 secs!
* Per-test file cleanup and new PsutilTestCase (#1743)Giampaolo Rodola2020-04-301-8/+5
| | | | | Test files/dirs are now removed after each test. when invoked via self.get_testfn(). Until now test files were stored in a global variable and were removed at process exit, via atexit.register(), but this didn't work with parallel tests because the fork()ed workers use os._exit(0), preventing cleanup functions to run. All test classes now inherit from PsutilTestCase class, which provides the most important methods requiring an automatic cleanup (get_test_subprocess() and others).
* Parallel build (#1741)Giampaolo Rodola2020-04-301-5/+6
|
* Git hook for renamed/added/deleted files + flake8 print() + tidelift (#1704)Giampaolo Rodola2020-02-211-1/+2
|
* Add C linter script (#1698)Giampaolo Rodola2020-02-181-2/+2
|
* get rid of pip_install() code for py2; move everything in runner.pyGiampaolo Rodola2020-02-151-1/+1
|
* point all shebangs to python 3Giampaolo Rodola2020-02-151-1/+1
|
* Add support for PyPy on Windows (#1686)Giampaolo Rodola2020-02-111-3/+9
|
* properly cleanup C threadGiampaolo Rodola2020-01-141-0/+2
|
* exec make install before 2 targetsGiampaolo Rodola2020-01-141-0/+2
|
* fix some win testsGiampaolo Rodola2020-01-061-1/+1
|
* winmake.py: use argparseGiampaolo Rodola2020-01-061-114/+84
|
* winmake.py: accept builtiple targets/argsGiampaolo Rodola2020-01-031-9/+12
|
* winmake / uninstall: remove installation path from easy-install.pth fileGiampaolo Rodola2020-01-021-0/+18
|
* Drop windows XP support (#1652)Giampaolo Rodola2020-01-011-1/+0
| | | | minimum supported now is Windows Vista
* highlight cmd.exe warnings/errs from VSGiampaolo Rodola2019-12-301-1/+53
|
* fix #1578: 'perf' lib was renamed to 'pyperf'Giampaolo Rodola2019-09-181-1/+1
|
* fix #1478: add make command to re-run tests failed on last runGiampaolo Rodola2019-04-051-0/+10
|
* update doc + change git hook locationGiampaolo Rodola2019-03-021-1/+1
|
* Restore Win-7 support on GIT master (5.5.1 was OK) (#1446)Giampaolo Rodola2019-02-281-8/+11
|
* appveyor: run print scripts after testsGiampaolo Rodola2019-02-271-4/+7
|
* move access_denied scriptGiampaolo Rodola2019-02-271-1/+1
|
* Process SE DEBUG mode was not set on Windows (#1429)Giampaolo Rodola2019-02-251-0/+7
|
* fix PEP8 style mistakesGiampaolo Rodola2018-11-071-1/+1
|
* Correct capitalization of PyPI (#1337)Jon Dufresne2018-09-261-1/+1
| | | As spelled on https://pypi.org/.
* winmake: add upload-wheels cmdGiampaolo Rodola2018-06-291-0/+7
|
* adust winmake individual testsGiampaolo Rodola2018-04-131-7/+7
|
* and another oneGiampaolo Rodola2018-04-121-1/+5
|
* adjust win depsGiampaolo Rodola2018-04-121-4/+8
|
* fix appveyor.yml syntaxGiampaolo Rodola2018-03-291-1/+1
|
* fix git commit hook cmd on winGiampaolo Rodola2018-03-241-1/+11
|
* Revert "Move tests out of package to top level directory (#1232)" (#1242)Giampaolo Rodola2018-03-111-7/+7
| | | This reverts commit b578d2febfd35f80fcfa1ce1bdf18d44d21b1581.
* Move tests out of package to top level directory (#1232)Jon Dufresne2018-03-091-7/+7
| | | | | | | | Cleanly separates tests from the package itself. Prevents the tests being installed to site-packages. Tests are still distributed with the source distribution by MANIFEST.in. Avoids installing tests in production environments, leading to less total code in the environment.
* 1173 debug mode (#1176)Giampaolo Rodola2017-11-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | * implement PSUTIL_DEBUG from C module * update doc * add psutil_debug() utility function * update doc * enable PSUTIL_DEBUG for tests * update appveyor.yml * change psutil_debug() signature so that it can accept variable num of args * provide DEBUG info in psutil_raise_for_pid() * properly print debug message * do not print too much
* refactor winmake.pyGiampaolo Rodola2017-11-121-12/+29
|
* #1053: drop python 3.3 supportGiampaolo Rodola2017-11-081-2/+2
|
* pep81151-PSUTIL_TESTING_env_varGiampaolo Rodola2017-10-201-1/+1
|
* automatically set PSUTIL_TEST env var during testsGiampaolo Rodola2017-10-201-12/+0
|
* re: #1120 / PEP527: no longer provide .exe files for WindowsGiampaolo Rodola2017-09-081-7/+0
|
* fix winmakeGiampaolo Rodola2017-05-201-2/+2
|
* import psutil right after make build to make sure compilation workedGiampaolo Rodola2017-05-201-1/+6
|
View Lorry Controller Status