summaryrefslogtreecommitdiff
path: root/pylint/lint/parallel.py
Commit message (Collapse)AuthorAgeFilesLines
* Load custom plugins when linting in parallel (#8683)Jacob Walls2023-05-141-0/+5
|
* [PyCQA migration] Upgrade links to the repositories in code and doc (#8514)Pierre Sassoulas2023-03-291-2/+2
|
* [ruff] Add RUF specific rules and autofix (#8449)Pierre Sassoulas2023-03-141-1/+1
|
* [FileState] Make Pylinter.current_name and msg_store non nullablePierre Sassoulas2023-03-081-13/+3
|
* Support Implicit Namespace Packages (PEP 420) (#8153)Alexey Pelykh2023-02-091-7/+9
| | | Co-authored-by: Andreas Finkler <3929834+DudeNr33@users.noreply.github.com>
* Avoid hanging forever after a parallel job was killed (#7834)Daniel2022-12-121-10/+11
| | | | | | | | | | | | | * Replace multiprocessing.pool with concurrent.futures.ProcessPoolExecutor to avoid deadlocks. In a multiprocessing.pool, if a process terminates in a non-clean fashion (for example, due to OOM or a segmentation fault), the pool will silently replace said process, but the work that the process was supposed to do will never be done, causing pylint to hang indefinitely. The concurrent.futures.ProcessPoolExecutor will raise a BrokenProcessPool exception in that case, avoiding the hang. Co-authored-by: Daniël van Noord <13665637+DanielNoord@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Rename `broad-except` and new check `broad-exception-raised` (#7709)Dani Alcala2022-11-051-1/+1
| | | | Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>
* Some initial line wrapping of docstrings (#6672)Daniël van Noord2022-05-231-1/+2
|
* Pass ``msg_store`` and ``node`` to ``FileState`` (#6558)Daniël van Noord2022-05-091-0/+1
|
* Disable fixme for internal uses before we make it optional in #3512 (#6482)Pierre Sassoulas2022-05-011-1/+1
|
* Deprecate ``MapReduceMixin`` and implements its methods in ``BaseChecker`` ↵Daniël van Noord2022-04-191-9/+6
| | | | | | (#6383) Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com> Co-authored-by: Jacob Walls <jacobtylerwalls@gmail.com>
* Add two temporary mypy disables to parallel.pyDaniël van Noord2022-04-191-2/+2
|
* Remove unnecessary insertion of stats on checkersDaniël van Noord2022-04-181-5/+0
|
* Add typing to ``pylint/lint/parallel.py`` (#6377)Daniël van Noord2022-04-181-10/+17
|
* Make ``Message`` a ``dataclass`` (#6381)Daniël van Noord2022-04-181-18/+10
|
* Add some manual typing changes (#6325)Daniël van Noord2022-04-141-2/+3
| | | Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
* Use ``--py39-plus`` and ``--full-reorder`` in ``python-typing-update`` (#6313)Daniël van Noord2022-04-141-1/+2
|
* Use ``python-typing-update`` on ``pylint/lint`` directoryDaniël van Noord2022-04-141-15/+8
|
* Close and join pools correctly in ``parallel.py``Daniël van Noord2022-04-011-0/+3
|
* Add a pre-commit hook to check the copyright noticePierre Sassoulas2022-03-241-1/+1
| | | | | Fix the existing file so they have a notice. No header for setup.py or examples or doc
* Simplify hard to maintain copyright noticePierre Sassoulas2022-03-241-0/+1
| | | | | | git is the source of truth for the copyright, copyrite (the tool) was taking exponentially longer with each release, and it's polluting the code with sometime as much as 50 lines of names.
* Upgrade ``pydocstringformatter`` to ``0.5.0`` (#5910)Daniël van Noord2022-03-131-1/+2
|
* Update ``pydocstringformatter`` to 0.4.0 (#5787)Daniël van Noord2022-02-101-2/+2
|
* Add ``DeprecationWarning`` for non-string ``current_name`` & ``file`` (#5580)Daniël van Noord2022-01-271-0/+9
| | | | And make ``current_name`` and ``current_file`` string-only
* Use ``with`` statement in ``parallel.py`` (#5612)Daniël van Noord2021-12-301-7/+3
|
* Use ``dill`` to pickle when run in parallel mode (#5609)Daniël van Noord2021-12-301-15/+34
|
* Fix typos accross the whole codebase (#5575)Pierre Sassoulas2021-12-211-2/+2
| | | | Co-authored-by: Daniël van Noord <13665637+DanielNoord@users.noreply.github.com> Co-authored-by: Jacob Walls <jacobtylerwalls@gmail.com>
* Make parallel checker use ``MessageLocationTuple`` for ``Message`` (#5381)Daniël van Noord2021-11-241-2/+4
|
* Add error codes to all ``type: ignore`` (#5329)Daniël van Noord2021-11-171-2/+2
|
* Refactor ``LinterStats`` (#5074)Daniël van Noord2021-10-071-2/+4
| | | | | | | | * Refactor ``self.stats`` on linter and checker This adds a new class ``LinterStats`` which is used to store all kinds of stats during a run of ``pylint``. Tests have been changed slightly to be able to use the new class. Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
* Move ``merge_stats`` to ``checkerstats.py``Daniël van Noord2021-10-011-25/+4
|
* Add typing to ``filepath`` (#4980)Daniël van Noord2021-09-161-9/+11
| | | | | | | | | | * Change tests for ``filepath`` changes * Add pylint/typing.py and FileItem NamedTuple * Use NamedTuple more efficiently * Fix errors and tests after adding warning * Add deprecation for future API change in Checker Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>
* Add typing to all calls to ``self.stats`` (#4973)Daniël van Noord2021-09-151-6/+11
| | | | | | | | | | | | | | | * Add typing to all calls to ``self.stats`` All checkers inherit from a baseclass which has a ``stats`` attribute. This attribute has a fairly unmanageable type, but the current typing includes all variations of the attribute. Other changes not directly related to ``self.stats`` are due to ``mypy``warnings. This incorporate the feedback received in #4954 * Add ``CheckerStatistic`` class to ``pylint/typing`` * Guard `typing.Counter` import Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Similar check: Passe min_lines to recombined.Julien Palard2021-07-281-1/+1
|
* Fix copyright links (#4647)Marc Mueller2021-07-011-1/+1
| | | | * Fix link in license header * Update link to astroid bump_changelog
* Changes after black updateMarc Mueller2021-04-261-1/+1
|
* Enhancement #3413 ``consider-using-with`` (#4372)DudeNr332021-04-231-1/+3
| | | | | | | | * Implement consider-using-with check * Fix or disable consider-using-with in codebase * Fix ticket number in ChangeLog * Move functional test for ``open()`` into separate testfile and exclude this test from running with PyPy Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>
* Rename copying to license and upgrade the setup.cfg (#4338)Pierre Sassoulas2021-04-111-1/+1
|
* mapreduce| Fixes -jN for map/reduce Checkers (e.g. SimilarChecker)Frank Harrison2021-01-021-9/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This integrate the map/reduce functionality into lint.check_process(). We previously had `map` being invoked, here we add `reduce` support. We do this by collecting the map-data by worker and then passing it to a reducer function on the Checker object, if available - determined by whether they confirm to the `mapreduce_checker.MapReduceMixin` mixin interface or nor. This allows Checker objects to function across file-streams when using multiprocessing/-j2+. For example SimilarChecker needs to be able to compare data across all files. The tests, that we also add here, check that a Checker instance returns and reports expected data and errors, such as error-messages and stats - at least in a exit-ok (0) situation. On a personal note, as we are copying more data across process boundaries, I suspect that the memory implications of this might cause issues for large projects already running with -jN and duplicate code detection on. That said, given that it takes a long time to perform lints of large code bases that is an issue for the [near?] future and likely to be part of the performance work. Either way but let's get it working first and deal with memory and perforamnce considerations later - I say this as there are many quick wins we can make here, e.g. file-batching, hashing lines, data compression and so on.
* lint multiprocessing pool shutdownmanderj2020-10-301-7/+11
|
* check_parallel| Fixes module-hiding by function via file-renameFrank Harrison2020-10-191-0/+125
View Lorry Controller Status