summaryrefslogtreecommitdiff
path: root/include/haproxy/pattern.h
Commit message (Collapse)AuthorAgeFilesLines
* CLEANUP: pattern: remove export of non-existent function pattern_delete()Dragan Dosen2021-05-251-1/+0
|
* CLEANUP: pattern: remove the unused and dangerous pat_ref_reload()Willy Tarreau2021-05-111-1/+0
| | | | | | | | This function was not used anymore after the atomic updates were implemented in 2.3, and it must not be used given that it does not yield and can easily make the process hang for tens of seconds on large acls/maps. Let's remove it before someone uses it as an example to implement something else!
* MINOR: pattern: support purging arbitrary ranges of generationsWilly Tarreau2021-04-301-1/+15
| | | | | | Instead of being able to purge only values older than a specific value, let's support arbitrary ranges and make pat_ref_purge_older() just be one special case of this one.
* CLEANUP: pattern: make all pattern tables read-onlyWilly Tarreau2021-04-101-6/+6
| | | | | | Interestingly, all arrays used to declare patterns were read-write while only hard-coded. Let's mark them const so that they move from data to rodata and don't risk to experience false sharing.
* CLEANUP: atomic: add an explicit _FETCH variant for add/sub/and/orWilly Tarreau2021-04-071-1/+1
| | | | | | | | | Currently our atomic ops return a value but it's never known whether the fetch is done before or after the operation, which causes some confusion each time the value is desired. Let's create an explicit variant of these operations suffixed with _FETCH to explicitly mention that the fetch occurs after the operation, and make use of it at the few call places.
* MINOR: pattern: add the missing generation ID manipulation functionsWilly Tarreau2021-01-151-0/+41
| | | | | | The functions needed to commit a pattern file generation number or increase it were still missing. Better not have the caller play with these.
* CLEANUP: pattern: rename pat_ref_commit() to pat_ref_commit_elt()Willy Tarreau2021-01-151-1/+1
| | | | | | | | It's about the third time I get confused by these functions, half of which manipulate the reference as a whole and those manipulating only an entry. For me "pat_ref_commit" means committing the pattern reference, not just an element, so let's rename it. A number of other ones should really be renamed before 2.4 gets released :-/
* CLEANUP: assorted typo fixes in the code and commentsIlya Shipitsin2020-12-211-1/+1
| | | | This is 13n iteration of typo fixes
* MINOR: pattern: add pat_ref_purge_older() to purge old entriesWilly Tarreau2020-11-051-0/+1
| | | | | | | | | | | | | | This function will be usable to purge at most a specified number of old entries from a reference. Entries are declared old if their generation number is in the past compared to the one passed in argument. This will ease removal of early entries when new ones have been appended. We also call malloc_trim() when available, at the end of the series, because this is one place where there is a lot of memory to save. Reloads of 1M IP addresses used in an ACL made the process grow up to 1.7 GB RSS after 10 reloads and roughly stabilize there without this call, versus only 260 MB when the call is present. Sadly there is no direct equivalent for jemalloc, which stabilizes around 800MB-1GB.
* MINOR: pattern: implement pat_ref_load() to load a pattern at a given generationWilly Tarreau2020-11-051-1/+1
| | | | | | | | | | pat_ref_load() basically combines pat_ref_append() and pat_ref_commit(). It's very similar to pat_ref_add() except that it also allows to set the generation ID and the line number. pat_ref_add() was modified to directly rely on it to avoid code duplication. Note that a previous declaration of pat_ref_load() was removed as it was just a leftover of an earlier incarnation of something possibly similar, so no existing functionality was changed here.
* MINOR: pattern: add pat_ref_commit() to commit a previously inserted elementWilly Tarreau2020-11-051-0/+1
| | | | | | | | | This function will be used after a successful pat_ref_append() to propagate the pattern to all use places (including parsing and indexing). On failure, it will entirely roll back all insertions and free the pattern itself. It also preserves the generation number so that it is convenient for use in association with pat_ref_append(). pat_ref_add() was modified to rely on it instead of open-coding the insertion and roll-back.
* MINOR: pattern: introduce pat_ref_delete_by_ptr() to delete a valid referenceWilly Tarreau2020-11-051-0/+1
| | | | | | | | Till now the only way to remove a known reference was via pat_ref_delete_by_id() which scans the whole list to find a matching pointer. Let's add pat_ref_delete_by_ptr() which takes a valid pointer. It can be called by the function above after the pointer is found, and can also be used to roll back a failed insertion much more efficiently.
* CLEANUP: pattern: remove pat_delete_fcts[] and pattern_head->delete()Willy Tarreau2020-11-051-1/+0
| | | | | | | These ones are not used anymore, so let's remove them to remove a bit of the complexity. The ACL keyword's delete() function could be removed as well, though most keyword declarations are positional and we have a high risk of introducing a mistake here, so let's not touch the ACL part.
* MINOR: pattern: remerge the list and tree deletion functionsWilly Tarreau2020-11-051-3/+2
| | | | | | | pat_del_tree_gen() was already chained onto pat_del_list_gen() to deal with remaining cases, so let's complete the merge and have a generic pattern deletion function acting on the reference and taking care of reliably removing all elements.
* MEDIUM: pattern: change the pat_del_* functions to delete from the referencesWilly Tarreau2020-11-051-7/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is the next step in speeding up entry removal. Now we don't scan the whole lists or trees for elements pointing to the target reference, instead we start from the reference and delete all linked patterns. This simplifies some delete functions since we don't need anymore to delete multiple times from an expression since all nodes appear after the reference element. We can now have one generic list and one generic tree deletion function. This required the replacement of pattern_delete() with an open-coded version since we now need to lock all expressions first before proceeding. This means there is a high risk of lock inversion here but given that the expressions are always scanned in the same order from the same head, this must not happen. Now deleting first entries is instantaneous, and it's still slow to delete the last ones when looking up their ID since it still requires to look them up by a full scan, but it's already way faster than previously. Typically removing the last 10 IP from a 20M entries ACL with a full-scan each took less than 2 seconds. It would be technically possible to make use of indexed entries to speed up most lookups for removal by value (e.g. IP addresses) but that's for later.
* MINOR: pattern: make the delete and prune functions more genericWilly Tarreau2020-11-051-6/+2
| | | | | | | Now we have a single prune() function to act on an expression, and one delete function for the lists and one for the trees. The presence of a pointer in the lists is enough to warrant a free, and we rely on the PAT_SF_REGFREE flag to decide whether to free using free() or regfree().
* MINOR: pattern: export pat_ref_push()Willy Tarreau2020-10-311-0/+1
| | | | | Strangely this one was marked static inline within the file itself. Let's export it.
* MINOR: pattern: make pat_ref_append() return the newly added elementWilly Tarreau2020-10-311-1/+1
| | | | | | | | It's more convenient to return the element than to return just 0 or 1, as the next thing we'll want to do is to act on this element! In addition it was using variable arguments instead of consts, causing some reuse constraints which were also addressed. This doesn't change its use as a boolean, hence why call places were not modified.
* MEDIUM: map: make the "clear map" operation yieldWilly Tarreau2020-06-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As reported in issue #419, a "clear map" operation on a very large map can take a lot of time and freeze the entire process for several seconds. This patch makes sure that pat_ref_prune() can regularly yield after clearing some entries so that the rest of the process continues to work. The first part, the removal of the patterns, can take quite some time by itself in one run but it's still relatively fast. It may block for up to 100ms for 16M IP addresses in a tree typically. This change needed to declare an I/O handler for the clear operation so that we can get back to it after yielding. The second part can be much slower because it deconstructs the elements and its users, but it iterates progressively so we can yield less often here. The patch was tested with traffic in parallel sollicitating the map being released and showed no problem. Some traffic will definitely notice an incomplete map but the filling is already not atomic anyway thus this is not different. It may be backported to stable versions once sufficiently tested for side effects, at least as far as 2.0 in order to avoid the watchdog triggering when the process is frozen there. For a better behaviour, all these prune_* functions should support yielding so that the callers have a chance to continue also yield in turn.
* CLEANUP: include: move sample_data out of sample-t.hWilly Tarreau2020-06-111-0/+1
| | | | | | | | The struct sample_data is used by pattern, map and vars, and currently requires to include sample-t which comes with many other dependencies. Let's move sample_data into its own file to shorten the dependency tree. This revealed a number of issues in adjacent files which were hidden by the fact that sample-t.h brought everything that was missing.
* REORG: include: move pattern.h to haproxy/pattern{,-t}.hWilly Tarreau2020-06-111-0/+220
It was moved as-is, except for extern declaration of pattern_reference. A few C files used to include it but didn't need it anymore after having been split apart so this was cleaned.
View Lorry Controller Status