summaryrefslogtreecommitdiff
path: root/src/backend/libpq
Commit message (Collapse)AuthorAgeFilesLines
* NULL is not an ideal way to spell bool "false".Tom Lane2023-04-141-1/+1
| | | | Thinko in commit 6633cfb21, detected by buildfarm member hamerkop.
* Remove code in charge of freeing regexps generation by Lab.cMichael Paquier2023-04-141-67/+5
| | | | | | | | | | | | | | | | | | | | | | | | | bea3d7e has redesigned the regexp engine so as all the allocations go through palloc() with a dedicated memory context. hba.c had to cope with the past memory management logic by going through all the HBA and ident lines generated, then directly free all the regexps found in AuthTokens to ensure that no leaks would happen. Such leaks could happen for example in the postmaster after a SIGHUP, in the event of an HBA and/or ident reload failure where all the new content parsed must be discarded, including all the regexps that may have been compiled. Now that regexps are palloc()'d in their own memory context, MemoryContextDelete() is enough to ensure that all the compiled regexps are properly gone. Simplifying this logic in hba.c has the effect to only remove code. Most of it is new in v16, except the part for regexps compiled in ident entries for the system username, so doing this cleanup now rather than when v17 opens for business will reduce future diffs with the upcoming REL_16_STABLE. Some comments were incorrect since bea3d7e, now fixed to reflect the reality. Reviewed-by: Bertrand Drouvot, Álvaro Herrera Discussion: https://postgr.es/m/ZDdJ289Ky2qEj4h+@paquier.xyz
* Explicitly require MIT Kerberos for GSSAPIStephen Frost2023-04-132-4/+6
| | | | | | | | | | WHen building with GSSAPI support, explicitly require MIT Kerberos and check for gssapi_ext.h in configure.ac and meson.build. Also add documentation explicitly stating that we now require MIT Kerberos when building with GSSAPI support. Reveiwed by: Johnathan Katz Discussion: https://postgr.es/m/abcc73d0-acf7-6896-e0dc-f5bc12a61bb1@postgresql.org
* De-Revert "Add support for Kerberos credential delegation"Stephen Frost2023-04-133-2/+90
| | | | | | | | | | | | | | | | | | This reverts commit 3d03b24c3 (Revert Add support for Kerberos credential delegation) which was committed on the grounds of concern about portability, but on further review and discussion, it's clear that we are better off explicitly requiring MIT Kerberos as that appears to be the only GSSAPI library currently that's under proper maintenance and ongoing development. The API used for storing credentials was added to MIT Kerberos over a decade ago while for the other libraries which appear to be mainly based on Heimdal, which exists explicitly to be a re-implementation of MIT Kerberos, the API never made it to a released version (even though it was added to the Heimdal git repo over 5 years ago..). This post-feature-freeze change was approved by the RMT. Discussion: https://postgr.es/m/ZDDO6jaESKaBgej0%40tamriel.snowman.net
* Revert "Add support for Kerberos credential delegation"Stephen Frost2023-04-083-90/+2
| | | | | | | | | | | This reverts commit 3d4fa227bce4294ce1cc214b4a9d3b7caa3f0454. Per discussion and buildfarm, this depends on APIs that seem to not be available on at least one platform (NetBSD). Should be certainly possible to rework to be optional on that platform if necessary but bit late for that at this point. Discussion: https://postgr.es/m/3286097.1680922218@sss.pgh.pa.us
* Add support for Kerberos credential delegationStephen Frost2023-04-073-2/+90
| | | | | | | | | | | | | | | | | | | Support GSSAPI/Kerberos credentials being delegated to the server by a client. With this, a user authenticating to PostgreSQL using Kerberos (GSSAPI) credentials can choose to delegate their credentials to the PostgreSQL server (which can choose to accept them, or not), allowing the server to then use those delegated credentials to connect to another service, such as with postgres_fdw or dblink or theoretically any other service which is able to be authenticated using Kerberos. Both postgres_fdw and dblink are changed to allow non-superuser password-less connections but only when GSSAPI credentials have been delegated to the server by the client and GSSAPI is used to authenticate to the remote system. Authors: Stephen Frost, Peifeng Qiu Reviewed-By: David Christensen Discussion: https://postgr.es/m/CO1PR05MB8023CC2CB575E0FAAD7DF4F8A8E29@CO1PR05MB8023.namprd05.prod.outlook.com
* Make SCRAM iteration count configurableDaniel Gustafsson2023-03-271-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace the hardcoded value with a GUC such that the iteration count can be raised in order to increase protection against brute-force attacks. The hardcoded value for SCRAM iteration count was defined to be 4096, which is taken from RFC 7677, so set the default for the GUC to 4096 to match. In RFC 7677 the recommendation is at least 15000 iterations but 4096 is listed as a SHOULD requirement given that it's estimated to yield a 0.5s processing time on a mobile handset of the time of RFC writing (late 2015). Raising the iteration count of SCRAM will make stored passwords more resilient to brute-force attacks at a higher computational cost during connection establishment. Lowering the count will reduce computational overhead during connections at the tradeoff of reducing strength against brute-force attacks. There are however platforms where even a modest iteration count yields a too high computational overhead, with weaker password encryption schemes chosen as a result. In these situations, SCRAM with a very low iteration count still gives benefits over weaker schemes like md5, so we allow the iteration count to be set to one at the low end. The new GUC is intentionally generically named such that it can be made to support future SCRAM standards should they emerge. At that point the value can be made into key:value pairs with an undefined key as a default which will be backwards compatible with this. Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org> Discussion: https://postgr.es/m/F72E7BC7-189F-4B17-BF47-9735EB72C364@yesql.se
* Add a hook for modifying the ldapbind passwordAndrew Dunstan2023-03-151-1/+11
| | | | | | | | | | | The hook can be installed by a shared_preload library. A similar mechanism could be used for radius paswords, for example, and the type name auth_password_hook_typ has been shosen with that in mind. John Naylor and Andrew Dunstan Discussion: https://postgr.es/m/469b06ed-69de-ba59-c13a-91d2372e52a9@dunslane.net
* Don't leak descriptors into subprograms.Thomas Munro2023-03-031-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Open long-lived data and WAL file descriptors with O_CLOEXEC. This flag was introduced by SUSv4 (POSIX.1-2008), and by now all of our target Unix systems have it. Our open() implementation for Windows already had that behavior, so provide a dummy O_CLOEXEC flag on that platform. For now, callers of open() and the "thin" wrappers in fd.c that deal in raw descriptors need to pass in O_CLOEXEC explicitly if desired. This commit does that for WAL files, and automatically for everything accessed via VFDs including SMgrRelation and BufFile. (With more discussion we might decide to turn it on automatically for the thin open()-wrappers too to avoid risk of missing places that need it, but these are typically used for short-lived descriptors where we don't expect to fork/exec, and it's remotely possible that extensions could be using these APIs and passing descriptors to subprograms deliberately, so that hasn't been done here.) Do the same for sockets and the postmaster pipe with FD_CLOEXEC. (Later commits might use modern interfaces to remove these extra fcntl() calls and more where possible, but we'll need them as a fallback for a couple of systems, so do it that way in this initial commit.) With this change, subprograms executed for archiving, copying etc will no longer have access to the server's descriptors, other than the ones that we decide to pass down. Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version) Discussion: https://postgr.es/m/CA%2BhUKGKb6FsAdQWcRL35KJsftv%2B9zXqQbzwkfRf1i0J2e57%2BhQ%40mail.gmail.com
* Fix outdated references to guc.cDaniel Gustafsson2023-03-021-1/+2
| | | | | | | | | Commit 0a20ff54f split out the GUC variables from guc.c into a new file guc_tables.c. This updates comments referencing guc.c regarding variables which are now in guc_tables.c. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/6B50C70C-8C1F-4F9A-A7C0-EEAFCC032406@yesql.se
* Add description for new patterns supported in HBA and ident sample filesMichael Paquier2023-02-162-8/+13
| | | | | | | | | | | | | | Support for regexps in database and role entries for pg_hba.conf has been added in 8fea8683, and efb6f4a has extended support of pg-user in pg_ident.conf, still both of them have missed a short description about the new patterns supported in their respective sample files. This commit closes the gap, by providing a short description of all the new features supported for each entry type. Reported-by: Pavel Luzanov Reviewed-by: Jelte Fennema, Pavel Luzanov Discussion: https://postgr.es/m/e495112d-8741-e651-64a2-ecb5728f1a56@postgrespro.ru
* Fix handling of SCRAM-SHA-256's channel binding with RSA-PSS certificatesMichael Paquier2023-02-151-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | OpenSSL 1.1.1 and newer versions have added support for RSA-PSS certificates, which requires the use of a specific routine in OpenSSL to determine which hash function to use when compiling it when using channel binding in SCRAM-SHA-256. X509_get_signature_nid(), that is the original routine the channel binding code has relied on, is not able to determine which hash algorithm to use for such certificates. However, X509_get_signature_info(), new to OpenSSL 1.1.1, is able to do it. This commit switches the channel binding logic to rely on X509_get_signature_info() over X509_get_signature_nid(), which would be the choice when building with 1.1.1 or newer. The error could have been triggered on the client or the server, hence libpq and the backend need to have their related code paths patched. Note that attempting to load an RSA-PSS certificate with OpenSSL 1.1.0 or older leads to a failure due to an unsupported algorithm. The discovery of relying on X509_get_signature_info() comes from Jacob, the tests have been written by Heikki (with few tweaks from me), while I have bundled the whole together while adding the bits needed for MSVC and meson. This issue exists since channel binding exists, so backpatch all the way down. Some tests are added in 15~, triggered if compiling with OpenSSL 1.1.1 or newer, where the certificate and key files can easily be generated for RSA-PSS. Reported-by: Gunnar "Nick" Bluth Author: Jacob Champion, Heikki Linnakangas Discussion: https://postgr.es/m/17760-b6c61e752ec07060@postgresql.org Backpatch-through: 11
* Change argument type of pq_sendbytes from char * to void *Peter Eisentraut2023-02-141-1/+1
| | | | | | | | This is a follow-up to 1f605b82ba66ece8b421b10d41094dd2e3c0c48b. It allows getting rid of further casts at call sites. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/783a4edb-84f9-6df2-7470-2ef5ccc6607a@enterprisedb.com
* Support the same patterns for pg-user in pg_ident.conf as in pg_hba.confMichael Paquier2023-01-201-31/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While pg_hba.conf has support for non-literal username matches, and this commit extends the capabilities that are supported for the PostgreSQL user listed in an ident entry part of pg_ident.conf, with support for: 1. The "all" keyword, where all the requested users are allowed. 2. Membership checks using the + prefix. 3. Using a regex to match against multiple roles. 1. is a feature that has been requested by Jelte Fennema, 2. something that has been mentioned independently by Andrew Dunstan, and 3. is something I came up with while discussing how to extend the first one, whose implementation is facilitated by 8fea868. This allows matching certain system users against many different postgres users with a single line in pg_ident.conf. Without this, one would need one line for each of the postgres users that a system user can log in as, which can be cumbersome to maintain. Tests are added to the TAP test of peer authentication to provide coverage for all that. Note that this introduces a set of backward-incompatible changes to be able to detect the new patterns, for the following cases: - A role named "all". - A role prefixed with '+' characters, which is something that would not have worked in HBA entries anyway. - A role prefixed by a slash character, similarly to 8fea868. Any of these can be still be handled by using quotes in the Postgres role defined in an ident entry. A huge advantage of this change is that the code applies the same checks for the Postgres roles in HBA and ident entries, via the common routine check_role(). **This compatibility change should be mentioned in the release notes.** Author: Jelte Fennema Discussion: https://postgr.es/m/DBBPR83MB0507FEC2E8965012990A80D0F7FC9@DBBPR83MB0507.EURPRD83.prod.outlook.com
* Store IdentLine->pg_user as an AuthTokenMichael Paquier2023-01-161-9/+11
| | | | | | | | | | | | | | | While system_user was stored as an AuthToken in IdentLine, pg_user was stored as a plain string. This commit changes the code as we start storing pg_user as an AuthToken too. This does not have any functional changes, as all the operations on pg_user only use the string from the AuthToken. There is no regexp compiled and no check based on its quoting, yet. This is in preparation of more features that intend to extend its capabilities, like support for regexps and group membership. Author: Jelte Fennema Discussion: https://postgr.es/m/CAGECzQRNow4MwkBjgPxywXdJU_K3a9+Pm78JB7De3yQwwkTDew@mail.gmail.com
* Rename some variables related to ident files in hba.{c,h}Michael Paquier2023-01-121-39/+39
| | | | | | | | | | | | | | | | | | | | | | | | The code that handles authentication for user maps was pretty confusing with its choice of variable names. It involves two types of users: a system user and a Postgres user (well, role), and these were not named consistently throughout the code that processes the user maps loaded from pg_ident.conf at authentication. This commit changes the following things to improve the situation: - Rename "pg_role" to "pg_user" and "token" to "system_user" in IndetLine. These choices are more consistent with the pg_ident.conf example in the docs, as well. "token" has been introduced recently in fc579e1, and it is way worse than the choice before that, "ident_user". - Switch the order of the fields in IdentLine to map with the order of the items in the ident files, as of map name, system user and PG user. - In check_ident_usermap(), rename "regexp_pgrole" to "expanded_pg_user" when processing a regexp for the system user entry in a user map. This variable does not store a regular expression at all: it would be either a string or a substitution to \1 if the Postgres role is specified as such. Author: Jelte Fennema Discussion: https://postgr.es/m/CAGECzQTkwELHUOAKhvdA+m3tWbUQySHHkExJV8GAZ1pwgbEgXg@mail.gmail.com
* Use WaitEventSet API for postmaster's event loop.Thomas Munro2023-01-122-42/+1
| | | | | | | | | | | | | | | | | | | | | | Switch to a design similar to regular backends, instead of the previous arrangement where signal handlers did non-trivial state management and called fork(). The main changes are: * The postmaster now has its own local latch to wait on. (For now, we don't want other backends setting its latch directly, but that could probably be made to work with more research on robustness.) * The existing signal handlers are cut in two: a handle_pm_XXX() part that just sets pending_pm_XXX flags and the latch, and a process_pm_XXX() part that runs later when the latch is seen. * Signal handlers are now installed with the regular pqsignal() function rather than the special pqsignal_pm() function; historical portability concerns about the effect of SA_RESTART on select() are no longer relevant, and we don't need to block signals anymore. Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CA%2BhUKG%2BZ-HpOj1JsO9eWUP%2Bar7npSVinsC_npxSy%2BjdOMsx%3DGg%40mail.gmail.com
* Common function for percent placeholder replacementPeter Eisentraut2023-01-111-30/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | There are a number of places where a shell command is constructed with percent-placeholders (like %x). It's cumbersome to have to open-code this several times. This factors out this logic into a separate function. This also allows us to ensure consistency for and document some subtle behaviors, such as what to do with unrecognized placeholders. The unified handling is now that incorrect and unknown placeholders are an error, where previously in most cases they were skipped or ignored. This affects the following settings: - archive_cleanup_command - archive_command - recovery_end_command - restore_command - ssl_passphrase_command The following settings are part of this refactoring but already had stricter error handling and should be unchanged in their behavior: - basebackup_to_shell.command Reviewed-by: Nathan Bossart <nathandbossart@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/5238bbed-0b01-83a6-d4b2-7eb0562a054e%40enterprisedb.com
* New header varatt.h split off from postgres.hPeter Eisentraut2023-01-102-0/+2
| | | | | | | | | This new header contains all the variable-length data types support (TOAST support) from postgres.h, which isn't needed by large parts of the backend code. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/ddcce239-0f29-6e62-4b47-1f8ca742addf%40enterprisedb.com
* Perform apply of large transactions by parallel workers.Amit Kapila2023-01-091-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, for large transactions, the publisher sends the data in multiple streams (changes divided into chunks depending upon logical_decoding_work_mem), and then on the subscriber-side, the apply worker writes the changes into temporary files and once it receives the commit, it reads from those files and applies the entire transaction. To improve the performance of such transactions, we can instead allow them to be applied via parallel workers. In this approach, we assign a new parallel apply worker (if available) as soon as the xact's first stream is received and the leader apply worker will send changes to this new worker via shared memory. The parallel apply worker will directly apply the change instead of writing it to temporary files. However, if the leader apply worker times out while attempting to send a message to the parallel apply worker, it will switch to "partial serialize" mode - in this mode, the leader serializes all remaining changes to a file and notifies the parallel apply workers to read and apply them at the end of the transaction. We use a non-blocking way to send the messages from the leader apply worker to the parallel apply to avoid deadlocks. We keep this parallel apply assigned till the transaction commit is received and also wait for the worker to finish at commit. This preserves commit ordering and avoid writing to and reading from files in most cases. We still need to spill if there is no worker available. This patch also extends the SUBSCRIPTION 'streaming' parameter so that the user can control whether to apply the streaming transaction in a parallel apply worker or spill the change to disk. The user can set the streaming parameter to 'on/off', or 'parallel'. The parameter value 'parallel' means the streaming will be applied via a parallel apply worker, if available. The parameter value 'on' means the streaming transaction will be spilled to disk. The default value is 'off' (same as current behaviour). In addition, the patch extends the logical replication STREAM_ABORT message so that abort_lsn and abort_time can also be sent which can be used to update the replication origin in parallel apply worker when the streaming transaction is aborted. Because this message extension is needed to support parallel streaming, parallel streaming is not supported for publications on servers < PG16. Author: Hou Zhijie, Wang wei, Amit Kapila with design inputs from Sawada Masahiko Reviewed-by: Sawada Masahiko, Peter Smith, Dilip Kumar, Shi yu, Kuroda Hayato, Shveta Mallik Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
* Update copyright for 2023Bruce Momjian2023-01-0217-17/+17
| | | | Backpatch-through: 11
* Add copyright notices to meson filesAndrew Dunstan2022-12-201-0/+2
| | | | Discussion: https://postgr.es/m/222b43a5-2fb3-2c1b-9cd0-375d376c8246@dunslane.net
* Remove hardcoded dependency to cryptohash type in the internals of SCRAMMichael Paquier2022-12-202-58/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SCRAM_KEY_LEN was a variable used in the internal routines of SCRAM to size a set of fixed-sized arrays used in the SHA and HMAC computations during the SASL exchange or when building a SCRAM password. This had a hard dependency on SHA-256, reducing the flexibility of SCRAM when it comes to the addition of more hash methods. A second issue was that SHA-256 is assumed as the cryptohash method to use all the time. This commit renames SCRAM_KEY_LEN to a more generic SCRAM_KEY_MAX_LEN, which is used as the size of the buffers used by the internal routines of SCRAM. This is aimed at tracking centrally the maximum size necessary for all the hash methods supported by SCRAM. A global variable has the advantage of keeping the code in its simplest form, reducing the need of more alloc/free logic for all the buffers used in the hash calculations. A second change is that the key length (SHA digest length) and hash types are now tracked by the state data in the backend and the frontend, the common portions being extended to handle these as arguments by the internal routines of SCRAM. There are a few RFC proposals floating around to extend the SCRAM protocol, including some to use stronger cryptohash algorithms, so this lifts some of the existing restrictions in the code. The code in charge of parsing and building SCRAM secrets is extended to rely on the key length and on the cryptohash type used for the exchange, assuming currently that only SHA-256 is supported for the moment. Note that the mock authentication simply enforces SHA-256. Author: Michael Paquier Reviewed-by: Peter Eisentraut, Jonathan Katz Discussion: https://postgr.es/m/Y5k3Qiweo/1g9CG6@paquier.xyz
* Static assertions cleanupPeter Eisentraut2022-12-152-7/+7
| | | | | | | | | | | | | | | | | | | | | Because we added StaticAssertStmt() first before StaticAssertDecl(), some uses as well as the instructions in c.h are now a bit backwards from the "native" way static assertions are meant to be used in C. This updates the guidance and moves some static assertions to better places. Specifically, since the addition of StaticAssertDecl(), we can put static assertions at the file level. This moves a number of static assertions out of function bodies, where they might have been stuck out of necessity, to perhaps better places at the file level or in header files. Also, when the static assertion appears in a position where a declaration is allowed, then using StaticAssertDecl() is more native than StaticAssertStmt(). Reviewed-by: John Naylor <john.naylor@enterprisedb.com> Discussion: https://www.postgresql.org/message-id/flat/941a04e7-dd6f-c0e4-8cdf-a33b3338cbda%40enterprisedb.com
* Fix typo in hba.cMichael Paquier2022-11-261-1/+1
| | | | Spotted while reading through the git history.
* Introduce variables for initial and max nesting depth on configuration filesMichael Paquier2022-11-251-5/+5
| | | | | | | | | | The code has been assuming already in a few places that the initial recursion nesting depth is 0, and the recent changes in hba.c (mainly 783e8c6) have relies on this assumption in more places. The maximum recursion nesting level is assumed to be 10 for hba.c and GUCs. Author: Julien Rouhaud Discussion: https://postgr.es/m/20221124090724.n7amf5kpdhx6vb76@jrouhaud
* Add support for file inclusions in HBA and ident configuration filesMichael Paquier2022-11-243-20/+225
| | | | | | | | | | | | | | | | | | | | | | | | | pg_hba.conf and pg_ident.conf gain support for three record keywords: - "include", to include a file. - "include_if_exists", to include a file, ignoring it if missing. - "include_dir", to include a directory of files. These are classified by name (C locale, mostly) and need to be prefixed by ".conf", hence following the same rules as GUCs. This commit relies on the refactoring pieces done in efc9816, ad6c528, 783e8c6 and 1b73d0b, adding a small wrapper to build a list of TokenizedAuthLines (tokenize_include_file), and the code is shaped to offer some symmetry with what is done for GUCs with the same options. pg_hba_file_rules and pg_ident_file_mappings gain a new field called file_name, to track from which file a record is located, taking advantage of the addition of rule_number in c591300 to offer an organized view of the HBA or ident records loaded. Bump catalog version. Author: Julien Rouhaud Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/20220223045959.35ipdsvbxcstrhya@jrouhaud
* Create memory context for tokenization after opening top-level file in hba.cMichael Paquier2022-11-241-16/+16
| | | | | | | | | | | | | | The memory context was created before attempting to open the first HBA or ident file, which would cause it to leak. This had no consequences for the system views for HBA and ident files, but this would cause memory leaks in the postmaster on reload if the initial HBA and/or ident files are missing, which is a valid behavior while the backend is running. Oversight in efc9816. Author: Ted Yu Discussion: https://postgr.es/m/CALte62xH6ivgiKKzPRJgfekPZC6FKLB3xbnf3=tZmc_gKj78dw@mail.gmail.com
* Add missing initialization in tokenize_expand_file() for output listMichael Paquier2022-11-241-1/+1
| | | | | | | | This should have been added in efc9816, but it looks like I have found a way to mess up a bit a patch split. This should have no consequence in practice, but let's be clean. Discussion: https://postgr.es/m/Y324HvGKiWxW2yxe@paquier.xyz
* Rework memory contexts in charge of HBA/ident tokenizationMichael Paquier2022-11-241-41/+117
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The list of TokenizedAuthLines generated at parsing for the HBA and ident files is now stored in a static context called tokenize_context, where only all the parsed tokens are stored. This context is created when opening the first authentication file of a HBA/ident set (hba_file or ident_file), and is cleaned up once we are done all the work around it through a new routine called free_auth_file(). One call of open_auth_file() should have one matching call of free_auth_file(), the creation and deletion of the tokenization context is controlled by the recursion depth of the tokenization. Rather than having tokenize_auth_file() return a memory context that includes all the records, the tokenization logic now creates and deletes one memory context each time this function is called. This will simplify recursive calls to this routine for the upcoming inclusion record logic. While on it, rename tokenize_inc_file() to tokenize_expand_file() as this would conflict with the upcoming patch that will add inclusion records for HBA/ident files. An '@' file has its tokens added to an existing list. Reloading HBA/indent configuration in a tight loop shows no leaks, as of one type of test done (with and without -DEXEC_BACKEND). Author: Michael Paquier Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/Y324HvGKiWxW2yxe@paquier.xyz
* Add error context callback when tokenizing authentication filesMichael Paquier2022-11-141-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | The parsing of the authentication files for HBA and ident entries happens in two phases: - Tokenization of the files, creating a list of TokenizedAuthLines. - Validation of the HBA and ident entries, building a set of HbaLines or IdentLines. The second phase doing the validation provides already some error context about the configuration file and the line where a problem happens, but there is no such information in the first phase when tokenizing the files. This commit adds an ErrorContextCallback in tokenize_auth_file(), with a context made of the line number and the configuration file name involved in a problem. This is useful for files included in an HBA file for user and database lists, and it will become much more handy to track problems for files included via a potential @include[_dir,_if_exists]. The error context is registered so as the full chain of events is reported when using cascaded inclusions when for example tokenize_auth_file() recurses over itself on new files, displaying one context line for each file gone through when tokenizing things. Author: Michael Paquier Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/Y2xUBJ+S+Z0zbxRW@paquier.xyz
* Invent open_auth_file() in hba.c to refactor authentication file openingMichael Paquier2022-11-141-28/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a check on the recursion depth when including authentication configuration files, something that has never been done when processing '@' files for database and user name lists in pg_hba.conf. On HEAD, this was leading to a rather confusing error, as of: FATAL: exceeded maxAllocatedDescs (NN) while trying to open file "/path/blah.conf" This refactors the code so as the error reported is now the following, which is the same as for GUCs: FATAL: could not open file "/path/blah.conf": maximum nesting depth exceeded This reduces a bit the verbosity of the error message used for files included in user and database lists, reporting only the file name of what's failing to load, without mentioning the relative or absolute path specified after '@' in a HBA file. The absolute path is built upon what '@' defines anyway, so there is no actual loss of information. This makes the future inclusion logic much simpler. A follow-up patch will add an error context to be able to track on which line of which file the inclusion is failing, to close the loop, providing all the information needed to know the full chain of events. This logic has been extracted from a larger patch written by Julien, rewritten by me to have a unique code path calling AllocateFile() on authentication files, and is useful on its own. This new interface will be used later for authentication files included with @include[_dir,_if_exists], in a follow-up patch. Author: Michael Paquier, Julien Rouhaud Discussion: https://www.postgresql.org/message-id/Y2xUBJ+S+Z0zbxRW@paquier.xyz
* Refactor ownercheck functionsPeter Eisentraut2022-11-131-1/+2
| | | | | | | | | | | | Instead of dozens of mostly-duplicate pg_foo_ownercheck() functions, write one common function object_ownercheck() that can handle almost all of them. We already have all the information we need, such as which system catalog corresponds to which catalog table and which column is the owner column. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Reviewed-by: Antonin Houska <ah@cybertec.at> Discussion: https://www.postgresql.org/message-id/flat/95c30f96-4060-2f48-98b5-a4392d3b6066@enterprisedb.com
* Add repalloc0 and repalloc0_arrayPeter Eisentraut2022-11-121-6/+3
| | | | | | | | These zero out the space added by repalloc. This is a common pattern that is quite hairy to code by hand. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/b66dfc89-9365-cb57-4e1f-b7d31813eeec@enterprisedb.com
* Provide sigaction() for Windows.Thomas Munro2022-11-091-9/+0
| | | | | | | | | | Commit 9abb2bfc left behind code to block signals inside signal handlers on Windows, because our signal porting layer didn't have sigaction(). Provide a minimal implementation that is capable of blocking signals, to get rid of platform differences. See also related commit c94ae9d8. Discussion: https://postgr.es/m/CA%2BhUKGKKKfcgx6jzok9AYenp2TNti_tfs8FMoJpL8%2B0Gsy%3D%3D_A%40mail.gmail.com
* Use AbsoluteConfigLocation() when building an included path in hba.cMichael Paquier2022-11-091-15/+2
| | | | | | | | | | | | | | | | | | | | | The code building an absolute path to a file included, as prefixed by '@' in authentication files, for user and database lists uses the same logic as for GUCs, except that it has no need to know about DataDir as there is always a calling file to rely to build the base directory path. The refactoring done in a1a7bb8 makes this move straight-forward, and unifies the code used for GUCs and authentication files, and the intention is to rely also on that for the upcoming patch to be able to include full files from HBA or ident files. Note that this gets rid of an inconsistency introduced in 370f909, that copied the logic coming from GUCs but applied it for files included in authentication files, where the result buffer given to join_path_components() must have a size of MAXPGPATH. Based on a double-check of the existing code, all the other callers of join_path_components() already do that, except the code path changed here. Discussion: https://postgr.es/m/Y2igk7q8OMpg+Yta@paquier.xyz
* Clean up some inconsistencies with GUC declarationsMichael Paquier2022-10-311-2/+2
| | | | | | | | | | | | | | | | | | | | This is similar to 7d25958, and this commit takes care of all the remaining inconsistencies between the initial value used in the C variable associated to a GUC and its default value stored in the GUC tables (as of pg_settings.boot_val). Some of the initial values of the GUCs updated rely on a compile-time default. These are refactored so as the GUC table and its C declaration use the same values. This makes everything consistent with other places, backend_flush_after, bgwriter_flush_after, port, checkpoint_flush_after doing so already, for example. Extracted from a larger patch by Peter Smith. The spots updated in the modules are from me. Author: Peter Smith, Michael Paquier Reviewed-by: Nathan Bossart, Tom Lane, Justin Pryzby Discussion: https://postgr.es/m/CAHut+PtHE0XSfjjRQ6D4v7+dqzCw=d+1a64ujra4EX8aoc_Z+w@mail.gmail.com
* Fix variable assignment thinko in hba.cMichael Paquier2022-10-261-1/+1
| | | | | | | | | | | The intention behind 1b73d0b was to limit the use of TokenizedAuthLine, but I have fat-fingered one location in parse_hba_line() when creating the HbaLine, where this should use the local variable and not the value coming from TokenizedAuthLine. This logic is the exactly the same, but let's be clean about all that on consistency grounds. Reported-by: Julien Rouhaud Discussion: https://postgr.es/m/20221026032730.k3sib5krgm7l6njk@jrouhaud
* Refactor code handling the names of files loaded in hba.cMichael Paquier2022-10-262-57/+63
| | | | | | | | | | | | | | | | | | | | | | | | This has the advantage to limit the presence of the GUC values hba_file and ident_file to the code paths where these files are loaded, easing the introduction of an upcoming feature aimed at adding inclusion logic for files and directories in HBA and ident files. Note that this needs the addition of the source file name to HbaLine, in addition to the line number, which is something needed by the backend in two places of auth.c (authentication failure details and auth_id log when log_connections is enabled). While on it, adjust a log generated on authentication failure to report the name of the actual HBA file on which the connection attempt matched, where the line number and the raw line written in the HBA file were already included. This was previously hardcoded as pg_hba.conf, which would be incorrect when a custom value is used at postmaster startup for the GUC hba_file. Extracted from a larger patch by the same author. Author: Julien Rouhaud Discussion: https://postgr.es/m/20220223045959.35ipdsvbxcstrhya@jrouhaud
* Add support for regexps on database and user entries in pg_hba.confMichael Paquier2022-10-241-7/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As of this commit, any database or user entry beginning with a slash (/) is considered as a regular expression. This is particularly useful for users, as now there is no clean way to match pattern on multiple HBA lines. For example, a user name mapping with a regular expression needs first to match with a HBA line, and we would skip the follow-up HBA entries if the ident regexp does *not* match with what has matched in the HBA line. pg_hba.conf is able to handle multiple databases and roles with a comma-separated list of these, hence individual regular expressions that include commas need to be double-quoted. At authentication time, user and database names are now checked in the following order: - Arbitrary keywords (like "all", the ones beginning by '+' for membership check), that we know will never have a regexp. A fancy case is for physical WAL senders, we *have* to only match "replication" for the database. - Regular expression matching. - Exact match. The previous logic did the same, but without the regexp step. We have discussed as well the possibility to support regexp pattern matching for host names, but these happen to lead to tricky issues based on what I understand, particularly with host entries that have CIDRs. This commit relies heavily on the refactoring done in a903971 and fc579e1, so as the amount of code required to compile and execute regular expressions is now minimal. When parsing pg_hba.conf, all the computed regexps needs to explicitely free()'d, same as pg_ident.conf. Documentation and TAP tests are added to cover this feature, including cases where the regexps use commas (for clarity in the docs, coverage for the parsing logic in the tests). Note that this introduces a breakage with older versions, where a database or user name beginning with a slash are treated as something to check for an equal match. Per discussion, we have discarded this as being much of an issue in practice as it would require a cluster to have database and/or role names that begin with a slash, as well as HBA entries using these. Hence, the consistency gained with regexps in pg_ident.conf is more appealing in the long term. **This compatibility change should be mentioned in the release notes.** Author: Bertrand Drouvot Reviewed-by: Jacob Champion, Tom Lane, Michael Paquier Discussion: https://postgr.es/m/fff0d7c1-8ad4-76a1-9db3-0ab6ec338bf7@amazon.com
* Refactor more logic for compilation of regular expressions in hba.cMichael Paquier2022-10-211-19/+25
| | | | | | | | | | | | | | | | | | | | It happens that the parts of hba.conf that are planned to be extended to support regular expressions would finish by using the same error message as the one used currently for pg_ident.conf when a regular expression cannot be compiled, as long as the routine centralizing the logic, regcomp_auth_token(), knows from which file the regexp comes from and its line location in the so-said file. This change makes the follow-up patches slightly simpler, and the logic remains the same. I suspect that this makes the proposal to add support for file inclusions in pg_ident.conf and pg_hba.conf slightly simpler, as well. Extracted from a larger patch by the same author. This is similar to the refactoring done in fc579e1. Author: Bertrand Drouvot Discussion: https://postgr.es/m/fff0d7c1-8ad4-76a1-9db3-0ab6ec338bf7@amazon.com
* Refactor regular expression handling in hba.cMichael Paquier2022-10-191-57/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | AuthToken gains a regular expression, and IdentLine is changed so as it uses an AuthToken rather than tracking separately the ident user string used for the regex compilation and its generated regex_t. In the case of pg_ident.conf, a set of AuthTokens is built in the pre-parsing phase of the file, and an extra regular expression is compiled when building the list of IdentLines, after checking the sanity of the fields in a pre-parsed entry. The logic in charge of computing and executing regular expressions is now done in a new set of routines called respectively regcomp_auth_token() and regexec_auth_token() that are wrappers around pg_regcomp() and pg_regexec(), working on AuthTokens. While on it, this patch adds a routine able to free an AuthToken, free_auth_token(), to simplify a bit the logic around the requirement of using a specific free routine for computed regular expressions. Note that there are no functional or behavior changes introduced by this commit. The goal of this patch is to ease the use of regular expressions with more items of pg_hba.conf (user list, database list, potentially hostnames) where AuthTokens are used extensively. This will be tackled later in a separate patch. Author: Bertrand Drouvot, Michael Paquier Discussion: https://postgr.es/m/fff0d7c1-8ad4-76a1-9db3-0ab6ec338bf7@amazon.com
* meson: Add initial version of meson based build systemAndres Freund2022-09-211-0/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Autoconf is showing its age, fewer and fewer contributors know how to wrangle it. Recursive make has a lot of hard to resolve dependency issues and slow incremental rebuilds. Our home-grown MSVC build system is hard to maintain for developers not using Windows and runs tests serially. While these and other issues could individually be addressed with incremental improvements, together they seem best addressed by moving to a more modern build system. After evaluating different build system choices, we chose to use meson, to a good degree based on the adoption by other open source projects. We decided that it's more realistic to commit a relatively early version of the new build system and mature it in tree. This commit adds an initial version of a meson based build system. It supports building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD, Solaris and Windows (however only gcc is supported on aix, solaris). For Windows/MSVC postgres can now be built with ninja (faster, particularly for incremental builds) and msbuild (supporting the visual studio GUI, but building slower). Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM bitcode generation, documentation adjustments) are done in subsequent commits requiring further review. Other aspects (e.g. not installing test-only extensions) are not yet addressed. When building on Windows with msbuild, builds are slower when using a visual studio version older than 2019, because those versions do not support MultiToolTask, required by meson for intra-target parallelism. The plan is to remove the MSVC specific build system in src/tools/msvc soon after reaching feature parity. However, we're not planning to remove the autoconf/make build system in the near future. Likely we're going to keep at least the parts required for PGXS to keep working around until all supported versions build with meson. Some initial help for postgres developers is at https://wiki.postgresql.org/wiki/Meson With contributions from Thomas Munro, John Naylor, Stone Tickle and others. Author: Andres Freund <andres@anarazel.de> Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Author: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de
* Harmonize more parameter names in bulk.Peter Geoghegan2022-09-201-2/+2
| | | | | | | | | | | | | | | | Make sure that function declarations use names that exactly match the corresponding names from function definitions in optimizer, parser, utility, libpq, and "commands" code, as well as in remaining library code. Do the same for all code related to frontend programs (with the exception of pg_dump/pg_dumpall related code). Like other recent commits that cleaned up function parameter names, this commit was written with help from clang-tidy. Later commits will handle ecpg and pg_dump/pg_dumpall. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAH2-WznJt9CMM9KJTMjJh_zbL5hD9oX44qdJ4aqZtjFi-zA3Tg@mail.gmail.com
* Split up guc.c for better build speed and ease of maintenance.Tom Lane2022-09-131-1/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | guc.c has grown to be one of our largest .c files, making it a bottleneck for compilation. It's also acquired a bunch of knowledge that'd be better kept elsewhere, because of our not very good habit of putting variable-specific check hooks here. Hence, split it up along these lines: * guc.c itself retains just the core GUC housekeeping mechanisms. * New file guc_funcs.c contains the SET/SHOW interfaces and some SQL-accessible functions for GUC manipulation. * New file guc_tables.c contains the data arrays that define the built-in GUC variables, along with some already-exported constant tables. * GUC check/assign/show hook functions are moved to the variable's home module, whenever that's clearly identifiable. A few hard- to-classify hooks ended up in commands/variable.c, which was already a home for miscellaneous GUC hook functions. To avoid cluttering a lot more header files with #include "guc.h", I also invented a new header file utils/guc_hooks.h and put all the GUC hook functions' declarations there, regardless of their originating module. That allowed removal of #include "guc.h" from some existing headers. The fallout from that (hopefully all caught here) demonstrates clearly why such inclusions are best minimized: there are a lot of files that, for example, were getting array.h at two or more levels of remove, despite not having any connection at all to GUCs in themselves. There is some very minor code beautification here, such as renaming a couple of inconsistently-named hook functions and improving some comments. But mostly this just moves code from point A to point B and deals with the ensuing needs for #include adjustments and exporting a few functions that previously weren't exported. Patch by me, per a suggestion from Andres Freund; thanks also to Michael Paquier for the idea to invent guc_funcs.c. Discussion: https://postgr.es/m/587607.1662836699@sss.pgh.pa.us
* Don't reflect unescaped cert data to the logsPeter Eisentraut2022-09-131-26/+31
| | | | | | | | | | | | | | | | | | | | | | Commit 3a0e385048 introduced a new path for unauthenticated bytes from the client certificate to be printed unescaped to the logs. There are a handful of these already, but it doesn't make sense to keep making the problem worse. \x-escape any unprintable bytes. The test case introduces a revoked UTF-8 certificate. This requires the addition of the `-utf8` flag to `openssl req`. Since the existing certificates all use an ASCII subset, this won't modify the existing certificates' subjects if/when they get regenerated; this was verified experimentally with $ make sslfiles-clean $ make sslfiles Unfortunately the test can't be run in the CI yet due to a test timing issue; see 55828a6b60. Author: Jacob Champion <jchampion@timescale.com> Discussion: https://www.postgresql.org/message-id/CAAWbhmgsvHrH9wLU2kYc3pOi1KSenHSLAHBbCVmmddW6-mc_=w@mail.gmail.com
* Replace loading of ldap_start_tls_sA() by direct function callMichael Paquier2022-09-121-50/+1
| | | | | | | | | | | | | | | This change impacts the backend-side code in charge of starting a LDAP TLS session. It is a bit sad that it is not possible to unify the WIN32 and non-WIN32 code paths, but the different number of arguments for both discard this possibility. This is similar to 47bd0b3, where this replaces the last function loading that seems worth it, any others being either environment or version-dependent. Reported-by: Thomas Munro Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/Yx0rxpNgDh8tN4XA@paquier.xyz
* Free correctly LDAPMessage returned by ldap_search_s() in auth.cMichael Paquier2022-09-101-0/+3
| | | | | | | | | | | | | | | | The LDAP wiki states that the search message should be freed regardless of the return value of ldap_search_s(), but we failed to do so in one backend code path when searching LDAP with a filter. This is not critical in an authentication code path failing in the backend as this causes such the process to exit promptly, but let's be clean and free the search message appropriately, as documented by upstream. All the other code paths failing a LDAP operation do that already, and somebody looking at this code in the future may miss what LDAP expects with the search message. Author: Zhihong Yu Discussion: https://postgr.es/m/CALNJ-vTf5Y+8RtzZ4GjOGE9qWVHZ8awfhnFYc_qGm8fMLUNRAg@mail.gmail.com
* Replace load of functions by direct calls for some WIN32Michael Paquier2022-09-091-28/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit changes the following code paths to do direct system calls to some WIN32 functions rather than loading them from an external library, shaving some code in the process: - Creation of restricted tokens in pg_ctl.c, introduced by a25cd81. - QuerySecurityContextToken() in auth.c for SSPI authentication in the backend, introduced in d602592. - CreateRestrictedToken() in src/common/. This change is similar to the case of pg_ctl.c. Most of these functions were loaded rather than directly called because, as mentioned in the code comments, MinGW headers were not declaring them. I have double-checked the recent MinGW code, and all the functions changed here are declared in its headers, so this change should be safe. Note that I do not have a MinGW environment at hand so I have not tested it directly, but that MSVC was fine with the change. The buildfarm will tell soon enough if this change is appropriate or not for a much broader set of environments. A few code paths still use GetProcAddress() to load some functions: - LDAP authentication for ldap_start_tls_sA(), where I am not confident that this change would work. - win32env.c and win32ntdll.c where we have a per-MSVC version dependency for the name of the library loaded. - crashdump.c for MiniDumpWriteDump() and EnumDirTree(), where direct calls were not able to work after testing. Reported-by: Thomas Munro Reviewed-by: Justin Prysby Discussion: https://postgr.es/m/CA+hUKG+BMdcaCe=P-EjMoLTCr3zrrzqbcVE=8h5LyNsSVHKXZA@mail.gmail.com
* Fix an assortment of improper usages of string functionsDavid Rowley2022-09-061-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | In a similar effort to f736e188c and 110d81728, fixup various usages of string functions where a more appropriate function is available and more fit for purpose. These changes include: 1. Use cstring_to_text_with_len() instead of cstring_to_text() when working with a StringInfoData and the length can easily be obtained. 2. Use appendStringInfoString() instead of appendStringInfo() when no formatting is required. 3. Use pstrdup(...) instead of psprintf("%s", ...) 4. Use pstrdup(...) instead of psprintf(...) (with no formatting) 5. Use appendPQExpBufferChar() instead of appendPQExpBufferStr() when the length of the string being appended is 1. 6. appendStringInfoChar() instead of appendStringInfo() when no formatting is required and string is 1 char long. 7. Use appendPQExpBufferStr(b, .) instead of appendPQExpBuffer(b, "%s", .) 8. Don't use pstrdup when it's fine to just point to the string constant. I (David) did find other cases of #8 but opted to use #4 instead as I wasn't certain enough that applying #8 was ok (e.g in hba.c) Author: Ranier Vilela, David Rowley Discussion: https://postgr.es/m/CAApHDvo2j2+RJBGhNtUz6BxabWWh2Jx16wMUMWKUjv70Ver1vg@mail.gmail.com
View Lorry Controller Status