<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/postgresql.git/src/include/statistics, branch master</title>
<subtitle>git.postgresql.org: git/postgresql.git
</subtitle>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/'/>
<entry>
<title>Update copyright for 2023</title>
<updated>2023-01-02T20:00:37+00:00</updated>
<author>
<name>Bruce Momjian</name>
<email>bruce@momjian.us</email>
</author>
<published>2023-01-02T20:00:37+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=c8e1ba736b2b9e8c98d37a5b77c4ed31baf94147'/>
<id>c8e1ba736b2b9e8c98d37a5b77c4ed31baf94147</id>
<content type='text'>
Backpatch-through: 11
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Backpatch-through: 11
</pre>
</div>
</content>
</entry>
<entry>
<title>Harmonize more parameter names in bulk.</title>
<updated>2022-09-20T20:09:30+00:00</updated>
<author>
<name>Peter Geoghegan</name>
<email>pg@bowt.ie</email>
</author>
<published>2022-09-20T20:09:30+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=a601366a460f68472bf70c4d94c57baa0a3ed1b2'/>
<id>a601366a460f68472bf70c4d94c57baa0a3ed1b2</id>
<content type='text'>
Make sure that function declarations use names that exactly match the
corresponding names from function definitions in optimizer, parser,
utility, libpq, and "commands" code, as well as in remaining library
code.  Do the same for all code related to frontend programs (with the
exception of pg_dump/pg_dumpall related code).

Like other recent commits that cleaned up function parameter names, this
commit was written with help from clang-tidy.  Later commits will handle
ecpg and pg_dump/pg_dumpall.

Author: Peter Geoghegan &lt;pg@bowt.ie&gt;
Reviewed-By: David Rowley &lt;dgrowleyml@gmail.com&gt;
Discussion: https://postgr.es/m/CAH2-WznJt9CMM9KJTMjJh_zbL5hD9oX44qdJ4aqZtjFi-zA3Tg@mail.gmail.com
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Make sure that function declarations use names that exactly match the
corresponding names from function definitions in optimizer, parser,
utility, libpq, and "commands" code, as well as in remaining library
code.  Do the same for all code related to frontend programs (with the
exception of pg_dump/pg_dumpall related code).

Like other recent commits that cleaned up function parameter names, this
commit was written with help from clang-tidy.  Later commits will handle
ecpg and pg_dump/pg_dumpall.

Author: Peter Geoghegan &lt;pg@bowt.ie&gt;
Reviewed-By: David Rowley &lt;dgrowleyml@gmail.com&gt;
Discussion: https://postgr.es/m/CAH2-WznJt9CMM9KJTMjJh_zbL5hD9oX44qdJ4aqZtjFi-zA3Tg@mail.gmail.com
</pre>
</div>
</content>
</entry>
<entry>
<title>Add stxdinherit flag to pg_statistic_ext_data</title>
<updated>2022-01-16T12:38:01+00:00</updated>
<author>
<name>Tomas Vondra</name>
<email>tomas.vondra@postgresql.org</email>
</author>
<published>2022-01-16T12:37:56+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=269b532aef55a579ae02a3e8e8df14101570dfd9'/>
<id>269b532aef55a579ae02a3e8e8df14101570dfd9</id>
<content type='text'>
Add pg_statistic_ext_data.stxdinherit flag, so that for each extended
statistics definition we can store two versions of data - one for the
relation alone, one for the whole inheritance tree. This is analogous to
pg_statistic.stainherit, but we failed to include such flag in catalogs
for extended statistics, and we had to work around it (see commits
859b3003de, 36c4bc6e72 and 20b9fa308e).

This changes the relationship between the two catalogs storing extended
statistics objects (pg_statistic_ext and pg_statistic_ext_data). Until
now, there was a simple 1:1 mapping - for each definition there was one
pg_statistic_ext_data row, and this row was inserted while creating the
statistics (and then updated during ANALYZE). With the stxdinherit flag,
we don't know how many rows there will be (child relations may be added
after the statistics object is defined), so there may be up to two rows.

We could make CREATE STATISTICS to always create both rows, but that
seems wasteful - without partitioning we only need stxdinherit=false
rows, and declaratively partitioned tables need only stxdinherit=true.
So we no longer initialize pg_statistic_ext_data in CREATE STATISTICS,
and instead make that a responsibility of ANALYZE. Which is what we do
for regular statistics too.

Patch by me, with extensive improvements and fixes by Justin Pryzby.

Author: Tomas Vondra, Justin Pryzby
Reviewed-by: Tomas Vondra, Justin Pryzby
Discussion: https://postgr.es/m/20210923212624.GI831%40telsasoft.com
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Add pg_statistic_ext_data.stxdinherit flag, so that for each extended
statistics definition we can store two versions of data - one for the
relation alone, one for the whole inheritance tree. This is analogous to
pg_statistic.stainherit, but we failed to include such flag in catalogs
for extended statistics, and we had to work around it (see commits
859b3003de, 36c4bc6e72 and 20b9fa308e).

This changes the relationship between the two catalogs storing extended
statistics objects (pg_statistic_ext and pg_statistic_ext_data). Until
now, there was a simple 1:1 mapping - for each definition there was one
pg_statistic_ext_data row, and this row was inserted while creating the
statistics (and then updated during ANALYZE). With the stxdinherit flag,
we don't know how many rows there will be (child relations may be added
after the statistics object is defined), so there may be up to two rows.

We could make CREATE STATISTICS to always create both rows, but that
seems wasteful - without partitioning we only need stxdinherit=false
rows, and declaratively partitioned tables need only stxdinherit=true.
So we no longer initialize pg_statistic_ext_data in CREATE STATISTICS,
and instead make that a responsibility of ANALYZE. Which is what we do
for regular statistics too.

Patch by me, with extensive improvements and fixes by Justin Pryzby.

Author: Tomas Vondra, Justin Pryzby
Reviewed-by: Tomas Vondra, Justin Pryzby
Discussion: https://postgr.es/m/20210923212624.GI831%40telsasoft.com
</pre>
</div>
</content>
</entry>
<entry>
<title>Update copyright for 2022</title>
<updated>2022-01-08T00:04:57+00:00</updated>
<author>
<name>Bruce Momjian</name>
<email>bruce@momjian.us</email>
</author>
<published>2022-01-08T00:04:57+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=27b77ecf9f4d5be211900eda54d8155ada50d696'/>
<id>27b77ecf9f4d5be211900eda54d8155ada50d696</id>
<content type='text'>
Backpatch-through: 10
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Backpatch-through: 10
</pre>
</div>
</content>
</entry>
<entry>
<title>Extended statistics on expressions</title>
<updated>2021-03-26T23:01:11+00:00</updated>
<author>
<name>Tomas Vondra</name>
<email>tomas.vondra@postgresql.org</email>
</author>
<published>2021-03-26T22:22:01+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=a4d75c86bf15220df22de0a92c819ecef9db3849'/>
<id>a4d75c86bf15220df22de0a92c819ecef9db3849</id>
<content type='text'>
Allow defining extended statistics on expressions, not just just on
simple column references.  With this commit, expressions are supported
by all existing extended statistics kinds, improving the same types of
estimates. A simple example may look like this:

  CREATE TABLE t (a int);
  CREATE STATISTICS s ON mod(a,10), mod(a,20) FROM t;
  ANALYZE t;

The collected statistics are useful e.g. to estimate queries with those
expressions in WHERE or GROUP BY clauses:

  SELECT * FROM t WHERE mod(a,10) = 0 AND mod(a,20) = 0;

  SELECT 1 FROM t GROUP BY mod(a,10), mod(a,20);

This introduces new internal statistics kind 'e' (expressions) which is
built automatically when the statistics object definition includes any
expressions. This represents single-expression statistics, as if there
was an expression index (but without the index maintenance overhead).
The statistics is stored in pg_statistics_ext_data as an array of
composite types, which is possible thanks to 79f6a942bd.

CREATE STATISTICS allows building statistics on a single expression, in
which case in which case it's not possible to specify statistics kinds.

A new system view pg_stats_ext_exprs can be used to display expression
statistics, similarly to pg_stats and pg_stats_ext views.

ALTER TABLE ... ALTER COLUMN ... TYPE now treats indexes the same way it
treats indexes, i.e. it drops and recreates the statistics. This means
all statistics are reset, and we no longer try to preserve at least the
functional dependencies. This should not be a major issue in practice,
as the functional dependencies actually rely on per-column statistics,
which were always reset anyway.

Author: Tomas Vondra
Reviewed-by: Justin Pryzby, Dean Rasheed, Zhihong Yu
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Allow defining extended statistics on expressions, not just just on
simple column references.  With this commit, expressions are supported
by all existing extended statistics kinds, improving the same types of
estimates. A simple example may look like this:

  CREATE TABLE t (a int);
  CREATE STATISTICS s ON mod(a,10), mod(a,20) FROM t;
  ANALYZE t;

The collected statistics are useful e.g. to estimate queries with those
expressions in WHERE or GROUP BY clauses:

  SELECT * FROM t WHERE mod(a,10) = 0 AND mod(a,20) = 0;

  SELECT 1 FROM t GROUP BY mod(a,10), mod(a,20);

This introduces new internal statistics kind 'e' (expressions) which is
built automatically when the statistics object definition includes any
expressions. This represents single-expression statistics, as if there
was an expression index (but without the index maintenance overhead).
The statistics is stored in pg_statistics_ext_data as an array of
composite types, which is possible thanks to 79f6a942bd.

CREATE STATISTICS allows building statistics on a single expression, in
which case in which case it's not possible to specify statistics kinds.

A new system view pg_stats_ext_exprs can be used to display expression
statistics, similarly to pg_stats and pg_stats_ext views.

ALTER TABLE ... ALTER COLUMN ... TYPE now treats indexes the same way it
treats indexes, i.e. it drops and recreates the statistics. This means
all statistics are reset, and we no longer try to preserve at least the
functional dependencies. This should not be a major issue in practice,
as the functional dependencies actually rely on per-column statistics,
which were always reset anyway.

Author: Tomas Vondra
Reviewed-by: Justin Pryzby, Dean Rasheed, Zhihong Yu
Discussion: https://postgr.es/m/ad7891d2-e90c-b446-9fe2-7419143847d7%40enterprisedb.com
</pre>
</div>
</content>
</entry>
<entry>
<title>Move bsearch_arg to src/port</title>
<updated>2021-03-22T23:11:22+00:00</updated>
<author>
<name>Tomas Vondra</name>
<email>tomas.vondra@postgresql.org</email>
</author>
<published>2021-03-22T23:11:20+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=bfa2cee784125047771db2768fcf7f04d8bd6bb4'/>
<id>bfa2cee784125047771db2768fcf7f04d8bd6bb4</id>
<content type='text'>
Until now the bsearch_arg function was used only in extended statistics
code, so it was defined in that code.  But we already have qsort_arg in
src/port, so let's move it next to it.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Until now the bsearch_arg function was used only in extended statistics
code, so it was defined in that code.  But we already have qsort_arg in
src/port, so let's move it next to it.
</pre>
</div>
</content>
</entry>
<entry>
<title>Update copyright for 2021</title>
<updated>2021-01-02T18:06:25+00:00</updated>
<author>
<name>Bruce Momjian</name>
<email>bruce@momjian.us</email>
</author>
<published>2021-01-02T18:06:25+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=ca3b37487be333a1d241dab1bbdd17a211a88f43'/>
<id>ca3b37487be333a1d241dab1bbdd17a211a88f43</id>
<content type='text'>
Backpatch-through: 9.5
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Backpatch-through: 9.5
</pre>
</div>
</content>
</entry>
<entry>
<title>Improve estimation of OR clauses using extended statistics.</title>
<updated>2020-12-03T10:03:49+00:00</updated>
<author>
<name>Dean Rasheed</name>
<email>dean.a.rasheed@gmail.com</email>
</author>
<published>2020-12-03T10:03:49+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=25a9e54d2db31b8031c2d8166114b187e8347098'/>
<id>25a9e54d2db31b8031c2d8166114b187e8347098</id>
<content type='text'>
Formerly we only applied extended statistics to an OR clause as part
of the clauselist_selectivity() code path for an OR clause appearing
in an implicitly-ANDed list of clauses. This meant that it could only
use extended statistics if all sub-clauses of the OR clause were
covered by a single extended statistics object.

Instead, teach clause_selectivity() how to apply extended statistics
to an OR clause by handling its ORed list of sub-clauses in a similar
manner to an implicitly-ANDed list of sub-clauses, but with different
combination rules. This allows one or more extended statistics objects
to be used to estimate all or part of the list of sub-clauses. Any
remaining sub-clauses are then treated as if they are independent.

Additionally, to avoid double-application of extended statistics, this
introduces "extended" versions of clause_selectivity() and
clauselist_selectivity(), which include an option to ignore extended
statistics. This replaces the old clauselist_selectivity_simple()
function which failed to completely ignore extended statistics when
called from the extended statistics code.

A known limitation of the current infrastructure is that an AND clause
under an OR clause is not treated as compatible with extended
statistics (because we don't build RestrictInfos for such sub-AND
clauses). Thus, for example, "(a=1 AND b=1) OR (a=2 AND b=2)" will
currently be treated as two independent AND clauses (each of which may
be estimated using extended statistics), but extended statistics will
not currently be used to account for any possible overlap between
those clauses. Improving that is left as a task for the future.

Original patch by Tomas Vondra, with additional improvements by me.

Discussion: https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Formerly we only applied extended statistics to an OR clause as part
of the clauselist_selectivity() code path for an OR clause appearing
in an implicitly-ANDed list of clauses. This meant that it could only
use extended statistics if all sub-clauses of the OR clause were
covered by a single extended statistics object.

Instead, teach clause_selectivity() how to apply extended statistics
to an OR clause by handling its ORed list of sub-clauses in a similar
manner to an implicitly-ANDed list of sub-clauses, but with different
combination rules. This allows one or more extended statistics objects
to be used to estimate all or part of the list of sub-clauses. Any
remaining sub-clauses are then treated as if they are independent.

Additionally, to avoid double-application of extended statistics, this
introduces "extended" versions of clause_selectivity() and
clauselist_selectivity(), which include an option to ignore extended
statistics. This replaces the old clauselist_selectivity_simple()
function which failed to completely ignore extended statistics when
called from the extended statistics code.

A known limitation of the current infrastructure is that an AND clause
under an OR clause is not treated as compatible with extended
statistics (because we don't build RestrictInfos for such sub-AND
clauses). Thus, for example, "(a=1 AND b=1) OR (a=2 AND b=2)" will
currently be treated as two independent AND clauses (each of which may
be estimated using extended statistics), but extended statistics will
not currently be used to account for any possible overlap between
those clauses. Improving that is left as a task for the future.

Original patch by Tomas Vondra, with additional improvements by me.

Discussion: https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development
</pre>
</div>
</content>
</entry>
<entry>
<title>Initial pgindent and pgperltidy run for v13.</title>
<updated>2020-05-14T17:06:50+00:00</updated>
<author>
<name>Tom Lane</name>
<email>tgl@sss.pgh.pa.us</email>
</author>
<published>2020-05-14T17:06:38+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=5cbfce562f7cd2aab0cdc4694ce298ec3567930e'/>
<id>5cbfce562f7cd2aab0cdc4694ce298ec3567930e</id>
<content type='text'>
Includes some manual cleanup of places that pgindent messed up,
most of which weren't per project style anyway.

Notably, it seems some people didn't absorb the style rules of
commit c9d297751, because there were a bunch of new occurrences
of function calls with a newline just after the left paren, all
with faulty expectations about how the rest of the call would get
indented.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Includes some manual cleanup of places that pgindent messed up,
most of which weren't per project style anyway.

Notably, it seems some people didn't absorb the style rules of
commit c9d297751, because there were a bunch of new occurrences
of function calls with a newline just after the left paren, all
with faulty expectations about how the rest of the call would get
indented.
</pre>
</div>
</content>
</entry>
<entry>
<title>Use multi-variate MCV lists to estimate ScalarArrayOpExpr</title>
<updated>2020-03-14T15:13:00+00:00</updated>
<author>
<name>Tomas Vondra</name>
<email>tomas.vondra@postgresql.org</email>
</author>
<published>2020-03-14T13:56:37+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/postgresql.git/commit/?id=e83daa7e331e0718bb254ab685f2ed95df423554'/>
<id>e83daa7e331e0718bb254ab685f2ed95df423554</id>
<content type='text'>
Commit 8f321bd16c added support for estimating ScalarArrayOpExpr clauses
(IN/ANY) clauses using functional dependencies. There's no good reason
not to support estimation of these clauses using multi-variate MCV lists
too, so this commits implements that. That makes the behavior consistent
and MCV lists can estimate all variants (ANY/ALL, inequalities, ...).

Author: Tomas Vondra
Review: Dean Rasheed
Discussion: https://www.postgresql.org/message-id/flat/13902317.Eha0YfKkKy%40pierred-pdoc
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Commit 8f321bd16c added support for estimating ScalarArrayOpExpr clauses
(IN/ANY) clauses using functional dependencies. There's no good reason
not to support estimation of these clauses using multi-variate MCV lists
too, so this commits implements that. That makes the behavior consistent
and MCV lists can estimate all variants (ANY/ALL, inequalities, ...).

Author: Tomas Vondra
Review: Dean Rasheed
Discussion: https://www.postgresql.org/message-id/flat/13902317.Eha0YfKkKy%40pierred-pdoc
</pre>
</div>
</content>
</entry>
</feed>
