| From: |
| PWN via PostgreSQL Announce <announce-noreply-AT-postgresql.org> |
| To: |
| PostgreSQL Announce <pgsql-announce-AT-lists.postgresql.org> |
| Subject: |
| PostgreSQL Weekly News - January 24, 2021 |
| Date: |
| Mon, 25 Jan 2021 13:30:44 +0000 |
| Message-ID: |
| <161158144427.14625.13221911530868763723@wrigleys.postgresql.org> |
| Archive-link: |
| Article |
# PostgreSQL Weekly News - January 24, 2021
Person of the week:
[https://postgresql.life/post/marc_linster/](https://postgresql.life/post/marc_linster/)
# PostgreSQL Product News
sqlite_fdw 1.3.1 released.
[https://github.com/pgspider/sqlite_fdw](https://github.com/pgspider/sqlite_fdw)
InfluxDB fdw 0.3 released
[https://github.com/pgspider/influxdb_fdw](https://github.com/pgspider/influxdb_fdw)
griddb_fdw 1.3 released.
[https://github.com/pgspider/griddb_fdw](https://github.com/pgspider/griddb_fdw)
pg_activity 2.0.0, a top-like application for PostgreSQL server activity
monitoring, released.
[https://github.com/dalibo/pg_activity/releases/tag/v2.0.0](https://github.com/dalibo/pg_activity/releases/tag/v2.0.0)
# PostgreSQL Jobs for January
[https://archives.postgresql.org/pgsql-jobs/2021-01/](https://archives.postgresql.org/pgsql-jobs/2021-01/)
# PostgreSQL in the News
Planet PostgreSQL: [https://planet.postgresql.org/](https://planet.postgresql.org/)
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org.
# Applied Patches
Tomáš Vondra pushed:
- Set `PD_ALL_VISIBLE` and visibility map bits in COPY FREEZE. Make sure COPY
FREEZE marks the pages as `PD_ALL_VISIBLE` and updates the visibility map. Until
now we only marked individual tuples as frozen, but page-level flags were not
updated, so the first VACUUM after the COPY FREEZE had to rewrite the whole
table. This is a fairly old patch, and multiple people worked on it. The
first version was written by Jeff Janes, and then reworked by Pavan Deolasee
and Anastasia Lubennikova. Author: Anastasia Lubennikova, Pavan Deolasee,
Jeff Janes Reviewed-by: Kuntal Ghosh, Jeff Janes, Tomas Vondra, Masahiko
Sawada, Andres Freund, Ibrar Ahmed, Robert Haas, Tatsuro Ishii,
Darafei Praliaskouski Discussion:
[https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmk...](https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmk...)
Discussion:
[https://postgr.es/m/CAMkU%3D1w3osJJ2FneELhhNRLxfZitDgp9FP...](https://postgr.es/m/CAMkU%3D1w3osJJ2FneELhhNRLxfZitDgp9FP...)
[https://git.postgresql.org/pg/commitdiff/7db0cd2145f2bce8...](https://git.postgresql.org/pg/commitdiff/7db0cd2145f2bce8...)
- psql \dX: list extended statistics objects. The new command lists extended
statistics objects. All past releases with extended statistics are supported.
This is a simplified version of commit 891a1d0bca, which had to be reverted
due to not considering pg_statistic_ext_data is not accessible by regular
users. Fields requiring access to this catalog were removed. It's possible to
add them, but it'll require changes to core. Author: Tatsuro Yamada
Reviewed-by: Julien Rouhaud, Alvaro Herrera, Tomas Vondra, Noriyoshi Shinoda
Discussion:
[https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%...](https://postgr.es/m/c027a541-5856-75a5-0868-341301e1624b%...)
[https://git.postgresql.org/pg/commitdiff/ad600bba0422dde4...](https://git.postgresql.org/pg/commitdiff/ad600bba0422dde4...)
- Implement support for bulk inserts in postgres_fdw. Extends the FDW API to
allow batching inserts into foreign tables. That is usually much more
efficient than inserting individual rows, due to high latency for each
round-trip to the foreign server. It was possible to implement something
similar in the regular FDW API, but it was inconvenient and there were issues
with reporting the number of actually inserted rows etc. This extends the FDW
API with two new functions: * GetForeignModifyBatchSize - allows the FDW
picking optimal batch size * ExecForeignBatchInsert - inserts a batch of rows
at once Currently, only INSERT queries support batching. Support for DELETE
and UPDATE may be added in the future. This also implements batching for
postgres_fdw. The batch size may be specified using "batch_size" option both
at the server and table level. The initial patch version was written by me,
but it was rewritten and improved in many ways by Takayuki Tsunakawa. Author:
Takayuki Tsunakawa Reviewed-by: Tomas Vondra, Amit Langote Discussion:
[https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@devel...](https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@devel...)
[https://git.postgresql.org/pg/commitdiff/b663a4136331de6c...](https://git.postgresql.org/pg/commitdiff/b663a4136331de6c...)
- Fix initialization of FDW batching in ExecInitModifyTable. ExecInitModifyTable
has to initialize batching for all result relations, not just the first one.
Furthermore, when junk filters were necessary, the pointer pointed past the
`mtstate->resultRelInfo` array. Per reports from multiple non-x86 animals
(florican, locust, ...). Discussion:
[https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@devel...](https://postgr.es/m/20200628151002.7x5laxwpgvkyiu3q@devel...)
[https://git.postgresql.org/pg/commitdiff/920f853dc948b98a...](https://git.postgresql.org/pg/commitdiff/920f853dc948b98a...)
- Fix `COPY FREEZE` with `CLOBBER_CACHE_ALWAYS. This adds code omitted from commit
7db0cd2145 by accident, which had two consequences. Firstly, only rows
inserted by heap_multi_insert were frozen as expected when running COPY
FREEZE, while heap_insert left rows unfrozen. That however includes rows in
TOAST tables, so a lot of data might have been left unfrozen. Secondly, page
might have been left partially empty after relcache invalidation. This
addresses both of those issues. Discussion:
[https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmk...](https://postgr.es/m/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmk...)
[https://git.postgresql.org/pg/commitdiff/39b66a91bdebb00a...](https://git.postgresql.org/pg/commitdiff/39b66a91bdebb00a...)
Heikki Linnakangas pushed:
- pageinspect: Fix relcache leak in gist_page_items(). The gist_page_items()
function opened the index relation on first call and closed it on the last
call. But there's no guarantee that the function is run to completion, leading
to a relcache leak and warning at the end of the transaction. To fix, refactor
the function to return all the rows in one call, as a tuplestore.
Reported-by: Tom Lane Discussion:
[https://www.postgresql.org/message-id/234863.1610916631%4...](https://www.postgresql.org/message-id/234863.1610916631%4...)
[https://git.postgresql.org/pg/commitdiff/04eb75e783ba49ca...](https://git.postgresql.org/pg/commitdiff/04eb75e783ba49ca...)
- Check for BuildIndexValueDescription returning NULL in gist_page_items. Per
Coverity. BuildIndexValueDescription() cannot actually return NULL in this
instance, because it only returns NULL if the user doesn't have the required
privileges, and this function can only be used by superuser. But better safe
than sorry.
[https://git.postgresql.org/pg/commitdiff/5d1e5c8b75877018...](https://git.postgresql.org/pg/commitdiff/5d1e5c8b75877018...)
- Fix bug in detecting concurrent page splits in GiST insert. In commit
9eb5607e699, I got the condition on checking for split or deleted page wrong:
I used && instead of ||. The comment correctly said "concurrent split _or_
deletion". As a result, GiST insertion could miss a concurrent split, and
insert to wrong page. Duncan Sands demonstrated this with a test script that
did a lot of concurrent inserts. Backpatch to v12, where this was introduced.
REINDEX is required to fix indexes that were affected by this bug.
Backpatch-through: 12 Reported-by: Duncan Sands Discussion:
[https://www.postgresql.org/message-id/a9690483-6c6c-3c82-...](https://www.postgresql.org/message-id/a9690483-6c6c-3c82-...)
[https://git.postgresql.org/pg/commitdiff/6b4d3046f422c268...](https://git.postgresql.org/pg/commitdiff/6b4d3046f422c268...)
- doc: Copy-edit the "Overview of PostgreSQL Internals" chapter. Rephrase a few
sentences to be more concise. Refer to the postmaster process as
"postmaster", not "postgres". This originally said "postmaster process", but
was changed to "postgres process" in commit 5266f221a2, when we merged the
"postmaster" and "postgres" commands, and "postmaster" became just a symlink.
That was a case of overzealous search & replace, because the process is still
called "postmaster". Author: Erik Rijkers and Jürgen Purtz Discussion:
[https://www.postgresql.org/message-id/aa31f359-1168-ded5-...](https://www.postgresql.org/message-id/aa31f359-1168-ded5-...)
[https://git.postgresql.org/pg/commitdiff/29ad6595ef7f568c...](https://git.postgresql.org/pg/commitdiff/29ad6595ef7f568c...)
Michaël Paquier pushed:
- Refactor option handling of CLUSTER, REINDEX and VACUUM. This continues the
work done in b5913f6. All the options of those commands are changed to use
hex values rather than enums to reduce the risk of compatibility bugs when
introducing new options. Each option set is moved into a new structure that
can be extended with more non-boolean options (this was already the case of
VACUUM). The code of REINDEX is restructured so as manual REINDEX commands go
through a single routine from utility.c, like VACUUM, to ease the allocation
handling of option parameters when a command needs to go through multiple
transactions. This can be used as a base infrastructure for future patches
related to those commands, including reindex filtering and tablespace support.
Per discussion with people mentioned below, as well as Alvaro Herrera and
Peter Eisentraut. Author: Michael Paquier, Justin Pryzby Reviewed-by: Alexey
Kondratov, Justin Pryzby Discussion:
[https://postgr.es/m/X8riynBLwxAD9uKk@paquier.xyz](https://postgr.es/m/X8riynBLwxAD9uKk@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/a3dc926009be833e...](https://git.postgresql.org/pg/commitdiff/a3dc926009be833e...)
- Fix ALTER DEFAULT PRIVILEGES with duplicated objects. Specifying duplicated
objects in this command would lead to unique constraint violations in
pg_default_acl or "tuple already updated by self" errors. Similarly to
GRANT/REVOKE, increment the command ID after each subcommand processing to
allow this case to work transparently. A regression test is added by tweaking
one of the existing queries of privileges.sql to stress this case.
Reported-by: Andrus Author: Michael Paquier Reviewed-by: Álvaro Herrera
Discussion:
[https://postgr.es/m/ae2a7dc1-9d71-8cba-3bb9-e4cb7eb1f44e@...](https://postgr.es/m/ae2a7dc1-9d71-8cba-3bb9-e4cb7eb1f44e@...)
Backpatch-through: 9.5
[https://git.postgresql.org/pg/commitdiff/21378e1fefedcaed...](https://git.postgresql.org/pg/commitdiff/21378e1fefedcaed...)
- Add regression test for DROP OWNED BY with default ACLs. DROP OWNED BY has a
specific code path to remove ACLs stored in pg_default_acl when cleaning up
shared dependencies that had no coverage with the existing tests. This issue
has been found while digging into the bug fixed by 21378e1. As ALTER DEFAULT
PRIVILEGES impacts the ACLs of all objects created while the default
permissions are visible, the test uses a transaction rollback to isolate the
test and avoid any impact with other sessions running in parallel.
Reviewed-by: Álvaro Herrera Discussion:
[https://postgr.es/m/YAbQ1OD+3ip4lRv8@paquier.xyz](https://postgr.es/m/YAbQ1OD+3ip4lRv8@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/a36dc04d424a6bfa...](https://git.postgresql.org/pg/commitdiff/a36dc04d424a6bfa...)
- Switch "cl /?" to "cl /help" in MSVC scripts for platform detection. "cl /?"
produces a different output if run on a real or a virtual drive (this can be
set with a simple subst command), causing an error in the MSVC scripts if
building on a virtual drive because the platform to use cannot be detected.
"cl /help", on the contrary, produces a consistent output if used on a real or
virtual drive. Changing to "/help" allows the compilation to work with a
virtual drive as long as the top of the code repository is part of the drive,
without impacting the build on real drives. Reported-by: Robert Grange
Author: Juan José Santamaría Flecha Discussion:
[https://postgr.es/m/16825-c4f104bcebc67034@postgresql.org](https://postgr.es/m/16825-c4f104bcebc67034@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/733d670073efd2c3...](https://git.postgresql.org/pg/commitdiff/733d670073efd2c3...)
- Move SSL information callback earlier to capture more information. The
callback for retrieving state change information during connection setup was
only installed when the connection was mostly set up, and thus didn't provide
much information and missed all the details related to the handshake. This
also extends the callback with SSL_state_string_long() to print more
information about the state change within the SSL object handled. While
there, fix some comments which were incorrectly referring to the callback and
its previous location in fe-secure.c. Author: Daniel Gustafsson Discussion:
[https://postgr.es/m/232CF476-94E1-42F1-9408-719E2AEC5491@...](https://postgr.es/m/232CF476-94E1-42F1-9408-719E2AEC5491@...)
[https://git.postgresql.org/pg/commitdiff/af0e79c8f4f4c3c2...](https://git.postgresql.org/pg/commitdiff/af0e79c8f4f4c3c2...)
- Introduce SHA1 implementations in the cryptohash infrastructure. With this
commit, SHA1 goes through the implementation provided by OpenSSL via EVP when
building the backend with it, and uses as fallback implementation KAME which
was located in pgcrypto and already shaped for an integration with a set of
init, update and final routines. Structures and routines have been renamed to
make things consistent with the fallback implementations of MD5 and SHA2.
uuid-ossp has used for ages a shortcut with pgcrypto to fetch a copy of SHA1
if needed. This was built depending on the build options within ./configure,
so this cleans up some code and removes the build dependency between pgcrypto
and uuid-ossp. Note that this will help with the refactoring of HMAC, as
pgcrypto offers the option to use MD5, SHA1 or SHA2, so only the second option
was missing to make that possible. Author: Michael Paquier Reviewed-by:
Heikki Linnakangas Discussion:
[https://postgr.es/m/X9HXKTgrvJvYO7Oh@paquier.xyz](https://postgr.es/m/X9HXKTgrvJvYO7Oh@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/a8ed6bb8f4cf259b...](https://git.postgresql.org/pg/commitdiff/a8ed6bb8f4cf259b...)
Fujii Masao pushed:
- postgres_fdw: Add function to list cached connections to foreign servers. This
commit adds function postgres_fdw_get_connections() to return the foreign
server names of all the open connections that postgres_fdw established from
the local session to the foreign servers. This function also returns whether
each connection is valid or not. This function is useful when checking all
the open foreign server connections. If we found some connection to drop, from
the result of function, probably we can explicitly close them by the function
that upcoming commit will add. This commit bumps the version of postgres_fdw
to 1.1 since it adds new function. Author: Bharath Rupireddy, tweaked by
Fujii Masao Reviewed-by: Zhijie Hou, Alexey Kondratov, Zhihong Yu, Fujii Masao
Discussion:
[https://postgr.es/m/2d5cb0b3-a6e8-9bbb-953f-879f47128faa@...](https://postgr.es/m/2d5cb0b3-a6e8-9bbb-953f-879f47128faa@...)
[https://git.postgresql.org/pg/commitdiff/708d165ddb92c540...](https://git.postgresql.org/pg/commitdiff/708d165ddb92c540...)
- doc: Add note about the server name of postgres_fdw_get_connections() returns.
Previously the document didn't mention the case where
postgres_fdw_get_connections() returns NULL in server_name column. Users might
be confused about why NULL was returned. This commit adds the note that, in
postgres_fdw_get_connections(), the server name of an invalid connection will
be NULL if the server is dropped. Suggested-by: Zhijie Hou Author: Bharath
Rupireddy Reviewed-by: Zhijie Hou, Fujii Masao Discussion:
[https://postgr.es/m/e7ddd14e96444fce88e47a709c196537@G08C...](https://postgr.es/m/e7ddd14e96444fce88e47a709c196537@G08C...)
[https://git.postgresql.org/pg/commitdiff/ee79a548e746da9a...](https://git.postgresql.org/pg/commitdiff/ee79a548e746da9a...)
Peter Eisentraut pushed:
- Pause recovery for insufficient parameter settings. When certain parameters
are changed on a physical replication primary, this is communicated to
standbys using the XLOG_PARAMETER_CHANGE WAL record. The standby then checks
whether its own settings are at least as big as the ones on the primary. If
not, the standby shuts down with a fatal error. This patch changes this
behavior for hot standbys to pause recovery at that point instead. That
allows read traffic on the standby to continue while database administrators
figure out next steps. When recovery is unpaused, the server shuts down (as
before). The idea is to fix the parameters while recovery is paused and then
restart when there is a maintenance window. Reviewed-by: Sergei Kornilov
<sk@zsrv.org> Discussion:
[https://www.postgresql.org/message-id/flat/4ad69a4c-cc9b-...](https://www.postgresql.org/message-id/flat/4ad69a4c-cc9b-...)
[https://git.postgresql.org/pg/commitdiff/15251c0a60be76ee...](https://git.postgresql.org/pg/commitdiff/15251c0a60be76ee...)
- pageinspect: Change block number arguments to bigint. Block numbers are 32-bit
unsigned integers. Therefore, the smallest SQL integer type that they can fit
in is bigint. However, in the pageinspect module, most input and output
parameters dealing with block numbers were declared as int. The behavior with
block numbers larger than a signed 32-bit integer was therefore dubious.
Change these arguments to type bigint and add some more explicit error
checking on the block range. (Other contrib modules appear to do this
correctly already.) Since we are changing argument types of existing
functions, in order to not misbehave if the binary is updated before the
extension is updated, we need to create new C symbols for the entry points,
similar to how it's done in other extensions as well. Reported-by: Ashutosh
Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera
<alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion:
[https://www.postgresql.org/message-id/flat/d8f6bdd536df40...](https://www.postgresql.org/message-id/flat/d8f6bdd536df40...)
[https://git.postgresql.org/pg/commitdiff/f18aa1b203930ed2...](https://git.postgresql.org/pg/commitdiff/f18aa1b203930ed2...)
- Remove bogus tracepoint. Calls to LWLockWaitForVar() fired the
TRACE_POSTGRESQL_LWLOCK_ACQUIRE tracepoint, but LWLockWaitForVar() never
actually acquires the LWLock. (Probably a copy/paste bug in 68a2e52bbaf.)
Remove it. Author: Craig Ringer <craig.ringer@enterprisedb.com> Discussion:
[https://www.postgresql.org/message-id/flat/CAGRY4nxJo+-HC...](https://www.postgresql.org/message-id/flat/CAGRY4nxJo+-HC...)
[https://git.postgresql.org/pg/commitdiff/09418bed67a7d5d5...](https://git.postgresql.org/pg/commitdiff/09418bed67a7d5d5...)
Magnus Hagander pushed:
- Bump PGSTAT_FILE_FORMAT_ID. This was missed in 960869da08 Reported-By:
Laurenz Albe Discussion:
[https://postgr.es/m/4f0aacc5fe1b4bfafa32b36ecd97469fae526...](https://postgr.es/m/4f0aacc5fe1b4bfafa32b36ecd97469fae526...)
[https://git.postgresql.org/pg/commitdiff/b2f87b46690ab8cd...](https://git.postgresql.org/pg/commitdiff/b2f87b46690ab8cd...)
- Remove reference to ftp servers from documentation. It's been a long time
since we used ftp, but there was a single reference left in the docs. Author:
Daniel Gustafsson <daniel@yesql.se> Discussion:
[https://postgr.es/m/6880D602-7286-46EC-8A03-14E3248FEC7A@...](https://postgr.es/m/6880D602-7286-46EC-8A03-14E3248FEC7A@...)
[https://git.postgresql.org/pg/commitdiff/0a9ae44288d122c1...](https://git.postgresql.org/pg/commitdiff/0a9ae44288d122c1...)
- Remove make_diff set of tools. These are mostly obsoleted by the switch to
git, and it's easier to remove them than to update the incorrect
documentation. Discussion:
[https://postgr.es/m/CABUevEwmASMn4WRJ6RagBx43sj10ctfMHcMA...](https://postgr.es/m/CABUevEwmASMn4WRJ6RagBx43sj10ctfMHcMA...)
[https://git.postgresql.org/pg/commitdiff/8a337b0ed21c6545...](https://git.postgresql.org/pg/commitdiff/8a337b0ed21c6545...)
Robert Haas pushed:
- Allow for error or refusal while absorbing a ProcSignalBarrier. Previously,
the per-barrier-type functions tasked with absorbing them were expected to
always succeed and never throw an error. However, that's a bit inconvenient.
Further study has revealed that there are realistic cases where it might not
be possible to absorb a ProcSignalBarrier without terminating the transaction,
or even the whole backend. Similarly, for some barrier types, there might be
other reasons where it's not reasonably possible to absorb the barrier at
certain points in the code, so provide a way for a per-barrier-type function
to reject absorbing the barrier. Unfortunately, there's still no committed
code making use of this infrastructure; hopefully, we'll get there. :-( Patch
by me, reviewed by Andres Freund and Amul Sul. Discussion:
[http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3....](http://postgr.es/m/20200908182005.xya7wetdh3pndzim@alap3....)
Discussion:
[http://postgr.es/m/CA+Tgmob56Pk1-5aTJdVPCWFHon7me4M96ENpG...](http://postgr.es/m/CA+Tgmob56Pk1-5aTJdVPCWFHon7me4M96ENpG...)
[https://git.postgresql.org/pg/commitdiff/a3ed4d1efe9f3a17...](https://git.postgresql.org/pg/commitdiff/a3ed4d1efe9f3a17...)
Tom Lane pushed:
- Add bytea equivalents of ltrim() and rtrim(). We had bytea btrim() already,
but for some reason not the other two. Joel Jacobson Discussion:
[https://postgr.es/m/d10cd5cd-a901-42f1-b832-763ac6f7ff3a@...](https://postgr.es/m/d10cd5cd-a901-42f1-b832-763ac6f7ff3a@...)
[https://git.postgresql.org/pg/commitdiff/a6cf3df4ebdcbc78...](https://git.postgresql.org/pg/commitdiff/a6cf3df4ebdcbc78...)
- Narrow the scope of a local variable. This is better style and more
symmetrical with the other if-branch. This likely should have been included in
9de77b545 (which created the opportunity), but it was overlooked. Japin Li
Discussion:
[https://postgr.es/m/MEYP282MB16699FA4A7CD57EB250E871FB6A4...](https://postgr.es/m/MEYP282MB16699FA4A7CD57EB250E871FB6A4...)
[https://git.postgresql.org/pg/commitdiff/3fd80c728dc36fbd...](https://git.postgresql.org/pg/commitdiff/3fd80c728dc36fbd...)
- Avoid crash with WHERE CURRENT OF and a custom scan plan. execCurrent.c's
search_plan_tree() assumed that ForeignScanStates and CustomScanStates
necessarily have a valid ss_currentRelation. This is demonstrably untrue for
postgres_fdw's remote join and remote aggregation plans, and non-leaf custom
scans might not have an identifiable scan relation either. Avoid crashing by
ignoring such nodes when the field is null. This solution will lead to errors
like 'cursor "foo" is not a simply updatable scan of table "bar"' in cases
where maybe we could have allowed WHERE CURRENT OF to work. That's not an
issue for postgres_fdw's usages, since joins or aggregations would render
WHERE CURRENT OF invalid anyway. But an otherwise-transparent upper level
custom scan node might find this annoying. When and if someone cares to
expend work on such a scenario, we could invent a custom-scan-provider
callback to determine what's safe. Report and patch by David Geier,
commentary by me. It's been like this for awhile, so back-patch to all
supported branches. Discussion:
[https://postgr.es/m/0253344d-9bdd-11c4-7f0d-d88c02cd7991@...](https://postgr.es/m/0253344d-9bdd-11c4-7f0d-d88c02cd7991@...)
[https://git.postgresql.org/pg/commitdiff/60661bbf2dca0c1e...](https://git.postgresql.org/pg/commitdiff/60661bbf2dca0c1e...)
- Remove faulty support for MergeAppend plan with WHERE CURRENT OF. Somebody
extended search_plan_tree() to treat MergeAppend exactly like Append, which is
100% wrong, because unlike Append we can't assume that only one input node is
actively returning tuples. Hence a cursor using a MergeAppend across a UNION
ALL or inheritance tree could falsely match a WHERE CURRENT OF query at a row
that isn't actually the cursor's current output row, but coincidentally has
the same TID (in a different table) as the current output row. Delete the
faulty code; this means that such a case will now return an error like 'cursor
"foo" is not a simply updatable scan of table "bar"', instead of silently
misbehaving. Users should not find that surprising though, as the same cursor
query could have failed that way already depending on the chosen plan. (It
would fail like that if the sort were done with an explicit Sort node instead
of MergeAppend.) Expand the clearly-inadequate commentary to be more explicit
about what this code is doing, in hopes of forestalling future mistakes. It's
been like this for awhile, so back-patch to all supported branches.
Discussion:
[https://postgr.es/m/482865.1611075182@sss.pgh.pa.us](https://postgr.es/m/482865.1611075182@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/a0efda88a679edae...](https://git.postgresql.org/pg/commitdiff/a0efda88a679edae...)
- Disable vacuum page skipping in selected test cases. By default VACUUM will
skip pages that it can't immediately get exclusive access to, which means that
even activities as harmless and unpredictable as checkpoint buffer writes
might prevent a page from being processed. Ordinarily this is no big deal,
but we have a small number of test cases that examine the results of VACUUM's
processing and therefore will fail if the page of interest is skipped. This
seems to be the explanation for some rare buildfarm failures. To fix, add the
DISABLE_PAGE_SKIPPING option to the VACUUM commands in tests where this could
be an issue. In passing, remove a duplicated query in
pageinspect/sql/page.sql. Back-patch as necessary (some of these cases are as
old as v10). Discussion:
[https://postgr.es/m/413923.1611006484@sss.pgh.pa.us](https://postgr.es/m/413923.1611006484@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/c2dc1a79767a0f94...](https://git.postgresql.org/pg/commitdiff/c2dc1a79767a0f94...)
- Further tweaking of PG_SYSROOT heuristics for macOS. It emerges that in some
phases of the moon (perhaps to do with directory entry order?), xcrun will
report that the SDK path is
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk which is normally a
symlink to a version-numbered sibling directory. Our heuristic to skip
non-version-numbered pathnames was rejecting that, which is the wrong thing to
do. We'd still like to end up with a version-numbered PG_SYSROOT value, but
we can have that by dereferencing the symlink. Like the previous fix,
back-patch to all supported versions. Discussion:
[https://postgr.es/m/522433.1611089678@sss.pgh.pa.us](https://postgr.es/m/522433.1611089678@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/9d23c15a034ba163...](https://git.postgresql.org/pg/commitdiff/9d23c15a034ba163...)
- Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being equal
to the relids in its contents, but that fails to account for the possibility
that we have to postpone evaluation of the PHV due to outer joins. This could
result in a malformed plan. The known cases end up triggering the "failed to
assign all NestLoopParams to plan nodes" sanity check in createplan.c, but
other symptoms may be possible. The right value to use is the join level we
actually intend to evaluate the PHV at. We can get that from the ph_eval_at
field of the associated PlaceHolderInfo. However, there are some places that
call pull_varnos() before the PlaceHolderInfos have been created; in that
case, fall back to the conservative assumption that the PHV will be evaluated
at its syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in practice.)
Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have reached
their final values by the time we need them. The main problem in making this
work is that pull_varnos() has no way to get at the PlaceHolderInfos. We can
fix that easily, if a bit tediously, in HEAD by passing it the planner "root"
pointer. In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones with the
additional parameter. (If an old entry point is called and encounters a PHV,
it'll fall back to using the syntactic level, again possibly missing some
valid optimization.) Back-patch to v12. The computation is surely also wrong
before that, but it appears that we cannot reach a bad plan thanks to join
order restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions. Per report from Stephan Springl. Discussion:
[https://postgr.es/m/171041.1610849523@sss.pgh.pa.us](https://postgr.es/m/171041.1610849523@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/55dc86eca70b1dc1...](https://git.postgresql.org/pg/commitdiff/55dc86eca70b1dc1...)
- Improve new wording of libpq's connection failure messages. "connection to
server so-and-so failed:" seems clearer than the previous wording "could not
connect to so-and-so:" (introduced by 52a10224e), because the latter suggests
a network-level connection failure. We're now prefixing this string to all
types of connection failures, for instance authentication failures; so we need
wording that doesn't imply a low-level error. Per discussion with Robert
Haas. Discussion:
[https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8Vc...](https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8Vc...)
[https://git.postgresql.org/pg/commitdiff/27a48e5a16ff2227...](https://git.postgresql.org/pg/commitdiff/27a48e5a16ff2227...)
- Doc: remove misleading claim in documentation of PQreset(). This text claimed
that the reconnection would occur "to the same server", but there is no such
guarantee in the code, nor would insisting on that be an improvement.
Back-patch to v10 where multi-host connection strings were added. Discussion:
[https://postgr.es/m/1095901.1611268376@sss.pgh.pa.us](https://postgr.es/m/1095901.1611268376@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/ab66645628d38a70...](https://git.postgresql.org/pg/commitdiff/ab66645628d38a70...)
- Re-allow DISTINCT in pl/pgsql expressions. I'd omitted this from the grammar
in commit c9d529848, figuring that it wasn't worth supporting. However we
already have one complaint, so it seems that judgment was wrong. It doesn't
require a huge amount of code, so add it back. (I'm still drawing the line at
UNION/INTERSECT/EXCEPT though: those'd require an unreasonable amount of
grammar refactoring, and the single-result-row restriction makes them near
useless anyway.) Also rethink the documentation: this behavior is a property
of all pl/pgsql expressions, not just assignments. Discussion:
[https://postgr.es/m/20210122134106.e94c5cd7@mail.verfriem...](https://postgr.es/m/20210122134106.e94c5cd7@mail.verfriem...)
[https://git.postgresql.org/pg/commitdiff/7cd9765f9bd3397b...](https://git.postgresql.org/pg/commitdiff/7cd9765f9bd3397b...)
- Avoid redundantly prefixing PQerrorMessage for a connection failure. libpq's
error messages for connection failures pretty well stand on their own,
especially since commits 52a10224e/27a48e5a1. Prefixing them with 'could not
connect to database "foo"' or the like is just redundant, and perhaps even
misleading if the specific database name isn't relevant to the failure. (When
it is, we trust that the backend's error message will include the DB name.)
Indeed, psql hasn't used any such prefix in a long time. So, make all our
other programs and documentation examples agree with psql's practice.
Discussion:
[https://postgr.es/m/1094524.1611266589@sss.pgh.pa.us](https://postgr.es/m/1094524.1611266589@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/58cd8dca3de0b3c7...](https://git.postgresql.org/pg/commitdiff/58cd8dca3de0b3c7...)
- Doc: improve directions for building on macOS. In light of recent discussions,
we should instruct people to install Apple's command line tools; installing
Xcode is secondary. Also, fix sample command for finding out the default
sysroot, as we now know that the command originally recommended can give a
result that doesn't match your OS version. Also document the workaround to
use if you really don't want configure to select a sysroot at all.
Discussion:
[https://postgr.es/m/20210119111625.20435-1-james.hilliard...](https://postgr.es/m/20210119111625.20435-1-james.hilliard...)
[https://git.postgresql.org/pg/commitdiff/50bebc1ae1804e0d...](https://git.postgresql.org/pg/commitdiff/50bebc1ae1804e0d...)
- Suppress bison warning in ecpg grammar. opt_distinct_clause is only used in
PLpgSQL_Expr, which ecpg ignores, so it needs to ignore opt_distinct_clause
too. My oversight in 7cd9765f9; reported by Bruce Momjian. Discussion:
[https://postgr.es/m/E1l33wr-0005sJ-9n@gemulon.postgresql.org](https://postgr.es/m/E1l33wr-0005sJ-9n@gemulon.postgresql.org)
[https://git.postgresql.org/pg/commitdiff/3fc81ce459e1696f...](https://git.postgresql.org/pg/commitdiff/3fc81ce459e1696f...)
- Update ecpg's connect-test1 for connection-failure message changes. I should
have updated this in commits 52a10224e and follow-ons, but I missed it because
it's not run by default, and none of the buildfarm runs it either. Maybe we
should try to improve that situation. Discussion:
[https://postgr.es/m/CAH2-Wz=j9SRW=s5BV4-3k+=tr4N3A03in+gT...](https://postgr.es/m/CAH2-Wz=j9SRW=s5BV4-3k+=tr4N3A03in+gT...)
[https://git.postgresql.org/pg/commitdiff/68d1c339417ea2e8...](https://git.postgresql.org/pg/commitdiff/68d1c339417ea2e8...)
- Doc: update example connection-failure messages in the documentation. Now that
the dust has more or less settled on 52a10224e and follow-ons, make sure the
examples in the documentation are up-to-date.
[https://git.postgresql.org/pg/commitdiff/183bbd1b6d4376f1...](https://git.postgresql.org/pg/commitdiff/183bbd1b6d4376f1...)
- Doc: clean up contrib/pageinspect's GIST function documentation. I came to fix
the overwidth-PDF-page warnings seen in the buildfarm, but stayed long enough
to copy-edit some nearby text.
[https://git.postgresql.org/pg/commitdiff/7e57255f6189380d...](https://git.postgresql.org/pg/commitdiff/7e57255f6189380d...)
- Add a simple test for contrib/auto_explain. This module formerly had zero test
coverage. Discussion:
[https://postgr.es/m/1445881.1611441692@sss.pgh.pa.us](https://postgr.es/m/1445881.1611441692@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/0c1e8845f28bd07a...](https://git.postgresql.org/pg/commitdiff/0c1e8845f28bd07a...)
Bruce Momjian pushed:
- doc: adjust alignment of doc file list for "pg_waldump.sgml".
Backpatch-through: 10
[https://git.postgresql.org/pg/commitdiff/9e7dbe3369cd8f5b...](https://git.postgresql.org/pg/commitdiff/9e7dbe3369cd8f5b...)
Amit Kapila pushed:
- pgindent worker.c. This is a leftover from commit 0926e96c49. Changing this
separately because this file is being modified for upcoming patch logical
replication of 2PC. Author: Peter Smith Discussion:
[https://postgr.es/m/CAHut+Ps+EgG8KzcmAyAgBUi_vuTps6o9ZA8D...](https://postgr.es/m/CAHut+Ps+EgG8KzcmAyAgBUi_vuTps6o9ZA8D...)
[https://git.postgresql.org/pg/commitdiff/ed43677e20369040...](https://git.postgresql.org/pg/commitdiff/ed43677e20369040...)
Thomas Munro pushed:
- Fix sample output of EXPLAIN ANALYZE. Since commit
f0f13a3a08b2757997410f3a1c38bdc22973c525, we estimate ModifyTable paths
without a RETURNING clause differently. Update an example from the manual
that showed the old behavior. Author: Takayuki Tsunakawa
<tsunakawa.takay@fujitsu.com> Reviewed-by: Laurenz Albe
<laurenz.albe@cybertec.at> Discussion:
[https://postgr.es/m/TYAPR01MB29905674F41693BBA9DA28CAFEA2...](https://postgr.es/m/TYAPR01MB29905674F41693BBA9DA28CAFEA2...)
[https://git.postgresql.org/pg/commitdiff/679744cf1b0d0569...](https://git.postgresql.org/pg/commitdiff/679744cf1b0d0569...)
# Pending Patches
Craig Ringer sent in a patch to cross-reference comments on signal handling
logic.
Joel Jacobson sent in a patch to implement catalog_oidjoins.pl, which parses
catalog references out of catalogs.sgml.
Bertrand Drouvot sent in another revision of a patch to enable minimal logical
decoding on standbys.
Justin Pryzby sent in another revision of a patch to g_dump: make CLUSTER ON a
separate dump object since it needs to be restored after any child indexes are
restored and attached.
John Naylor sent in another revision of a patch to make it possible to truncate
timestamps on a wider selection of intervals.
Robert Haas sent in a patch to remove CheckpointLock.
Craig Ringer sent in a patch to add ProcessInterrupts_hook.
Atsushi Torikoshi sent in a patch to fix a corner case where the TOAST condition
was wrong for the column size.
Kyotaro HORIGUCHI and Noah Misch traded patches to test for snapshot too old and
wal_level=minimal, avoid using RelationNeedsWAL to identify relation
persistence, and keep page-LSN updated while WAL-skipping.
Heikki Linnakangas sent in two more revisions of a patch to move a few
ResourceOwnerEnlarge() calls for safety and clarity, make resowners more easily
extensible, and make hash_resource_elem work better for the case where there's
a 64-bit input.
Michail Nikolaev sent in a patch to add full support for indexing LP_DEAD hint
bits on standbys.
Vigneshwaran C sent in another revision of a patch to add schema-level support
for PUBLICATIONs.
Masahiko Sawada sent in two more revisions of a patch to introduce an IndexAM
API for choosing the index vacuum strategy, choose index vacuum strategy based
on same, and skip B-tree bulkdelete if the index doesn't grow.
Craig Ringer sent in a patch to add more docs on what (not) to do in extension
code.
Atsushi Torikoshi sent in two more revisions of a patch to add a waitstart field
to the pg_lock_status view and supporting function.
Heikki Linnakangas and Jürgen Purtz traded patches to add a chapter on
architecture to the tutorial.
Justin Pryzby and Aleksey Kondratov traded patches to make it possible for
REINDEX, CLUSTER and VACUUM FULL to change tablespaces on the fly.
Pavel Stěhule sent in three more revisions of a patch to implement schema
variables.
Peter Smith sent in three more revisions of a patch to make it possible to use
multiple background workers for tablesync.
Dilip Kumar sent in three more revisions of a patch to make
pg_is_wal_replay_paused return the status of recovery pause.
Amit Langote sent in five revisions of a patch to make get_partition_for_tuple()
usable in broader contexts, and use same to avoid using the SPI interface for
some referential integrity checks.
Takamichi Osumi sent in another revision of a patch to make stronger safeguards
for archive recovery.
Bruce Momjian sent in another revision of a patch to add key management.
Buzhen (步真) sent in a patch to change from a FIFO strategy to LRU to sweep a
valid cache.
Kyotaro HORIGUCHI sent in another revision of a patch to make it possible to
specify a CRL (certificate revocation list) directory, bringing it into line
with the implementation of X509_STORE_load_locations.
Mark G sent in a patch to make the gaps array static in heapam.c.
Jie Zhang sent in a patch to make the output of `pg_dump -?/--help` more
legible.
Michaël Paquier sent in a patch to paint some PG_USED_FOR_ASSERTS_ONLY in inline
functions of ilist.h and bufpage.h.
David Fetter sent in another revision of a patch to surface popcount
(count_set_bits) to SQL.
Greg Sabino Mullane sent in another revision of a patch to help psql's \df
choose among the possible functions by supplying input argument types.
Justin Pryzby sent in another revision of a patch to implement CREATE TABLE
(LIKE .. INCLUDING ACCESS METHOD).
Li Japin sent in a patch to use an array of booleans rather than a character
pointer to denote NULLness of parameters in SPI.
Julien Rouhaud sent in another revision of a patch to add queryId to the
pg_catalog.pg_stat_activity view.
Stephen Frost sent in two more revisions of a patch to change the default of
checkpoint_completion_target to 0.9.
Dilip Kumar sent in another revision of a patch to implement custom compression
methods for tables.
Stephen Frost and Craig Ringer traded patches to add a docs section for
obsoleted and renamed functions and settings.
Vigneshwaran C sent in another revision of a patch to make it possible to print
backtraces of postgres processes that are part of the instance in which
pg_print_callstack() is called. This capability is guarded by a GUC.
Dmitry Dolgov, Pavel Stěhule, and Dian M Fay traded patches to use the generic
type subscripting infrastructure for JSONB.
Daniel Gustafsson and Jacob Champion traded patches to support NSS as a libpq
TLS backend.
Daniel Gustafsson sent in another revision of a patch to support checksum
enable/disable in a running cluster.
Tom Lane sent in a patch to fix detection of pwritev support for OSX.
Hou Zhijie sent in another revision of a patch to add a nullif case for
eval_const_expressions.
Masahiro Ikeda sent in a patch to refactor the variable names of global
statistics messages to make it clear that they are messages.
Peter Smith sent in another revision of a patch to implement logical decoding of
two-phase transactions.
Justin Pryzby sent in another revision of a patch to make INSERT SELECT use a
BulkInsertState.
Alexander Korotkov sent in another revision of a patch to fix an infelicity
between phrase search and multi-lexeme tokens.
Alexander Korotkov sent in a patch to make everything after the json ```**```
operator use strict mode, as the lax mode produces surprises.
Nikita Glukhov and Erik Rijkers traded patches to implement SQL/JSON functions
per the standard.
Anastasia Lubennikova sent in two more revisions of a patch intended to fix a
bug that manifested as pg_upgrade fails with non-standard ACL by adding some
extra checks to cover that situation.
David Rowley sent in a patch to correctly figure out the pages backwards scan
with heap_setscanlimits().
Nikita Glukhov sent in another revision of a patch to implement JSON_TABLE per
the standard.
Kyotaro HORIGUCHI sent in another revision of a patch to refactor the stats
collector to use shared memory instead of files for temporary storage.
Julien Rouhaud sent in another revision of a patch to add a new COLLATION option
to REINDEX.
David Rowley sent in another revision of a patch to make it possible to scan
ranges of TIDs short of the entire relation, as sequential scan does.
Kyotaro HORIGUCHI sent in another revision of a patch to add a new pg_waitlsn()
function.
Daniel Gustafsson sent in a patch to refactor the library-specific SSL test
setup into a separate module in preparation for supporting TLS implementations
other than OpenSSL.
Masahiko Sawada sent in a patch to add a bound check for encoded itermpointers
before bsearch() in lazy_tid_reaped(), and inlines lazy_tid_reaped().
Pavel Stěhule sent in three revisions of a patch to add a plan cache for CALL.
Amul Sul sent in another revision of a patch to implement ALTER SYSTEM READ
{ONLY,WRITE}.
Matthias van de Meent sent in a patch to Update postgres_fdw to import
partitions when named in IMPORT SCHEMA LIMIT TO.
Vigneshwaran C sent in a patch to identify missing publications from publisher
during CREATE/ALTER SUBSCRIPTION.
Álvaro Herrera sent in another revision of a patch to add tracing capability to
libpq.
Peter Eisentraut and Tom Lane traded patches to add primary keys to the system
catalogs.
Denis Laxalde sent in a patch to disable background workers during servers start
in pg_upgrade.
Greg Nancarrow sent in another revision of a patch to make it possible to
parallelize the execution of INSERT (INTO ... SELECT ...).
Álvaro Herrera sent in another revision of a patch to add batch/pipelining
support for libpq.
Andrey Borodin sent in another revision of a patch to reorganize the pglz
compression code to be more efficient, speeding it up by a factor of about 1.4.
Yugo Nagata sent in another revision of a patch to implement incremental
maintenance of materialized views.
Tomáš Vondra sent in two more revisions of a patch to implement extended
statistics for expressions.
Heikki Linnakangas sent in another revision of a patch to pg_rewind which makes
it fetch small files according to new size.
Tomáš Vondra sent in another revision of a patch to make it possible to use
non-volatile memory, if available, for WAL buffers.
Heikki Linnakangas sent in another revision of a patch to refactor the
LogicalTapeSet/LogicalTape interface to make all the tape functions, like
LogicalTapeRead and LogicalTapeWrite, take a LogicalTape as argument
instead of LogicalTapeSet+tape number, replace the old polyphase merge algorithm
with a simple balanced k-way merge, and fix the sizing of tape read buffers.
Tomáš Vondra sent in another revision of a patch to implement BRIN multi-range
indexes.
Masahiro Ikeda sent in two more revisions of a patch to add columns to the
pg_stat_wal view to track WAL I/O activity, and a new GUC, track_wal_io_timing,
to enable collecting this, as the overhead could be large.
Jim Finnerty sent in another revision of a patch to prepare to implement 64-bit
xids by adding a way to have 64-bit GUCs, using XID_FMT to format xids, and
using ClogPageNumber in place of int for type safety.
Michaël Paquier sent in another revision of a patch to refactor the HMAC
implementations into common/.
Amit Kapila and Bharath Rupireddy traded patches to fix the ALTER
PUBLICATION...DROP TABLE behavior by initializing pubactions when
RelationSyncEntry gets invalidated.
Mark Rofail sent in three more revisions of a patch to make it possible to check
that each element of an array matches a primary key in (in general) a different
table.
Thomas Munro sent in another revision of a patch to Use pg_pwrite() in
pg_test_fsync.
Tomáš Vondra sent in a patch to test TOAST visibility.
Andy Fan sent in a patch to build some implied pruning quals to extend the use
case of planning time partition pruning and init partition pruning.
