You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some extensions generate their versioned SQL (the install and upgrade
scripts) from .sql.in at build time and gitignore the generated .sql (so the
repo tracks only the .sql.in sources). cat_tools does this.
Problem
DATA is a PGXS variable, but pgxntool's base.mk seeds it with:
DATA = $(EXTENSION_VERSION_FILES)$(wildcard sql/*--*--*.sql)
$(wildcard ...) is expanded — and GNU Make caches the sql/ directory listing
— at parse time, which is beforemake all generates the .sql. On a
clean tree (fresh clone / after make clean) the generated upgrade scripts don't
exist yet, so the wildcard returns nothing for them, they never enter DATA, and make install silently omits them. Because DATA is recursive, one might expect
the recipe-time re-expansion to pick them up once all has generated them, but
Make's cached directory listing means the wildcard stays empty for the run.
Concrete impact (cat_tools)
A clean make install omits the generated upgrade scripts, so a user who
installed an older version cannot ALTER EXTENSION ... UPDATE — the upgrade
path files simply aren't installed.
On a branch that also has an upgrade build test (test/build/upgrade.sql doing CREATE EXTENSION VERSION '<old>' + ALTER EXTENSION UPDATE), a clean make test fails with ERROR: no update path from "X" to "Y".
DATA itself is PGXS, so there's a limit to what pgxntool can change — but the
choice to populate it via a parse-time $(wildcard) over generated output is
pgxntool's, so this seems like the right place to raise it. The underlying issue
is really "pgxntool + generating .sql", not a bug in any one line.
An idea (leaving the design to you)
It would be nice if pgxntool offered first-class support for generated SQL — for
example, a convention where an extension drops generated .sql into a .generated/ directory that pgxntool knows to build first and then install, so DATA never depends on parse-time globbing of not-yet-generated files. Not
prescribing a solution; just flagging the problem and this direction.
Companion to #42 (test/install ALTER-UPDATE pattern), which came out of the same
cat_tools work.
DATA's parse-time$(wildcard sql/*--*--*.sql)misses generated (gitignored) scriptsContext
Some extensions generate their versioned SQL (the install and upgrade
scripts) from
.sql.inat build time and gitignore the generated.sql(so therepo tracks only the
.sql.insources). cat_tools does this.Problem
DATAis a PGXS variable, but pgxntool'sbase.mkseeds it with:$(wildcard ...)is expanded — and GNU Make caches thesql/directory listing— at parse time, which is before
make allgenerates the.sql. On aclean tree (fresh clone / after
make clean) the generated upgrade scripts don'texist yet, so the wildcard returns nothing for them, they never enter
DATA, andmake installsilently omits them. BecauseDATAis recursive, one might expectthe recipe-time re-expansion to pick them up once
allhas generated them, butMake's cached directory listing means the wildcard stays empty for the run.
Concrete impact (cat_tools)
make installomits the generated upgrade scripts, so a user whoinstalled an older version cannot
ALTER EXTENSION ... UPDATE— the upgradepath files simply aren't installed.
test/build/upgrade.sqldoingCREATE EXTENSION VERSION '<old>'+ALTER EXTENSION UPDATE), a cleanmake testfails withERROR: no update path from "X" to "Y".scripts in
DATAexplicitly from their.sql.insource names (which areparse-stable) — cat_tools PR Clarify debug() level scheme: use ranges, not multiples of 10 #32.
Scope note
DATAitself is PGXS, so there's a limit to what pgxntool can change — but thechoice to populate it via a parse-time
$(wildcard)over generated output ispgxntool's, so this seems like the right place to raise it. The underlying issue
is really "pgxntool + generating
.sql", not a bug in any one line.An idea (leaving the design to you)
It would be nice if pgxntool offered first-class support for generated SQL — for
example, a convention where an extension drops generated
.sqlinto a.generated/directory that pgxntool knows to build first and then install, soDATAnever depends on parse-time globbing of not-yet-generated files. Notprescribing a solution; just flagging the problem and this direction.
Companion to #42 (test/install ALTER-UPDATE pattern), which came out of the same
cat_tools work.