Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ results/
regression.diffs
regression.out

# Generated sql/ directory for test/build
# Created by make test-build. See README.asc for details.
test/build/sql/

# Auto-generated schedule file for test/install
# Created by make when test/install/*.sql files exist.
test/install/schedule

# Misc
tmp/
.DS_Store
Expand Down
19 changes: 0 additions & 19 deletions pgxntool/.claude/settings.json

This file was deleted.

1 change: 1 addition & 0 deletions pgxntool/.gitattributes
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
.gitattributes export-ignore
.claude/ export-ignore
.github/ export-ignore
*.md export-ignore
.DS_Store export-ignore
*.asc export-ignore
Expand Down
35 changes: 32 additions & 3 deletions pgxntool/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,34 @@

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## CI Monitoring After Every Push

**REQUIRED**: After every `git push`, immediately start a background task to
monitor the CI run for that push. If you pushed to both pgxntool and
pgxntool-test, start a background task for each repo — do not monitor them
sequentially.

The CI monitor lives in the pgxntool-test checkout: run
`bash ../pgxntool-test/.claude/skills/ci/scripts/monitor-ci.sh` (the `/ci`
skill). It monitors both repos and derives the owner from the current repo.
Pass the exact push SHA when available — `gh run list --branch` has a race
condition: if two pushes land close together on the same branch, `--branch`
may pick up the wrong run. `--commit SHA` targets the exact push and avoids it.

## Scope of This File

**CLAUDE.md is for people USING pgxntool** — extension developers who have embedded
pgxntool into their project via `git subtree`. It documents the build system, available
commands, and how pgxntool works.

**If you are making changes to pgxntool itself**, stop — you are in the wrong place.
See `.claude/` in this directory for developer guidelines. More importantly, pgxntool
development must be done from the **pgxntool-test** repository, not from here. See the
`Development Workflow` section below.

Any agent working in an extension project should always defer to that project's own
CLAUDE.md and instructions over anything stated here.

## Git Commit Guidelines

**IMPORTANT**: When creating commit messages, do not attribute commits to yourself (Claude). Commit messages should reflect the work being done without AI attribution in the message body. The standard Co-Authored-By trailer is acceptable.
Expand Down Expand Up @@ -181,9 +209,10 @@ When tests fail, examine the diff output carefully. The actual test output in `t
- Validates repo is clean before tagging

### Subtree Sync Support
- `make pgxntool-sync` pulls latest release
- Multiple sync targets: release, stable, local variants
- Uses `git subtree pull --squash`
- `make pgxntool-sync` pulls the latest release (the `release` tag) from the canonical repo
- `pgxntool/pgxntool-sync.sh [<repo> [<ref>]]` does the work and can be run without make
- `make pgxntool-sync-<name>` pulls from the `pgxntool-sync-<name>` variable (`<repo> <ref>`)
- Uses `git subtree pull --squash`, then `update-setup-files.sh` for a 3-way merge of copied files
- Requires clean repo (no uncommitted changes)

### pg_tle Support
Expand Down
74 changes: 73 additions & 1 deletion pgxntool/HISTORY.asc
Original file line number Diff line number Diff line change
@@ -1,3 +1,75 @@
2.1.0
-----
== Fix setup.sh / pgxntool-sync.sh / update-setup-files.sh inside a git worktree
These scripts guarded the project root with `[ -d .git ]`. In a linked worktree
`.git` is a file, not a directory, so the check failed: setup.sh would wrongly
re-run `git init`, and the sync scripts aborted with "Not in a git repository."
They now detect the repo with `git rev-parse --git-dir`, which works in both a
normal clone and a worktree.

== Fix `verify-results` checking stale results in `make results`
`make results` ran `verify-results` before `make test`, so it checked stale
`regression.diffs` from a prior run. Reordered so `verify-results` always
checks the fresh results.

== Fix `PGXNTOOL_ENABLE_TEST_BUILD`/`PGXNTOOL_ENABLE_TEST_INSTALL` ignoring command-line values
Without `override`, the `pgxntool_validate_yesno` normalization was silently
skipped when these variables were set on the command line.

2.0.3
-----
== Fix pgxntool-sync remote, and make it runnable without make
`make pgxntool-sync` pointed at the old SSH URL, which fails without
GitHub SSH keys; it now uses `https://github.com/Postgres-Extensions/pgxntool.git`
over HTTPS (still the `release` tag). The subtree pull + `update-setup-files.sh`
logic moved into `pgxntool/pgxntool-sync.sh`, which the make targets now wrap, so
you can also sync without `make` by running it directly.

2.0.2
-----
== Fix parse_control_file corrupting values with trailing comments
Control file values like `default_version = '1.0.0' # comment` were parsed
incorrectly — the trailing quote was left in the value due to comment removal
happening after quote stripping. Fixed by removing comments first.

2.0.1
-----
== Improve bash compatibility (specifically for Mac OS)
Mac OS uses the (very old) bash 3.2 as the default shell; fix a few compatibility bugs.

2.0.0
-----
== Remove .source file support
PostgreSQL removed `.source` file processing from `pg_regress` in PG15.
The `input/*.source` → `sql/*.sql` and `output/*.source` →
`expected/*.out` conversion mechanism no longer functions. All related
variables (`TEST__SOURCE__*`), the `make_results.sh` helper script, and
the special-case logic in `make results` have been removed. Extensions
that used `.source` files should convert them to regular `test/sql/*.sql`
and `test/expected/*.out` files using relative paths.

== Add test-build for pre-test validation
When CREATE EXTENSION fails due to a SQL syntax error, PostgreSQL reports only a cryptic error with limited context. test-build runs your extension SQL directly through pg_regress first, so syntax errors show the exact file, line, and position — cutting debugging time significantly. Place SQL files in `test/build/` to enable; auto-detects based on file presence.

== Add test/install for one-time test setup
Extensions that install dependencies or run expensive setup in every test file pay that cost once per test. test/install runs setup SQL once before the entire test suite, and all regular tests share the resulting database state. This can dramatically speed up test suites that install extensions or load fixtures. Place SQL files in `test/install/` to enable; auto-detects based on file presence.

== Add verify-results safeguard for make results
`make results` now refuses to run when tests are failing (detected via `regression.diffs`). Prevents accidentally blessing incorrect output as the new expected results. Enabled by default; disable with `PGXNTOOL_ENABLE_VERIFY_RESULTS=no`.

== Fix bash 3.2 compatibility and shebang portability
`${#ARRAY[@]:-0}` is a syntax error in bash 3.2; replaced with `${#ARRAY[@]}`.
Shell scripts now use `#!/usr/bin/env bash`.

1.1.2
-----
== Fix double --dbname bug that defeated unique test database names
The unique database naming introduced in 1.1.0 was ineffective because
base.mk added --dbname=$(REGRESS_DBNAME) to REGRESS_OPTS while PGXS
also appends --dbname=$(CONTRIB_TESTDB). The second --dbname caused
pg_regress to create a contrib_regression database that collided across
projects. Fixed by overriding CONTRIB_TESTDB after include $(PGXS) instead.

1.1.1
-----
== Fix pg_tle exception handler and empty upgrade files
Expand Down Expand Up @@ -72,7 +144,7 @@ VERSION is defined by PGXS itself, so trying to use it causes problems.
Old code didn't deal with the lack of a . that can appear in a 10+ version.

0.1.10
------
-----
### Remove invalid `git subtree pull` options

0.1.9
Expand Down
154 changes: 150 additions & 4 deletions pgxntool/README.asc
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@ TODO: Create a nice script that will init a new project for you.

== Development

If you want to contribute to pgxntool development, work from the https://github.com/decibel/pgxntool-test[pgxntool-test] repository, not from this repository. That repository contains the test infrastructure and development tools needed to validate changes to pgxntool. This repository contains only the framework files that get embedded into extension projects via `git subtree`.
If you want to contribute to pgxntool development, work from the https://github.com/Postgres-Extensions/pgxntool-test[pgxntool-test] repository, not from this repository. That repository contains the test infrastructure and development tools needed to validate changes to pgxntool. This repository contains only the framework files that get embedded into extension projects via `git subtree`.

Changes are normally paired across both repos: a `pgxntool` change should come with a matching branch (same name, on the same account) and PR in `pgxntool-test`, and CI enforces this pairing. See the https://github.com/Postgres-Extensions/pgxntool-test#ci-and-contributing[CI and Contributing] section in pgxntool-test for the full workflow.

== Usage
Typically, you can just create a simple Makefile that does nothing but include base.mk:
Expand All @@ -47,6 +49,120 @@ Runs unit tests via the PGXS `installcheck` target. Unlike a simple `make instal

NOTE: While you can still run `make installcheck` or any other valid PGXS make target directly, it's recommended to use `make test` when using pgxntool. The `test` target ensures clean builds, proper test isolation, and correct dependency installation.

=== test-build
Validates that extension SQL files are syntactically correct before running the full test suite. This feature runs SQL files from `test/build/` through `pg_regress`, providing better error messages than `CREATE EXTENSION` failures when there are syntax errors in your extension code.

**How it works:**

1. Place SQL files in `test/build/*.sql`
2. Place expected output in `test/build/expected/*.out`
3. These files run through `pg_regress` before `make test` runs the main test suite
4. If any build test fails, the test run stops immediately with clear error messages

**Directory structure:**

----
test/build/
├── *.sql # SQL test files (checked in)
├── expected/ # Expected output files (checked in)
│ └── *.out
└── sql/ # GENERATED - do not edit or check in
└── *.sql # Synced from *.sql above
----

The `sql/` subdirectory is generated automatically by `make test-build`. It is listed in `.gitignore` and removed by `make clean`. Do not place files directly in `test/build/sql/`.

**Configuration:**

The feature auto-detects based on whether `test/build/*.sql` files exist:

- Files present → feature enabled automatically
- No files → feature disabled (no impact on existing projects)

You can override auto-detection by setting `PGXNTOOL_ENABLE_TEST_BUILD`:
----
# In your Makefile
PGXNTOOL_ENABLE_TEST_BUILD = yes # or no
----

**Example: Validate extension SQL compiles**

Create `test/build/build.sql` to run your extension's SQL directly:

----
\set ECHO none
-- Sets ON_ERROR_STOP, VERBOSITY verbose, and ON_ERROR_ROLLBACK
\i test/pgxntool/psql.sql
-- Suppress column headers and row counts for cleaner expected output
\t

BEGIN;
SET client_min_messages = WARNING;

-- Install dependencies your extension requires
CREATE EXTENSION IF NOT EXISTS pgtap CASCADE;

-- Clean slate
DROP EXTENSION IF EXISTS myext;
DROP SCHEMA IF EXISTS myext;
CREATE SCHEMA myext;

-- Run the actual extension SQL (not CREATE EXTENSION)
-- psql.sql above ensures errors abort immediately with clear messages
\i sql/myext.sql

-- If we get here, the build succeeded; ON_ERROR_STOP would have aborted on any error above
\echo # BUILD TEST SUCCEEDED
ROLLBACK;
----

This approach catches SQL syntax errors *before* running `CREATE EXTENSION`, giving clearer error messages with line numbers. The `ROLLBACK` ensures nothing persists—this is purely validation.

**Why use `\i` instead of `CREATE EXTENSION`?**

When `CREATE EXTENSION` fails, PostgreSQL shows only "syntax error" with limited context. Running the SQL directly via `\i` shows the exact line and position of errors, making debugging much faster.

=== test/install
Runs setup files before the main test suite within the same `pg_regress` invocation. This allows expensive one-time operations (like extension installation) to set up state that persists into the regular test files.

**How it works:**

1. Place SQL files in `test/install/*.sql`
2. Place expected output alongside as `test/install/*.out`
3. A schedule file is auto-generated that lists install files with `../install/` relative paths
4. `pg_regress` processes the install schedule first, then runs regular test files — all in one invocation, so database state persists

**Directory structure:**

----
test/install/
├── *.sql # SQL setup files (checked in)
├── *.out # Expected output (checked in, alongside .sql)
├── .gitignore # Ignores pg_regress artifacts (*.out.diff)
└── schedule # GENERATED - auto-created by make
----

The `schedule` file is generated automatically and listed in `.gitignore`. Do not edit it.

**Configuration:**

The feature auto-detects based on whether `test/install/*.sql` files exist:

- Files present → feature enabled automatically
- No files → feature disabled (no impact on existing projects)

You can override auto-detection by setting `PGXNTOOL_ENABLE_TEST_INSTALL`:
----
# In your Makefile
PGXNTOOL_ENABLE_TEST_INSTALL = yes # or no
----

**Why this is useful:**

Without `test/install`, each test file typically needs to run `CREATE EXTENSION` in its setup, which adds overhead and doesn't allow validating the installation step separately. With `test/install`, setup runs once before all tests, and any state it creates (tables, extensions, etc.) is available to every subsequent test file.

**Key detail:** Install files and regular tests run in a single `pg_regress` invocation. This means the database is NOT dropped between install and test phases — state created by install files persists into the main test suite.

=== testdeps
This rule allows you to ensure certain actions have taken place before running tests. By default it has a single prerequisite, `pgtap`, which will attempt to install http://pgtap.org[pgtap] from PGXN. This depneds on having the pgxn client installed.

Expand Down Expand Up @@ -78,6 +194,24 @@ IMPORTANT: *`make results` requires manual verification first*. The correct work

Never run `make results` without first verifying the test changes are correct. The `results` target copies files from `test/results/` to `test/expected/`, so running it blindly will make incorrect output become the new expected behavior.

==== verify-results safeguard
By default, `make results` will refuse to run if `test/results/regression.diffs` exists (indicating failing tests). This prevents accidentally updating expected files with incorrect output.

If tests are failing, you'll see:
----
ERROR: Tests are failing. Cannot run 'make results'.
Fix test failures first, then run 'make results'.
----

To disable this safeguard (not recommended):
----
# In your Makefile
PGXNTOOL_ENABLE_VERIFY_RESULTS = no

# Or on the command line
make PGXNTOOL_ENABLE_VERIFY_RESULTS=no results
----

=== tag
`make tag` will create a git branch for the current version of your extension, as determined by the META.json file. The reason to do this is so you can always refer to the exact code that went into a released version.

Expand All @@ -91,11 +225,23 @@ WARNING: You will be very unhappy if you forget to update the .control file for
NOTE: Part of the `clean` recipe is cleaning up these .zip files. If you accidentally clean before uploading, just run `make dist-only`.

=== pgxntool-sync
This rule will pull down the latest released version of PGXNtool via `git subtree pull`.
This rule will pull down the latest released version of PGXNtool via `git subtree pull` and then reconcile the files `setup.sh` copied into your project (`.gitignore`, `test/deps.sql`) with a 3-way merge.

NOTE: Your repository must be clean (no modified files) in order to run this. Running this command will produce a git commit of the merge.

TIP: There is also a `pgxntool-sync-%` rule if you need to do more advanced things.
TIP: The actual work is done by `pgxntool/pgxntool-sync.sh`, so you can run it directly (`pgxntool/pgxntool-sync.sh`) if you'd rather not go through `make`. It optionally takes `<repo>` and `<ref>` arguments to pull from somewhere other than the default.

TIP: There is also a `pgxntool-sync-%` rule if you need to do more advanced things. `make pgxntool-sync-<name>` pulls from the `<repo> <ref>` defined by the `pgxntool-sync-<name>` make variable.

=== distclean
`make distclean` removes generated configuration files (`META.json`, `meta.mk`, `control.mk`) that survive a normal `make clean`.

NOTE: PGXS doesn't provide any special support for `distclean` — its built-in `distclean` target simply depends on `clean`. PGXNtool uses its own `PGXNTOOL_distclean` variable to track files that should only be removed by `distclean`, not `clean`.

If your extension generates additional files that should be removed by `distclean` but not `clean`, you can add them:
----
PGXNTOOL_distclean += my_generated_config.mk
----

=== pgtle
Generates pg_tle (Trusted Language Extensions) registration SQL files for deploying extensions in managed environments like AWS RDS/Aurora. See <<_pg_tle_Support>> for complete documentation.
Expand Down Expand Up @@ -223,7 +369,7 @@ Location of `asciidoc` or equivalent executable.
If not set PGXNtool will search for first `asciidoctor`, then `asciidoc`.
ASCIIDOC_EXTS::
File extensions to consider as Asciidoc.
Defined as `+= adoc asciidoc`.
Defined as `+= adoc asciidoc asc`.
ASCIIDOC_FILES::
Asciidoc input files.
PGXNtool searches each `$(DOC_DIRS)` directory, looking for files with any `$(ASCIIDOC_EXTS)` extension.
Expand Down
Loading
Loading