Store keys hash_prefix/schema_prefix are accepted but not honored by writers or GC

[dimitri-yatsenko]

Found during review of #1479 (GC). The per-store config keys hash_prefix, schema_prefix, and filepath_prefix are accepted and validated by settings (Config.get_store_spec → _validate_prefix_separation; also listed in storage_adapter.py COMMON_SPEC_KEYS) — but only <filepath@> consumes them, and only as reserved-namespace guards (builtin_codecs/filepath.py: filepath may not write under hash_prefix/schema_prefix; must stay under filepath_prefix).

Neither the hash writer (hash_registry.build_hash_path — fixed _hash/{schema}/... layout, now via the HASH_STORAGE_PREFIX constant) nor the schema-addressed writer (storage.build_object_path — {schema}/{table}/{pk}/...) nor GC honors them as relocation knobs.

Why this needs a decision

• A user who sets stores.x.hash_prefix = "content" expecting relocation gets a validated, silent no-op — misleading config surface.
• Worse, the trap is loaded: if a future change wires hash_prefix into the writer without teaching GC, hash objects land outside _hash/, get enumerated by the schema-addressed walk, match no schema reference, and are deleted as orphans — data loss. (fix(#1469): codec-driven GC reference discovery + file-level orphan matching #1479 added a comment on HASH_STORAGE_PREFIX flagging exactly this.)

Options

1. Declare the layout fixed (recommended): document hash_prefix/schema_prefix as what they actually are today — reserved-namespace declarations for <filepath@> cohabitation validation, not relocation settings — and rename or clearly re-document them (e.g. in the object-store-configuration spec).
2. Wire them through: honor the prefixes in build_hash_path/build_object_path AND in gc.list_stored_hashes/list_schema_paths in the same change, with migration guidance for existing stores. Larger and of unclear demand.

Either way, the current state — validate-and-ignore — should not survive. Related: the <filepath@>-cohabitation GC hazard noted on #1479 (unattributable files) and the fail-safe planned with #1445/#1478.

[dimitri-yatsenko]

Being resolved within PR #1479 per review (option 1 + GC-side teeth): the keys are re-documented as namespace declarations — not relocation settings — and gc.list_schema_paths now skips the declared filepath_prefix subtree so user-managed files can never be collected (regression-tested, incl. pinning the undeclared-cohabitation hazard). The relocation semantics is explicitly not supported; HASH_STORAGE_PREFIX remains the fixed layout. Will auto-close on merge of #1479.

[dimitri-yatsenko]

Correction to my comment above, from continued review: get_store_spec defaults hash_prefix to _hash and schema_prefix to _schema (settings.py:633-634), so this wasn't pure validate-and-ignore — filepath validation did reserve _hash/ by default. The sharper defects, both now handled on #1479:

1. Overriding hash_prefix un-reserved the real namespace: the key is declarative (writers never consume it), so setting e.g. hash_prefix="content" replaced the default and left the actual _hash/ layout unprotected — precisely when a user misread it as a relocation knob. filepath validation now always reserves the fixed HASH_STORAGE_PREFIX in addition to declared values (commit 6e139a3, with a unit test pinning the override case).
2. schema_prefix's default _schema is vestigial — schema-addressed objects live in root-level {schema}/ directories, so the reservation protects a location nothing writes to. Documented in the settings docstring; whether to retire the key remains open for whoever revisits store-spec surface.

[dimitri-yatsenko]

Final direction, per maintainer review (commit c74a830 on #1479): option 2 — the settings are authoritative. The documentation was the tiebreaker: how-to/configure-storage describes the sections as configured via hash_prefix (default _hash/) and schema_prefix (default _schema/), and migrate-to-v20 gives the path format parameterized as {location}/{hash_prefix}/{schema}/{hash}. The code was out of spec, not the settings.

Wired through:

• put_hash/build_hash_path take hash_prefix from the store spec.
• build_object_path (codec + staged-insert) takes schema_prefix from the spec — new writes land under the documented _schema/ section by default (previously root-level {schema}/…, which was the out-of-spec behavior).
• GC scans the configured hash section and skips the configured hash/filepath sections; the schema walk covers the whole store minus reserved sections so pre-existing root-level objects stay reclaimable (coverage-based matching handles mixed layouts since metadata records full paths).
• filepath validation stays spec-driven; overriding hash_prefix now moves the writer and the reservation together, so the earlier hardcoded-reservation fix is superseded and reverted.
• HASH_STORAGE_PREFIX removed (defaults remain as builder fallbacks). My earlier "fixed layout convention" comments are all corrected.

End-to-end test pins a store with hash_prefix="content" + schema_prefix="objects": writers place objects there, GC scans and collects there. Closes via #1479 as before.