Skip to content

feat(mg/adapt): land custom-MG parallel support + NVB adapt-on-top (stranded branch, closes #365)#369

Merged
lmoresi merged 56 commits into
developmentfrom
feature/land-custom-mg-parallel
Jul 16, 2026
Merged

feat(mg/adapt): land custom-MG parallel support + NVB adapt-on-top (stranded branch, closes #365)#369
lmoresi merged 56 commits into
developmentfrom
feature/land-custom-mg-parallel

Conversation

@lmoresi

@lmoresi lmoresi commented Jul 16, 2026

Copy link
Copy Markdown
Member

Closes #365.

This lands the stranded bugfix/custom-mg-parallel branch — the lost parallel-machine session's work (52 commits, +5,274/−115, last commit 2026-07-03) that was pushed to origin but never got a PR — reconciled with the post-quality-campaign development tip.

Co-landing decision

bugfix/custom-mg-parallel contains feature/adapt-on-top in full (0 divergent commits). Landing both together is the maintainer decision (2026-07-16): the custom-MG parallel work is built on the adapt-on-top NVB engine, and they were developed and tested as one stack.

What the branch delivers (summarised from its own commit log)

Custom-P geometric MG (utilities/custom_mg.py)

  • Generalized FMG hierarchy with BC-per-level reduction (set_custom_fmg), covering all solver families: scalar Poisson, SNES_Vector, Stokes velocity block, constrained Stokes (serial)
  • Parallel (np>1) custom-P FMG: rank-local point location (ghost-inclusive coarse coords), global-section reduced numbering, MPIAIJ transfers on the nested co-partitioned hierarchy
  • Cross-partition transfer for non-nested coarse tails: when the coarse mesh is partitioned independently of the fine mesh, an all-gathered coarse cloud lets every rank locate its fine nodes (cross_partition="auto")
  • Operator-faithful finest reduced map on adapt children (SNES setUp before reading the finest map)
  • auto_inject_custom_mg: solvers pick up a mesh-owned (adapt-child) hierarchy with no per-solver setup

Adapt-on-top (mesh.adapt returns a refined child)

  • mesh.adapt(metric) now performs nested skeleton-based refinement on top of the static base and returns a child mesh owning a [base L0 … base finest, refine-1 … refine-n] custom-P MG tail (one MG level per refinement step); the old in-place MMG adapter is renamed mesh.remesh() (with a deprecated adapter="mmg" shim)
  • NVB graded engine (engine="nvb"): newest-vertex bisection with bounded conforming closure — graded refinement, DOFs concentrated at the feature; native uwnvb DMPlexTransform (utilities/_nvb_transform.pyx) runs it in parallel, in-place/co-partitioned, bit-confluent serial↔parallel; serial NVBMesh cell-list fallback when the extension is absent
  • Callable exact-distance metrics (resolved at each refined level, not P1-aliased), Surface.remap_to(mesh) + director property for adapt-on-top faults
  • Parent/child variable transfer: copy_into/add_into dispatch on the relationship kind (submesh restrict/prolongate vs refinement prolongate/restrict)

Reconciliation (merge commit 8576118)

The branch predates the entire quality campaign (Waves A–E, #363, #364, #366, #368). Development's refactored structure wins in conflicts; the branch's functionality is ported into it.

File Conflict Resolution
discretisation/discretisation_mesh.py branch's +506 vs the #344 Mesh.__init__ phase-method refactor development's structure wins (_load_dm_from_file / _patch_boundary_enum); branch's lineage + mesh-owned-MG attributes (_relationship_kind, _custom_mg_coarse_meshes, _custom_mg_builder) ported into _patch_boundary_enum; remesh docstring combines dev's modern access pattern with the branch's rename. All other branch machinery (adapt/_adapt_nested/remesh, _refine_prolongate/_refine_restrict, _wrap_coarse_level) auto-merged
cython/petsc_generic_snes_solvers.pyx branch's Stokes MG hooks vs the #342/#366 de-duplicated Picard/Newton block development's single solve block wins; branch's unconditional auto_inject_custom_mg(self, field_id=0) ported into it (SNES_Scalar / SNES_Vector hook sites auto-merged)
pixi.toml / pixi.lock development's side verbatim (no environment changes)

Compliance notes: the ported/merged adapt code contains no points - pStart coordinate-row arithmetic (it works via _get_coords_for_basis, DM centroids, and its own global-section reduced maps), so the #360/#363 navigation fixes are respected. No live callers of the old in-place adapt() remain in src/ (only docstring mentions, pre-Charter drift left for the follow-up de-drift pass).

Gates (amr-dev env, clean rm -rf build/ && ./uw build; _nvb_transform compiles and imports)

Gate development baseline merged branch
pytest tests/ -m "level_1 and tier_a" 382 passed, 12 skipped, 2 xfailed, 1 xpassed 382 passed, 12 skipped, 2 xfailed, 1 xpassed (+40 new collected items from the branch's test files, deselected as level_2/tier_b — nothing lost)
Branch's own suite serial (0835, 0836, 0837, 0838, 0839, 1016, 0810, 0830) 48 passed
np2: 0839 (NVB parallel adapt) + 1017 (custom-MG parallel) 12 passed
np4: 0839 + 1017 1017: 4 passed, 1 failed (pre-existing) 0839: 6 passed; 1017: 5 passed, 1 failed (same pre-existing)
np2 + np4 campaign regression (0775 deform-kdtree #360 guard, 1062, 1064) 10 passed at np2, 10 passed at np4
NVB confluence (0839 partition-independent asserts) green at np1, np2, np4
Style gate (check_deprecated_patterns.py) green green, no allowlist additions (one pre-Charter except-pass in custom_mg.py fixed minimally with a sanctioned-failure-mode comment)
Docs build 38 warnings 38 warnings — the warning set is byte-identical to development's (the two new design docs are wired into the toctree; an earlier "37, one fewer" reading was a Sphinx incremental-build artifact)

Pre-existing np4 failure (NOT from this landing): test_1017::test_parallel_custom_fmg_stokes_constrained fails at np4 with a marginal tolerance miss (rel-L2 0.00562 vs 0.005) on unmerged development (0.0056254) and on the merged branch (0.0056239) identically; it passes at np2 on both. The test was skip-guarded behind #291 on the branch tip and auto-enabled when #291 was fixed on development. Suggest tracking as its own small issue (tolerance vs genuine np4 conditioning of the constrained velocity block).

Flags for the maintainer

  • Committed generated C: the branch ships src/underworld3/utilities/nvb_transform.c (1,207 lines) as a checked-in generated file alongside _nvb_transform.pyx. Left exactly as the branch shipped it — style/build question for a follow-up.
  • The branch's code was written pre-Charter; per the landing scope it was not restyled (only the one style-gate blocker fixed). A de-drift pass (REMEDIATION-WORKLIST D-79..D-87 custom_mg rows, WC-04 builder= unification) is unblocked once this merges.
  • Docstring mentions of mesh.adapt() with the old in-place meaning remain in adaptivity.py / surfaces.py — pre-existing drift for the de-drift pass.

Post-review hardening (adversarial review, 2026-07-16)

The review found no defects in the merge decisions but flagged the un-reconciled user-facing surface of the adapt -> remesh rename (commit 01e9972):

  • Legacy-call guard: mesh.adapt(metric) with no new-API keyword now raises a TypeError explaining that the in-place metric adaptation is mesh.remesh(metric) and that adapt(...) returns a NEW child mesh (opt in with any keyword, e.g. max_levels=2 / engine="nvb"). No silent redirection — the return semantics differ. All existing callers (branch tests, shipped skills) already pass explicit keywords. New tests/test_0834_adapt_legacy_call_guard.py (level_1, tier_a, 4 tests) pins the guard, the message content, and the keyword opt-in.
  • Stale doc surfaces updated to remesh where in-place semantics are meant: docs/advanced/mesh-adaptation.md, adaptivity.py docstrings, the _set_coords guidance message in discretisation_mesh.py, _replace_from_adapted_mesh (mesh variables), and Surface._on_mesh_adapted / refinement-metric docstrings in surfaces.py.
  • nvb.py Moving mesh (mesh.deform / free surface / r-adapt) crashes at np>1: _build_kd_tree_index nav_coords[points - pStart] out of bounds #360 nitpick marked: TODO(#360) comments on the two coordinate-row == vertex-point-order assumptions (from_dm, to_dm). Serial-only reachable (np>1 requires the native transform and raises before these paths); the assumption holds for the fresh createFromCellList / undistributed base DMs this engine sees.
  • Residual footgun note (accepted, not over-engineered): a caller who passes an explicit new-API keyword on a refined mesh and still discards the returned child gets no warning — but they have explicitly opted in to the child-returning API at that point; the ambiguous legacy shape (bare metric, optionally verbose=) is fully covered by the guard.
  • _registered_children (write-only WeakSet) is left as-is: it is the stranded author's scaffolding for the marker-replay checkpoint follow-up.

Re-run gates after hardening: adapt/NVB serial suite 52 passed (48 + 4 new guard tests); style gate green (no allowlist additions); docs build 38 warnings, set identical to development.

Underworld development team with AI support from Claude Code

lmoresi added 30 commits June 28, 2026 10:31
Add set_custom_mg() + utilities/custom_mg.py: drive PCMG geometric multigrid
with a prolongation we build ourselves (barycentric or RBF) from independent,
possibly non-nested coarse meshes. Coarse operators come from Galerkin RAP
(P^T A P), so only the prolongation list is supplied. This decouples geometric
multigrid from a nested refine() hierarchy.

Mechanism: inject the custom P before the first PCSetUp (avoids the
MatProductReplaceMats shape error a live matrix swap triggers under Galerkin),
and KSPSetDMActive(OPERATOR, False) so PETSc uses our explicit P instead of
re-deriving DM-hierarchy interpolation. Finest level reduced to the
BC-eliminated global ordering; coarse levels use full DOF coords.

Validated (scalar Poisson, refinement=2 box): custom barycentric P from
independent coarse meshes reaches FMG iteration counts (1-3 vs FMG 2) and the
same solution. On a mesh with NO refine hierarchy (FMG unavailable -> GAMG, 13
iters) custom-P geometric MG converges in ~9 iters. test_1015 (4 tests);
test_1014 (11) still pass.

Single-field (scalar/vector) only; Stokes velocity-block path is a follow-up.

Underworld development team with AI support from Claude Code
…coped SBR

Phase 1 of hardening the custom-prolongation work into a generalized FMG hierarchy.

- CustomMGHierarchy: adapter-agnostic multi-level hierarchy that builds custom-P
  transfers (barycentric/rbf) with essential BCs applied at EVERY level
  (transfers map reduced->reduced). This is the load-bearing invariant: on an
  exactly-nested hierarchy, omitting per-level reduction makes a coarse boundary
  DOF coincide with a BC-removed fine DOF -> zero column -> singular Galerkin
  coarse operator. build() guards against zero columns.
- set_custom_fmg(): register a hierarchy + per-level solver factory; built and
  installed at solve() time (build-time injection, DMActive(OPERATOR,False),
  Galerkin RAP). Scalar / single-field-vector (top-level PC); Stokes velocity
  block is Phase 2.
- sbr_refine / sbr_refine_where: local Skeleton-Based Refinement (no MMG,
  on-rank, conforming) with SCOPED dm_plex_transform_type (leaking it globally
  breaks UW's uniform refine() with err73).
- Legacy finest-only path kept for back-compat (test_1015).

Validated: scalar jump-coefficient Poisson on a 5-level (3 uniform + 2 SBR)
hierarchy converges in 3 FMG iters vs GAMG 46, solution matches to 2.4e-8.
test_1016 (3 tests); test_1015 (4) + test_1014 (11) still pass.

Design: docs/developer/design/GENERALIZED_FMG_HIERARCHY_AND_ADAPT.md

Underworld development team with AI support from Claude Code
Layer 1 (generalized FMG hierarchy) is an independent, working capability
(arbitrary nested/non-nested coarse grids -> geometric FMG, BC-per-level).
Current scope experimental: scalar/single-field, serial, factory API. Remaining
before merge-ready general feature: Stokes velocity-block, drop the factory
(label-based BC reduction), parallel.

Underworld development team with AI support from Claude Code
Integrate the generalized FMG hierarchy into the saddle-point solver's
velocity sub-block. set_custom_fmg(..., field_id=0) now drives geometric
multigrid on the velocity block with our supplied (barycentric / RBF)
prolongations + Galerkin RAP.

The velocity sub-PC is unreachable until the monolithic Jacobian is
assembled (PCFieldSplit forms A_vv via MatCreateSubMatrix; snes.setUp
builds structure only -> err73). So _install_velocity_block_transfers:
forces a Jacobian assembly (computeFunction + computeJacobian at the zero
guess; throwaway max_it=0 fallback), reaches the velocity sub-PC,
reset()s it and rebuilds a FRESH PCMG from our P (mirrors the proven
standalone recipe; sidesteps the MatProductReplaceMats live-swap bug),
re-attaches the coupled Stokes nullspace. _configure_pcmg now derives the
options prefix from pc.getOptionsPrefix() so both the scalar top-level PC
and the velocity sub-PC are configured correctly.

inject_custom_mg is wired into SNES_Stokes_SaddlePt.solve (guarded no-op
unless set_custom_fmg was called), injected after setFromOptions /
nullspace, before the real solve.

Validated (SolCx eta_B=1e6, 3-level nested hierarchy, in-solver via
solver.solve()): velocity block converges in 6 MG iters vs GAMG 198,
solution matches the GAMG reference to 1.5e-9. test_1017 (barycentric +
rbf). test_1014/1015/1016 unchanged (20 pass total).

Underworld development team with AI support from Claude Code
…, Step 1)

Underworld development team with AI support from Claude Code
Each coarse level's BC-constrained reduced map is now derived directly
from the coarse mesh DM via _coarse_reduced_map: clone the DM, copy the
(built) finest solver's fields + DS onto it, createDS, read the global
section. The DS carries UW's exact essential-BC definitions and is a
topology-independent discretisation spec, so it constrains the matching
boundary DOFs on ANY coarse mesh that shares the solver's boundary labels
(nested or not).

This removes the throwaway-solver factory (heavy, leak-prone, and an API
wart): set_custom_fmg and CustomMGHierarchy.build no longer take a
level_solver_factory. Leak-free — DM ops only, no SNES / JIT.

Validated identical to the old factory path (reduced maps byte-equal:
126 + 510 DOFs on the SolCx 3-level hierarchy). test_1014/1015/1016/1017
all green (20). custom-P vs native FMG unchanged (6 vs 5 iters, sol match
3.7e-10).

Underworld development team with AI support from Claude Code
Underworld development team with AI support from Claude Code
…tial)

The reduced maps use rank-local DOF indices and the prolongations assemble
as serial AIJ, so at np>1 custom-P would silently build wrong P (or hit a
cryptic broadcast error). Add _require_serial: set_custom_fmg and
inject_custom_mg now raise a clear NotImplementedError on >1 ranks,
pointing to preconditioner='fmg'/'gamg'. Serial behaviour unchanged.

Parallel test tests/parallel/test_1017_custom_mg_serial_guard_mpi.py
(np=2) asserts both the set-time and legacy solve-time guards fire.

Full parallel custom-P (nested co-partitioned, rank-local point location +
ghosting, MPIAIJ assembly with global-section reduction) remains a
designed fast-follow.

Underworld development team with AI support from Claude Code
Underworld development team with AI support from Claude Code
…(Phase 1, Step 3)

The hierarchy path now builds parallel-correct transfers. Each rank builds
its block of P rank-locally: ghost-inclusive coarse coords mean every owned
fine node lands in a local coarse simplex (verified 0 misses np=2/4), and
the reduced global numbering rides the DM global section via a clean
local->global-reduced map (_level_dof_layout scatters owned global indices
out through globalToLocal; constrained DOFs stay -1, ghosts resolve to the
owner's global index). Transfers assemble as MPIAIJ (owned fine rows, global
coarse cols incl. off-rank); constrained coarse DOFs drop -> reduced->reduced.
Parallel zero-column guard via P^T·1 + allreduce.

_coarse_dof_layout reuses the leak-free copyDS trick for coarse levels.
CustomMGHierarchy.build branches serial (scipy CSR) vs parallel (MPIAIJ).
The guard now blocks only the legacy finest-only path (still serial).

Validated np=1/2/4: scalar Poisson custom-P 4 iters and Stokes SolCx
velocity block 6 iters, both matching the GAMG reference (and each other
across rank counts). New tests/parallel/test_1017_custom_mg_parallel_mpi.py
(scalar + Stokes correctness + legacy serial-guard). Serial 20 unchanged.

Underworld development team with AI support from Claude Code
Underworld development team with AI support from Claude Code
…+ Constrained tests)

Verify custom-P works for every solver family that consumes the mesh, so a
mesh-owned adapted hierarchy drives MG uniformly:

- SNES_Vector.solve was MISSING the inject_custom_mg hook (only SNES_Scalar
  and SNES_Stokes_SaddlePt had it) — so Vector_Projection silently stayed on
  GAMG. Add the hook (mirrors the scalar one). All 8 solver-subclass solve()
  overrides delegate to a base solve(), so this completes coverage:
  scalar branch -> Poisson/Darcy/Projection/AdvDiff/Diffusion;
  vector branch -> Vector_Projection; velocity-block -> Stokes/VE/Constrained/NS.

- test_1017: add SNES_Vector (Vector_Projection, top-level vector PC,
  field_id=None) and Stokes_Constrained (free-slip multipliers, grouped [p,h]
  split, field_id=0) serial tests — both drive custom-P 'mg', constrained
  matches analytic SolCx to 4.4e-4.

- parallel test: add Vector (passes np=2) and Constrained. The Constrained
  parallel test is SKIPPED: Stokes_Constrained is not parallel-safe yet — it
  segfaults at np>1 independently of custom-P (canonical
  test_1062_constrained_solcx also segfaults at np=2 under plain GAMG). It
  auto-enables when the constrained solver becomes parallel-ready.

Serial: custom_mg 22 + projections + constrained green. Parallel np=2:
scalar + Stokes-velocity + vector + legacy-guard pass.

Underworld development team with AI support from Claude Code
… limitation

Underworld development team with AI support from Claude Code
Underworld development team with AI support from Claude Code
Captures the Layer-2 design: integrate a nested on-rank SBR adapter into
mesh.adapt(adapter="sbr", max_levels, node_budget) with the existing
isotropic-metric interface; re-adapt (non-cumulative, no node translation
— unlike MMPDE) discarding the SBR top and re-marking from the static base
each step; mesh-owned hierarchy that ALL solvers consume; Eulerian REMAP
field transfer via the existing remesh machinery; no-redistribution as a
correctness requirement (custom-P parallel needs the finest co-partitioned
with the static coarse tail); static-coarse/transient-fine data model with
top-only transfer rebuild + coarse-transfer caching; marker-sidecar
checkpoint scheme (designed, deferred). Phased plan + invariants.

Underworld development team with AI support from Claude Code
SBR adapt-on-top + custom-P + REMAP + non-cumulative re-adapt loop proven
end-to-end with Layer-1 code only; np=2 confirms the no-redistribute /
co-partitioned correctness (refined finest stays co-partitioned with the
static base coarse levels). 4 MG iters, matches GAMG.

Underworld development team with AI support from Claude Code
…k fieldsplit

The Stokes velocity-block injection forces computeJacobian, then reaches the
fieldsplit BEFORE the SNES solve. SNESSolve normally wires the freshly-assembled
Jacobian into the KSP/outer-PC; doing it ourselves was missing, so for some
configurations (e.g. Stokes on an SBR-refined child mesh with a mesh-variable
bodyforce) the outer PC carried an UNASSEMBLED operator and PCSetUp failed with
"Matrix must be set first" (err73). Fix: snes.getKSP().setOperators(J, Pmat)
after forcing assembly, before outer_pc.setUp().

Found via the Layer-2 adapt-on-top prototype (Stokes on a fault-refined child).
Diagnosed: ksp.A set but asm=False while J.assembled=True -> operator not wired.

New regression test_custom_fmg_stokes_on_sbr_child (Stokes velocity-block custom-P
on an SBR child with a mesh-variable bodyforce) — fails pre-fix, passes after.
test_1014/1015/1016/1017 green (serial); parallel np2 unchanged.

Underworld development team with AI support from Claude Code
Wire the Layer-2 nested adapt-on-top path into the mesh API (de-risked
prototype now integrated). The naming settled as: adapt RETURNS a refined
child; the old in-place MMG adapt is renamed remesh.

mesh.adapt(metric, max_levels=2, node_budget=None, builder, adapter="sbr"):
  - marks base-finest cells whose size exceeds the metric target, up to
    max_levels SBR passes (on-rank, distribute=False; node_budget keeps the
    highest-metric cells first); metric sampled by evaluate (serial) /
    global_evaluate (np>1);
  - wraps the refined finest as a child (child.parent = self,
    _relationship_kind="refinement", registered in _registered_children);
    the base mesh is untouched (re-adapt is non-cumulative);
  - the child owns the static coarse tail (_custom_mg_coarse_meshes, cached
    on the parent) so solvers auto-pickup geometric MG.

Solver auto-pickup: custom_mg.maybe_inject_custom_mg (called from the four
solve hooks) builds a CustomMGHierarchy from a mesh-owned coarse tail when no
solver-set hierarchy is present (field_id 0 for the Stokes velocity block,
None for scalar/vector) — every solver on an adapted mesh drives geometric MG
with no per-solver set_custom_fmg. Clean no-op for plain (unadapted) solves.

copy_into/add_into dispatch on the child kind: refinement children prolongate
parent->child (FE-exact barycentric custom-P / global_evaluate REMAP at np>1)
and restrict child->parent (nearest-node injection / global_evaluate). Submesh
children keep the existing subpoint_is path.

mesh.remesh(metric) is the renamed in-place MMG path (byte-identical body);
mesh.adapt(adapter="mmg") is a deprecation shim that warns and forwards.
Migrated the MMG regression tests (test_0810/0830, ptest_0763) to remesh and
hardened test_0810 with a runtime backend probe (skips cleanly where MMG is
non-functional).

Validation: tests/test_0835_sbr_adapt_on_top.py (7 tests, tier_b; SBR needs no
MMG build) + study script validate_mesh_adapt_integration.py (serial + np=2,
all OK): child link + finer, solver auto-pickup pc=mg 4 iters == GAMG,
copy_into both directions, non-cumulative re-adapt, node_budget localises,
shim warns. Submesh + custom-mg regressions (test_0771/0772/1015/1017) green.

Docs: LAYER2_SBR_ADAPT_ON_TOP.md "Implementation" section (as-built API);
submesh-solver-architecture.md reframed under the parent/child DAG (submesh
and refinement are two kinds of child).

Underworld development team with AI support from Claude Code
Canonical SolCx (eta-jump 1e6) Stokes on an SBR-adapted child refined near the
viscosity jump (128/512 cells -> child 992, genuinely local). The velocity
block auto-picks-up the mesh-owned custom-P FMG (field_id=0, full cycle, fgmres
outer) with NO set_custom_fmg, converges in 6 iters == GAMG, matches the GAMG
solution to rel 0 and the analytic velocity to GAMG accuracy. Validated serial
+ np=2 via demo_stokes_on_adapted_child.py.

Underworld development team with AI support from Claude Code
Previously max_levels>1 collapsed all SBR levels into a single base-finest ->
child custom-P transfer, so the whole refinement got just one MG level (the
coarse-grid correction made one big jump). Now each SBR pass becomes its own MG
level: the child's mesh-owned tail is

  [base L0 ... base finest] + [SBR level 1 ... SBR level n-1]

and the solver appends its own mesh (the finest SBR level = child), so transfers
between consecutive levels each span a single refinement. _adapt_sbr keeps every
intermediate SBR DM, wraps it as a coarse-level mesh (transient, on the child),
and appends them after the static base tail.

Effect: MG levels now scale with depth (4/5/6 for max_levels 1/2/3), matching
the design doc's intended [base levels] + [SBR levels] hierarchy. The 2-level
box-fault Stokes solve improves 4 levels/8 iters -> 5 levels/5 iters; np=2
max_levels=2 Poisson drives 5 levels, converges in 2 iters (intermediate SBR
meshes co-partition correctly, no redistribute).

test_0835::test_each_sbr_level_is_its_own_mg_level asserts MG levels =
base levels + n_sbr and that a 2-level solve drives all of them. 9 tests pass.

Underworld development team with AI support from Claude Code
…honestly

Investigation (recorded in the docstrings): a per-level node_budget cannot make
the finest SBR level hug a feature, because SBR's conforming closure re-refines
the whole connected patch from any seed (measured: 30/60/120/240 seeds ->
identical output). The funnel is controlled by the METRIC SHAPE: a wedge (size
growing with distance, Surface.refinement_metric(..., profile="linear")) makes
each level's band ~half the previous width so the finest level concentrates at
the feature with bounded per-level growth; a flat-core Gaussian refines the
whole core uniformly at every level (no funnel).

- adapt(): new Notes section on controlling the grading/funnel via the metric;
  node_budget docstring corrected to state it caps SEEDS only (not output DOFs)
  and points to the metric profile for the funnel.
- test_0835::test_wedge_metric_funnels_finest_level_to_feature asserts the wedge
  metric's per-level marked-cell growth (finest/coarsest) is milder than a flat
  gaussian's on the same fault (the funnel signature). 10 tests pass.

Underworld development team with AI support from Claude Code
…n-top)

Records the SBR/longest-edge grading limitation (uniform-patch refill, measured)
and designs the fix: newest-vertex bisection (NVB), whose closure is provably
bounded (O(#marked)) and which produces the graded level+1/level+2 staircase
keeping simplices (p4est ruled out).

Covers: NVB rule (refinement edge / newest-vertex), bounded conforming
completion, initial labelling, two implementation routes (Route A = serial numpy
on the triangulation + DMPlex-from-cells, proposed first; Route B = native PETSc
DMPlexTransform, parallel, deferred), integration seam (engine="nvb" in the
existing _adapt_nested; custom-P hierarchy unchanged since transfers are
coordinate-built), parallel/checkpoint considerations, a complexity table, a
4-step phasing, and alternatives (red-green, longest-edge, p4est).

docs/developer/design/NVB_GRADED_ADAPT.md.

Underworld development team with AI support from Claude Code
…tion

Add the conceptual resolution of "why can we build an embedded uniform patch but
not refine inside it without draining to the edge": graded meshes are valid and
constructible; longest-edge bisection just has no local termination in a uniform
bisected patch, so its closure drains to the existing outer interface and cannot
carve a new interior interface (measured: 1 centre cell of an r<0.25 patch ->
+3592 cells out to the patch edge, both structured and unstructured). NVB's
combinatorial marked edge restores local termination -> bounded interior
interfaces = grading.

Add a 3D section: NVB generalises to tetrahedra (Bansch/Maubach/Stevenson,
bounded closure in n-D); longest-edge is worse in 3D; Route A carries over to
(N,4) cells; octree stays hex-only (ruled out).

Underworld development team with AI support from Claude Code
Phase-1 core of the newest-vertex-bisection design: ~135 lines of numpy
(nvb_prototype_2d.py, NVBMesh) implementing the newest-vertex data model and the
recursive compatible-bisection with bounded conforming closure.

Validated against the SBR longest-edge pathology on the same nested-disk bullseye:
- one cell deep in a uniform patch -> +2 cells, LOCAL (r 0.016-0.030), where SBR
  drained +3592 cells to the patch edge;
- the finest band stays confined (gen 6 at r<0.126) where SBR refilled to r=0.337;
- conforming at every step (0 hanging nodes, 0 over-shared edges);
- 3024 cells vs SBR's 7968 for the same target;
- renders as clean concentric generation rings (true grading, not a uniform core).

So the hard part (the closure algorithm) works. Remaining Phase-1 = DMPlex wrap +
boundary/region label transfer (engineering). Design note updated with the result
and the prototype reference.

Underworld development team with AI support from Claude Code
Add depth bookkeeping (children inherit parent depth+1) and a depths() accessor,
so refinement level can be coloured cleanly on a NON-UNIFORM base (area-derived
generation conflates base size variation with refinement). Validated: NVB fault
funnel on an irregular Delaunay base grades cleanly by depth (0..7), conforming
(0 hanging nodes) and bounded (no cascade) at every step — where longest-edge SBR
cascaded across the domain on an unstructured mesh.

Underworld development team with AI support from Claude Code
… A vs B

Sharpen the parallel section: the binding requirement is preserving the parent's
decomposition (co-partitioned finest, no redistribute) so custom-P's rank-local
point location holds — not load balance. This DISQUALIFIES Route A
(createFromCellList) for parallel: it loses the parent point-SF (serial DM or
fresh DMPlexDistribute partition); preserving it would need manual SF rebuild +
consistent cross-rank marked-edge bisection + closure exchange. SBR preserves it
today via in-place adaptLabel (SF propagated by the transform framework) — the
model to match. So Route A = serial stepping stone; Route B (native
DMPlexTransform) is the real parallel implementation, justified by decomposition
preservation. Adds the np>1 acceptance test (child centroids in locally-owned
base cells; coarse-tail partition bit-identical).

Underworld development team with AI support from Claude Code
Port the validated NVB serial core into src as utilities/nvb.py (NVBMesh:
newest-vertex data model, recursive bounded compatible-bisection, from_dm /
to_dm with boundary-edge + region-cell label transport, conformity /
similarity-class diagnostics). createFromCellList needs consistent CCW winding
(the (peak,b0,b1) refinement-edge order is geometry-agnostic) — reorient the
exported cell list only; labels matched by vertex pair (coordinate, never index).

custom_mg.nvb_refine mirrors sbr_refine (single-shot). Mesh._adapt_sbr renamed
to _adapt_nested with engine='sbr'|'nvb' (default sbr, behaviour unchanged). The
NVB path drives a persistent NVBMesh across 2*max_levels generations (1->2 area
halving) so the refinement-edge labelling — hence the similarity-class bound —
propagates parent->child; each generation snapshots a DM for the engine-agnostic
coordinate-based custom-P FMG tail. NotImplementedError for engine='nvb' at np>1
(cell-list rebuild loses the parent decomposition the custom-P parallel path
needs — that is Route B) and for dim!=2.

Validated end-to-end serial: graded NVB child (fewer DOFs than the SBR uniform
patch), Poisson FMG (5 levels) matches GAMG bit-identically (exact err 3.5e-10),
SolCx Stokes (eta jump 1e6) velocity-block FMG matches GAMG iter-for-iter (8) and
bit-identically. Existing SBR tests (test_0835) all pass.

Underworld development team with AI support from Claude Code
tests/test_0836_nvb_graded_adapt.py (tier_b, 12 tests): engine-level properties
(conformity 0 hanging/0 over-shared every generation; bounded closure — single
cell local +O(1) and #added ≤ C·#marked; shape-regularity via bounded similarity
classes; graded bullseye + fault funnel confine the finest generation near the
feature with coarse cells surviving) and the integrated path (graded child link/
engine tag, fewer DOFs than the SBR patch, non-cumulative re-adapt, Poisson FMG
matches GAMG, SolCx Stokes η-jump-1e6 velocity-block FMG matches GAMG, 3D guard).

Update NVB_GRADED_ADAPT.md: phases 2 (DMPlex wrap) + 3 (Layer-2 wiring) DONE with
the resolved gotchas (CCW winding, vertex-pair label match, edge+vertex labels);
Route B (parallel native transform) is next and the seam is ready.

Underworld development team with AI support from Claude Code
…locking

Audit against the 'must not block parallel' requirement:
- co-partitioning invariant is structurally preserved by NVB (bisection replaces a
  cell with same-owner children; closure only triggers same-owner bisections), so
  any in-place transform keeps the child co-partitioned with the coarse tail — the
  custom-P parallel transfer's load-bearing assumption. Only Route A's rank-0
  cell-list rebuild breaks it. Measured bar: custom-P FMG already solves np=2 on a
  locally-refined SBR adapt-on-top child (4-level mg, exact err 2e-11).
- nothing in the serial code blocks Route B: engine= dispatch seam, engine-agnostic
  coordinate custom-P tail (parallel path already built+proven), lineage/copy_into/
  snapshots all shared; NVBMesh cell-list to_dm + label transfer are simply not on
  the parallel path.
- irreducible hard kernel = the parallel conforming-closure fixpoint; NVB's
  labelling is cross-rank-consistent for free (geometric longest-edge seed +
  deterministic midpoint rule). No PETSc transform bisects a labelled edge, so
  Route B needs a new C transform type (reusing PETSc bisection-closure infra).
- options: manual-SF pure-Python (fragile) vs native C transform (robust);
  recommend prototyping the parallel closure-fixpoint in Python first to de-risk.

Underworld development team with AI support from Claude Code
lmoresi added 21 commits July 1, 2026 15:12
…(Route B Stage 2a)

Fills in the Stage-1 "uwnvb" transform (previously an identity gate) with a
self-contained clone of PETSc's SBR longest-edge bisection, registered from UW
with no PETSc rebuild. It reproduces refine_sbr byte-for-byte — identical
triangulation (per-cell vertex geometry, not just counts) for every marking
pattern including full uniform refinement — which exercises the entire native
transform stack on the pinned PETSc 3.25: registration, the SetUp closure,
DMLabelPropagate cross-rank propagation, cell transform, subcell orientation,
and coordinate mapping.

Depends only on: exported libpetsc symbols (DMPlexPointQueue*, DMLabelPropagate*,
the Identity helpers, registration); static-inline header helpers
(DMPolytopeTypeGetArrangement, ComposeOrientation, DMPlex_DistD_Internal); and
four non-exported PETSc helpers reimplemented verbatim (SetDimensions,
MapCoordinatesBarycenter, and the SEGMENT/TRIANGLE cases of the regular
cell-refine and subcell-orientation) — all verified byte-identical to the 3.25.0
source. No PETSc rebuild, no fork.

This is the SBR-equivalent base; the graded newest-vertex edge choice layers on
top (a single-bisection, multi-pass transform — see the design note's Stage 2b
finding for why a one-pass double/triple-split scheme cannot grade).

- tests/test_0837_nvb_native_transform.py: uwnvb == refine_sbr equivalence
  (tier_b, skips cleanly without the custom-PETSc extension).
- docs/developer/design/NVB_GRADED_ADAPT.md: Stage 2a done; Stage 2b finding and
  the single-bisection multi-pass redesign; FMG-hierarchy-index label reuse for
  checkpointing (to investigate next phase).

Underworld development team with AI support from Claude Code
…ransform (Route B Stage 2b)

Adds graded newest-vertex refinement on top of the Stage-2a native transform, and
fixes the grading failure of the earlier one-pass approach. A single conforming
DMPlexTransform pass is forced into green/blue multi-edge splits that corrupt the
refinement-edge structure; instead we refine as repeated CONFORMING single
bisections of *compatible* refinement edges, which keeps every child's slot
trivially correct and the closure bounded.

As built (all in C, so the SF-sensitive parallel logic stays in one place):
- each triangle carries a uwnvb_refedge cone-slot label (refedge = cone[slot];
  longest-edge seed on the base). No generation label is needed — single bisection
  makes each child's newest vertex the new midpoint, so its refinement edge is the
  edge opposite that midpoint (geometric).
- uwnvb_bisect: a dumb transform that single-splits exactly the edges named by a
  label, reusing the Stage-2a cell-transform/orientation/coordinate ops.
- UWNVBRefine(dm, want_label): closure (grow the bisect set by LEPP blocking
  neighbours) + drain (each sub-pass single-splits the compatible refinement edges
  — an edge that is the refinement edge of all its incident cells, reconciled
  across ranks by an MPI_LAND PetscSFReduce — then rebuilds the slot + set). Exposed
  as underworld3.utilities._nvb_transform.refine(dm, want_label).

Validated (tests/test_0838, tier_b): refinement edges match the serial NVBMesh
reference EXACTLY over four uniform refines (0 mismatch, 64/128/256/512); a mark
deep in a 2x-refined patch adds +2 cells (== NVBMesh) versus SBR's +4824; a
shrinking bullseye grades to 805 cells versus SBR's 102868 uniform-patch cells;
conforming (0 hanging / 0 over-shared) and deterministic. Stage-2a
(uwnvb == refine_sbr, test_0837) still passes. Runs conforming at np=1/2/3.

Remaining for full parallel confluence: the closure is on-rank only, so a closure
crossing a partition under-refines slightly there (still conforming — the
SF-reconciled compatibility test prevents cross-partition hanging nodes). The
SF-reconciled cross-rank closure (DMLabelPropagate of requested refinement edges,
as SBR's SetUp does) is the next step, followed by wiring _adapt_nested(engine=nvb)
at np>1. See docs/developer/design/NVB_GRADED_ADAPT.md.

Underworld development team with AI support from Claude Code
Replaces the on-rank-only closure with the SF-reconciled version: each C-cell's
refinement edge is marked in a `req` edge label and propagated across the point
SF with DMLabelPropagate (the same mechanism SBR's SetUp uses — edges are shared
SF points, cells are not), so a cell blocked by an off-rank neighbour marks that
neighbour on its owning rank. Each rank derives its own C-cells locally from the
requested edges incident to them. This makes the LEPP closure cross-rank complete.

Serial behaviour is unchanged (tests 0837 + 0838 still pass); the driver runs at
np=1/2/3 producing conforming, graded output (locally 0 over-shared; 0 hanging
nodes at np=1; SF-reconciled compatibility prevents one-sided edge bisection).

Known limitation (documented in NVB_GRADED_ADAPT.md): the parallel result is valid
but not yet bit-confluent — global cell counts differ by a few cells across np
(uniform refine 159/352/760 at np=1 vs 157/346/743 at np=2). This is a drain
sub-pass sequencing issue, NOT the closure (it affects uniform refine, whose
closure is trivial), NOT the longest-edge seed (no exact ties → geometrically
deterministic), and NOT the SF-LAND compatibility indexing (verified point-SF
ilocal = point numbers). Confluence is a focused follow-up (per-sub-pass
serial-vs-parallel comparison), then Stage 2c wires _adapt_nested(engine=nvb) np>1.

Underworld development team with AI support from Claude Code
…ction set

The nested-distributed-refine SEGFAULT and the parallel non-confluence were the
same single bug: the drain marked edges to bisect purely from local C cells, so a
shared edge could be split on one rank and kept whole on its neighbour. That makes
DMPlexTransformCreateSF map a split-edge child on one rank onto an unsplit vertex
on the other — a stratum-crossing point SF. On the next refine, DMLabelPropagate
reduces an edge's request onto that vertex (whose support is edges, not cells) and
reads out of range → SEGV; and the divergent split sets produced slightly
different (each valid) meshes per partition.

Fix: uwnvb_sf_lor — an MPI_LOR PetscSFReduce+Bcast of the bisect-mark array over
the point SF, so an edge chosen on any rank is split on every rank owning a copy.
Since agree[] is already MPI_LAND-reconciled (the edge is the refinement edge of
the incident cell on all sharers), splitting it everywhere is exactly the
conforming closure — the OR fixes the SF and completes the cross-partition closure
in one step. Serial is unchanged (no SF graph → no-op).

Now bit-confluent with serial at any communicator size: uniform 159/352/760,
graded bullseye 215, deep 5-level nesting 96/180/288/390/472 all equal at
np=1/2/3/4, 0 hanging nodes, stratum-consistent output SF. New regression test
test_parallel_confluence pins it (passes np=1/2/3).

Underworld development team with AI support from Claude Code
The four NVBMesh-reference tests compare per-rank local cells to the serial
reference engine, so they only make sense at np=1. Mark them serial_only;
test_parallel_confluence (partition-independent global counts) is the parallel
pin. np=2/3: 1 passed + 4 skipped; serial: all 9 pass.

Underworld development team with AI support from Claude Code
…apt (parallel)

mesh.adapt(engine="nvb") now dispatches to the native uwnvb DMPlexTransform when
the compiled extension is present, giving graded NVB refinement in parallel (was
serial-only via the NVBMesh cell-list engine, which raised at np>1). The transform
is in-place — the child stays co-partitioned with the parent, so the mesh-owned
custom-P geometric-MG tail is valid at np>1 — and carries the boundary/region
labels forward automatically.

_adapt_nested: for engine="nvb" with the extension available, mark cells whose
current size exceeds the metric target and refine once per pass via
_nvb_transform.refine (up to 2*max_levels passes; a bisection halves the area).
Collective stop via allreduce(sel.size) so a rank with no local marks still joins
the refine when a neighbour's edge forces a closure bisection. metric evaluated
with global_evaluate at np>1. NVBMesh remains the serial fallback (raises at np>1
only when the extension is absent).

Verified: adapt(engine="nvb") is bit-confluent (child=366 cells at np=1/2/4),
boundaries + 4-level MG tail preserved; Poisson on the graded child drives
custom-P FMG converging in 3 iters == GAMG at np=1/2/4. New
test_0839_nvb_parallel_adapt.py (confluence + FMG acceptance, passes np=1/2/3);
existing SBR + serial-NVB adapt suites (22 tests) still green.

Underworld development team with AI support from Claude Code
Underworld development team with AI support from Claude Code
Adds test_solcx_stokes_velocity_fmg_on_nvb_child to test_0839: SolCx (η jump 1e6)
on a graded NVB child, velocity block auto-picks-up the mesh-owned custom-P FMG
(geometric mg, FULL/FMG cycle) and matches the GAMG reference. Confluent child
(800 cells) and a partition-independent solution. Verified np=1/2/4:
FMG vel-iters == GAMG (8/9/7), rel|u| ~0.

Underworld development team with AI support from Claude Code
… patchy grading)

adapt(metric, engine=...) now accepts any evaluatable metric (sympy / UWexpression),
not only a MeshVariable: metric_sym = getattr(metric_field, "sym", metric_field),
used in all three refinement branches.

Motivation: a baked P1 refinement_metric field stores M = 1/h², which for a typical
h_far/h_near ratio swings ~80x across one base cell. P1-interpolating that peaked
quantity on the coarse base aliases badly, so a thin finite-width feature gets
*patchy* refinement levels along its length (measured: along-fault h_target varied
3.9x even though the distance is ~0 all along the fault). Passing the analytic
wedge instead samples the metric exactly at each centroid → uniform along-feature
resolution (residual 1.3x is only the P1 distance field's own error; exact distance
gives 1.0x).

MeshVariable metrics are unchanged (metric_sym == metric_field.sym). New
test_adapt_accepts_analytic_metric (confluent, conforming, np1/2). Existing SBR +
NVB adapt suites (22) still pass.

Underworld development team with AI support from Claude Code
…refined level

The adapt metric is re-evaluated on the centroids of every refined NVB/SBR level,
but a distance-driven fault metric supplied as a P1 MeshVariable (or an expression
referencing Surface.distance.sym) is interpolated from the *base* mesh — so it
aliases: baking M = 1/h² into a P1 field swings ~80x across one base cell (patchy
along-fault levels, 3.9x h_target variation), and even an analytic wedge still rides
the P1 *distance* field (1.3x).

Let mesh.adapt() accept a plain callable metric(centroids) -> M, evaluated directly
at each level's centroids, and add Surface.refinement_metric_function(...) that
returns one built on the surface's EXACT signed distance. Surface.signed_distance /
unsigned_distance expose the exact point-to-polyline (2D) / implicit-distance (3D)
primitive at arbitrary query points; _compute_distance_field is refactored to share
it (_signed_distance_at) so the P1 field and the callable use one code path.

The distance now resolves itself at the new resolution as levels appear (on-fault
h_target uniform to ~1e-9, 1.0x). A coordinate-driven callable is also
partition-independent, so it needs no global_evaluate/swarm migration and the child
stays bit-confluent (206 cells at np=1/2/4 in the test; 1590 in the weak-fault demo).

tests/test_0839_nvb_parallel_adapt.py:
  test_adapt_accepts_callable_metric_exact_surface_distance (np1/np2).

Underworld development team with AI support from Claude Code
…uate is the field adapter

_adapt_nested had two implicit paths (callable vs field/expr) branched inside a
per-call helper. Collapse to a single eval_metric(centroids) -> M callable built
once: a user callable is used as-is; a MeshVariable/expression is wrapped in an
uw.function.(global_)evaluate adapter. So fn.evaluate is not a separate path — it
IS a callable in the framework, just the default one for a field/expression. One
code path, evaluated at each refined level's centroids.

This also documents the composition the wrapper makes obvious: a user callable may
call global_evaluate itself when the metric genuinely depends on a base field (e.g.
|grad T| for convection adaptivity) — with the caveat that a field is sampled from
the base-mesh interpolant (base-resolution, no self-sharpening, unlike exact
geometry) and that np>1 needs global_evaluate, not evaluate.

Behaviour unchanged: test_0839 (5) passes np1/np2.

Underworld development team with AI support from Claude Code
…on-top faults

remap_to(new_mesh): re-home a Surface onto an adapt() child (drops cached distance
fields bound to the old mesh; geometry unchanged; re-registers). Lets ONE fault
Surface drive both the geometry-only refinement metric (evaluated before the child
exists) and the child-side constitutive model (distance/normal live on the child) —
the nested SBR/NVB adapt returns a child and does not auto-notify surfaces the way
in-place remesh does.

director: unit surface-normal as a symbolic column vector = normalised grad of the
signed-distance field. The natural TI weak-plane director; exact for a planar fault.

Underworld development team with AI support from Claude Code
…on-top

# Conflicts:
#	src/underworld3/cython/petsc_generic_snes_solvers.pyx
#	src/underworld3/utilities/custom_mg.py
…ers on adapt children

An adapt() child owns a custom-P FMG coarse tail that solve() auto-picks-up via
maybe_inject_custom_mg. That tail is built to accelerate the Stokes VELOCITY block,
but the hook also fired for scalar solvers (field_id=None). For a semi-Lagrangian
advection-diffusion solver the assembled operator is boundary-reduced in a way the
coarse DS-copy reduced map does not reproduce, so the per-level BC reductions
disagree and the transfers don't chain — PCMG's Galerkin PtAP gets a rectangular A
(e.g. 12725x12486) and dies with PETSc error 60. This bit any AdvDiffusionSLCN on an
adapted mesh.

Fix, in maybe_inject_custom_mg (mesh-owned/opportunistic path only):
  - skip the auto-pickup when the solver carries a DuDt trace-back operator
    (semi-Lagrangian advection-diffusion) — its operator structure is incompatible
    and a scalar AD solve is cheap (GAMG is fine); an explicit set_custom_fmg() still
    works if wanted;
  - plus a dimensional guard (finest transfer must match the assembled operator and
    actually coarsen) as defense-in-depth for other field_id=None cases.
Restructured so the solver-set (set_custom_fmg) path is unchanged and errors there
still surface. Velocity-block (field_id=0) and P1-scalar-Poisson auto-pickup are
unaffected.

test_0839: test_advdiff_scalar_on_nvb_child_no_custom_mg_crash (raised PETSc 60
before; passes now). Full NVB suite 11/11.

Underworld development team with AI support from Claude Code
…tom_mg

"maybe_" is a hedging, non-descriptive prefix. The function is the solve-time hook
that AUTO-injects a custom-P MG hierarchy when one is available (solver-set via
set_custom_fmg, or mesh-owned on an adapt() child) and no-ops otherwise — so name it
for what it does. Pure rename across the solver pyx (4 call sites) + custom_mg.py +
the test comment; behaviour unchanged. NVB suite 11/11.

Underworld development team with AI support from Claude Code
The custom-P finest reduced map was read from the DM global section via
_reduced_map(solver.dm, field_id). On an adapt() child the section can be
read before the SNES finalizes it, so it could disagree with the assembled
operator the PCMG Galerkins against -> a rectangular finest transfer and a
bare PETSc error 60 in the PtAP. auto_inject_custom_mg worked around this by
SKIPPING custom-P for any semi-Lagrangian AdvDiffusion (DuDt present),
falling back to GAMG.

Fix, in CustomMGHierarchy.build:
- call solver.snes.setUp() before reading the finest reduced map, so the DM
  global section is the finalized space the operator lives on;
- validate the finest reduced-map size against the assembled operator
  (_assert_finest_matches_operator) and raise an actionable error instead of
  a cryptic error 60 if a genuine adapt-child section inconsistency ever
  arises. The mesh-owned auto-pickup wraps build() in try/except, so this
  degrades gracefully to the default preconditioner.

Remove the vestigial DuDt skip-guard: a semi-Lagrangian AdvDiffusion on an
NVB adapt child now installs custom-P geometric MG (PC 'mg') and solves
correctly, matching a default-preconditioner solve to iterative tolerance.

Tests: test_1016 gains test_finest_map_operator_mismatch_raises (guard fires)
and test_advdiff_on_nvb_adapt_child_gets_custom_mg (real auto-pickup path, no
skip, correct result). test_0838, test_0839, and the full custom_mg suite
(1014/1015/1016/1017, 47 tests) pass.

Underworld development team with AI support from Claude Code
… at np>1

The parallel custom-P path assembled each rank's prolongation block from that
rank's LOCAL coarse coords, which is correct only for CO-PARTITIONED NESTED
levels (refine() / on-rank adapt). For a genuinely non-nested coarse mesh
(independent gmsh mesh at a different cellSize) at np>1 the coarse and fine
partitions differ: a fine leaf on rank r can sit in a coarse cell owned by
rank s, so rank-local point location misses it -> zero columns (the guard
fired: 23 at np2, 48 at np4) or a wrong nearest-DOF fallback. So
set_custom_fmg([independent coarse meshes]) was serial-only.

Add a cross-partition transfer (_build_crosspart_transfer / _gather_coarse_cloud):
a coarse MG level is small, so all-gather the coarse node cloud (coords + each
node/component's GLOBAL reduced column index, -1 for constrained) deduplicated
by coordinate; every rank then locates its OWNED fine nodes against the FULL
coarse mesh. Columns are coarse global reduced indices (MPIAIJ handles off-rank);
constrained coarse DOFs stay as barycentric vertices but drop from the columns
(reduced->reduced). Fine rows stay rank-local.

Routing via cross_partition on CustomMGHierarchy / set_custom_fmg:
- "auto" (default): rank-local co-partitioned build first, rebuild a level
  cross-partition only if it has zero columns -> validated nested/adapt path
  stays bit-identical on the fast builder, non-nested tails fixed automatically;
- True: force cross-partition; False: force rank-local.

Validated: an independent (non-co-partitioned) coarse box tail converges in the
same iteration count as serial (6 iters) and matches a GAMG reference to ~1e-8
at np1/np2/np4. New tests/parallel test_parallel_custom_fmg_nonnested (np2, np4).
Nested co-partitioned parallel tests (scalar, Stokes velocity block, vector) and
the serial suite unchanged. Design note updated.

Underworld development team with AI support from Claude Code
Reconciliation merge landing the stranded bugfix/custom-mg-parallel branch
(issue #365: custom-MG parallel support + the NVB/SBR adapt-on-top engine
it contains, including feature/adapt-on-top in full) onto the
post-quality-campaign development tip.

Conflict resolutions:

- src/underworld3/discretisation/discretisation_mesh.py (2 hunks):
  development's refactored Mesh.__init__ wins (the #344 phase-method split:
  _load_dm_from_file / _patch_boundary_enum helpers). The branch's
  functional additions in the conflicted region — parent/child lineage
  attributes (_relationship_kind) and the mesh-owned custom-MG hierarchy
  attributes (_custom_mg_coarse_meshes, _custom_mg_builder) — are ported
  into _patch_boundary_enum beside parent/subpoint_is. Second hunk
  (remesh docstring example): development's modern access pattern
  (var.data / mesh.X.coords) combined with the branch's adapt->remesh
  rename (mesh.remesh(...) in the example).

- src/underworld3/cython/petsc_generic_snes_solvers.pyx (1 hunk):
  development's de-duplicated Picard/Newton solve block (#342/#366) wins
  structurally; the branch's functional change — unconditional
  auto_inject_custom_mg(self, field_id=0) replacing the guarded
  inject_custom_mg call, so mesh-owned (adapt-child) hierarchies are
  picked up without explicit set_custom_fmg — is ported into it. The
  SNES_Scalar/SNES_Vector hook sites auto-merged.

- pixi.toml / pixi.lock: development's side verbatim (no environment
  changes in this landing).

Everything else (custom_mg.py, nvb.py, _nvb_transform.pyx, surfaces.py,
enhanced_variables.py, setup.py extension registration, tests, design
docs) auto-merged.

Underworld development team with AI support from Claude Code
The style gate (Charter S4 except-pass rule, #364) flags a bare
'except Exception: pass' introduced on the stranded branch pre-Charter.
Minimal fix: document the sanctioned failure mode; no behaviour change.

Underworld development team with AI support from Claude Code
LAYER2_SBR_ADAPT_ON_TOP and NVB_GRADED_ADAPT arrive with the custom-mg
landing; wiring them into docs/developer/index.md keeps the docs build
at (below) the development warning baseline.

Underworld development team with AI support from Claude Code
Copilot AI review requested due to automatic review settings July 16, 2026 04:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Post-review hardening for the #365 landing. The branch renamed the
in-place MMG remesher adapt() -> remesh() and gave adapt() new
child-returning semantics, but a legacy caller got no signal: on a plain
mesh a bare adapt(metric) raised a confusing "needs refinement>=1"
RuntimeError, and on a refined mesh it silently returned an ignored
child, leaving the base unmodified.

- mesh.adapt(metric) with no new-API keyword now raises a TypeError
  directing the caller to mesh.remesh(metric) for in-place adaptation,
  or to pass an explicit keyword (max_levels=, engine=, ...) to opt in
  to the child-returning nested path. No silent redirection: the return
  semantics differ. All existing callers (tests, skills) already pass
  explicit keywords.
- tests/test_0834_adapt_legacy_call_guard.py (level_1, tier_a): the
  guard fires for bare and verbose-only calls, the message names both
  paths, and an explicit keyword opts in.
- Stale in-place docstring/doc surfaces updated to remesh:
  docs/advanced/mesh-adaptation.md, adaptivity.py docstrings, the
  _set_coords guidance message, _replace_from_adapted_mesh and
  Surface._on_mesh_adapted / refinement-metric docstrings.
- nvb.py: TODO(#360) comments on the two coordinate-row == vertex-order
  assumptions (serial-only reachable; holds for fresh createFromCellList
  / undistributed base DMs; np>1 uses the native transform).

Underworld development team with AI support from Claude Code
@lmoresi lmoresi merged commit 6ca0e60 into development Jul 16, 2026
2 checks passed
lmoresi added a commit that referenced this pull request Jul 16, 2026
…id design note (PROPOSED) (#373)

* refactor(custom-mg): D-79..D-87 readability rows + minimal-control multigrid design note

Wave D behaviour-neutral readability pass over utilities/custom_mg.py
(docs/reviews/2026-07 REMEDIATION-WORKLIST, D-custom_mg table; all nine
rows re-verified live at the post-#369 base):

- D-79 LevelLayout NamedTuple replaces the magic lay[3] tuple indexing
- D-80 _clone_dm_with_solver_discretisation extracted (copyDS trick
  documented once, on the helper)
- D-81 _assert_no_zero_columns_serial extracted beside the parallel guard
- D-82 getNumFields except narrowed to PETSc.Error + sanctioned comment
- D-83 parallel-path parameters renamed to the serial vocabulary
  (coarse_coords/fine_coords/coarse_layout/fine_layout)
- D-84 semicolon-packed lines split (6 sites)
- D-85 'sub' prefix-string collision renamed vel_prefix, moved into the
  verbose block
- D-86 RBF r=0 clamp commented (r^2 log r -> 0; clamp keeps log finite)
- D-87 TODO(deprecate) lifespan marker on the legacy finest-only path,
  stating the one behavioural difference (no BC-per-level reduction)

Plus docs/developer/design/MULTIGRID_MINIMAL_CONTROL_2026-07.md
(Status: PROPOSED): survey of the five existing MG paths, a proposed
minimal-control auto ladder, and seven numbered maintainer questions.
No implementation.

Bit-identical check: custom-MG suites identical pre/post edits
(serial 1015+1016+1017: 14 passed; np2 1017: 6 passed). Serial
'level_1 and tier_a' gate identical to the development baseline
(386 passed / 12 skipped / 2 xfailed / 1 xpassed). Scanner green;
docs build green.

Underworld development team with AI support from Claude Code

* docs(design): record the maintainer's seven multigrid rulings (2026-07-17); SNESFAS not-viable notes

The minimal-control design note moves PROPOSED -> RULED: GAMG stays the
automatic default (custom-P FMG opt-in); two-tier failure loudness;
wants_mg class attribute; solver.multigrid capability spelling;
set_custom_mg removed next cycle; no default hierarchies. SNESFAS is
ruled NOT viable at present (no good preconditioners; loses the robust
linear-solver path) - noted in the solver-strategies catalogue and the
plasticity-solvers skill as future investigation only.

Underworld development team with AI support from Claude Code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants