Harden Newton runtime contracts and differentiable stepping

[yuecideng]

Description

This stacked draft PR updates the Newton backend work in #276 with the current 38-commit branch state: 31 upstream main synchronization commits and 7 Newton-focused commits.

The Newton-focused changes:

• define the generation, ownership, rebuild, and differentiable-trajectory contracts;
• make solver dynamics the default differentiable environment route through a manager-owned Newton trajectory and trajectory-local control;
• retain an explicit pure-kinematics step mode;
• preserve the Warp tape and Newton trajectory through PyTorch backward, including non-zero gradient, central finite-difference, and continuous multi-step coverage;
• defer terminal resets until after backward on grad-tracked steps while retaining synchronous no-grad auto-reset;
• isolate Franka kinematics primals from live Newton state mutations.

Motivation: the previous bridge could bypass real solver dynamics, publish or mutate live state while a tape was active, and lose the trajectory required by backward. The new contract gives DexSim ownership of detached states, contacts, control, forces, publication, and release.

Dependencies: DexSim branch feature/embodichain-newton-contracts, including 6cc24c223 and 7ee975be5. This PR should remain draft until that dependency is published or integrated.

Related: #276

Type of change

• Bug fix (non-breaking change which fixes an issue)
• Enhancement (non-breaking change which improves an existing functionality)
• New feature (non-breaking change which adds functionality)
• Breaking change (existing functionality will not work without user modification)
• Documentation update

Screenshots

Not applicable.

Verification

• black .: 953 files unchanged
• EmbodiChain differentiable environment contract: 39 passed, 2 asset/GPU Franka smoke tests deselected
• DexSim manager-owned trajectory: 10 passed
• DexSim real solver gradient, central finite difference, and continuous multi-step: 3 passed
• DexSim multi-world runtime: 11 passed
• Independent final review: APPROVE, no P0/P1

The known broad same-process native teardown hang/segfault is baseline-reproducible and is not included in the passing test claim.

Checklist

• I have run the black . command to format the code base.
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• Dependencies have been updated, if applicable.