~45 ms added latency on sandbox tunnels/egress tinygrams due to Nagle × delayed-ACK issue

[purp]

Agent Diagnostic

Investigated with a coding agent pointed at the repo, plus a purpose-built
component-isolation benchmark ladder (tools and methodology temporarily available on this
relay-latency-benchmarking branch):

• Measured the marginal cost of a small HTTPS request/response over an
  established keep-alive connection, from inside a sandbox to an echo
  server on the host: ~44 ms, identical in L4 passthrough and L7
  terminate-and-inspect modes.
• Controls: the same exchange from a plain docker run container (no
  OpenShell egress proxy) is 15–20 ms; host loopback floor is 0.06 ms.
• time_starttransfer is only 2–3 ms — the stall is on the tail of the
  small response, after the first byte.
• The penalty inverts with payload size: a 1 KB response takes ~63 ms
  end-to-end while a 100 KB response takes ~15 ms. A fixed ~40–45 ms
  quantum on small writes that disappears with streaming volume is the
  classic Nagle × delayed-ACK interaction.
• Verified by grep that nothing in the repo calls set_nodelay /
  TCP_NODELAY on any socket — neither the egress proxy's accepted
  sandbox-side sockets nor its upstream TcpStream::connect sockets
  (crates/openshell-supervisor-network/src/proxy.rs), nor the tunnel-path
  sockets (gateway public listener, CLI edge tunnel / service-forward,
  supervisor direct-tcpip and TCP-relay connects).
• The agent built and locally validated a candidate fix (TCP_NODELAY on
  all relay-path sockets); A/B ladder numbers are in Logs below.

Description

Every small request/response (aka "tinygram") an agent sends through the
sandbox egress path — API calls, git smart-protocol negotiation, MCP
messages — carries a fixed ~44 ms penalty due to the
well understood conflict between the Nagle Algorithm and Delayed ACK configuration which is generally solved by setting TCP_NODELAY on one or both relay sockets.

Expected: latency close to the plain-Docker control
(15–20 ms), which is dominated by TLS connection setup rather than
per-request cost. The same missing-TCP_NODELAY pattern applies to the
sandbox tunnel hops (gateway public listener, CLI edge tunnel and
service-forward, supervisor direct-tcpip/TCP-relay connects), so tunneled
traffic is affected as well, not just egress.

Because the penalty is transport-level it affects L4 and L7 modes equally,
and it dwarfs actual inspection cost: the entire L7 stack (TLS
termination, header parse, policy eval, OCSF emit, credential rewrite)
measures on the close order of 200 µs per request (effectively 0) against
L4 passthrough — post-fix keep-alive marginals are 0.45 ms (L7) vs 0.27 ms
(L4), and single-shot 1 KB medians differ by ~20 µs. With the penalty
removed, governed small exchanges sit within ~0.2–0.4 ms of the host
loopback floor.

Setting TCP_NODELAY on both relay sockets is standard practice for
proxies (hyper/tonic already do it in their own connectors); the relay's
extra store-and-forward hop changes small-packet ACK timing enough to
trigger the stall.

Reproduction Steps

Full harness with one-command ladder and analysis:
experiments/latency-benchmarking
(./run-ladder.sh 30, then python3 analyze.py results-v2/<timestamp>).

Minimal manual version:

1. Start a local gateway: mise run gateway:docker (main @ 420a855).

2. Start a TLS server on the host (port 8443) that can return a small
   (~1 KB) body.

3. Create a sandbox whose policy allows egress to
   host.docker.internal:8443.

4. Inside the sandbox, issue N requests over ONE keep-alive connection and
   read the marginal per-request time from curl's per-URL timings:

   openshell sandbox exec -n bench -- bash -c '
     urls=$(printf "https://host.docker.internal:8443/ok %.0s" $(seq 100))
     curl -sS -o /dev/null -w "%{time_total}\n" $urls'

   Requests 2..N each take ~44 ms through the sandbox. The same loop
   from a plain docker run debian container: well under 1 ms per request.

5. Optional: repeat with a 100 KB response — it completes faster
   end-to-end than the 1 KB one (~15 ms vs ~63 ms), the Nagle signature.

Environment

• OS: macOS 26.4.1, Apple Silicon (arm64)
• Docker: Docker Engine 29.5.3 (linux/arm64, Docker Desktop)
• OpenShell: built from source at main @ 420a855, standalone gateway via
  mise run gateway:docker, Docker compute driver
• Sandbox: debian-slim + curl

Logs

Benchmark results, same machine/day. All numbers are medians over 30
repetitions per cell (IQRs in the analyzer output on the
benchmark branch);
keep-alive marginal = median per-request total for requests after the
first on a single connection, i.e. with connection setup amortized away.
Baseline = main @ 420a855; fix = baseline + TCP_NODELAY on all
relay-path sockets.

Keep-alive marginal cost per small request (the bug):

Path	baseline	with fix	host floor
L4 passthrough, 100-req conn	43.58 ms	0.27 ms	0.06 ms
L7 terminate+inspect, 100-req conn	44.49 ms	0.45 ms	0.06 ms

Single-shot small exchange (TLS connect + 1 request):

Cell	baseline	with fix
1 KB response	62.64 ms	14.66 ms
1 KB request	61.07 ms	14.64 ms

No-regression checks (worst cases):

Cell	baseline	with fix
10 MB response	58.57 ms	58.42 ms
10 MB request	90.41 ms	46.09 ms
L7 body-rewrite rung, 1 KB	63.7 ms	14.7 ms

(10 MB uploads turned out to be Nagle-throttled too — the fix halves them.)

Agent-First Checklist

• I pointed my agent at the repo and had it investigate this issue
• I loaded relevant skills
• My agent could not resolve this — the diagnostic above explains why

The agent did resolve this locally; @purp wanted to expose this as an
issue for possible discussion ahead of a PR.