Pulling admission-events.jsonl, prefill-0.log, and request-metrics
sampling shows the 1054 failures are NOT timeouts as initially
assumed. They are a 3-layer cascade:
L1: 562 "no-space" + 43 "session-not-resident" worker admission
rejects (51% of all admit attempts) because D0/D1 KV pools
saturate while D2 stays empty.
L2: rejects re-route to seed/reseed which need mooncake P→D KV
transfer; the backlog drops mooncake heartbeats and prefill-0
logs "Decode instance could be dead, remote mooncake session
... is not alive".
L3: SGLang aborts the request, SSE stream closes with 0 tokens,
agentic-pd-hybrid raises "generate stream ended before
producing any token" (the literal error string for all 1054).
E1 didn't hit this because pd-disaggregation has no admission RPC —
sessions just queue behind the running batch, paying TTFT instead
of failing. KVC v2's worker admission is supposed to be a safety
valve; on the cold-D pathology it becomes a failure amplifier.
The real fix is upstream D rebalancing (cold-D bonus or pre-warm),
not relaxing admission.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
E2 finished 1h33min wall. Headline contrast on the matched Inferact
50-session subset:
E1 (naive 1P3D + kv-aware + RDMA):
1200/1285 succ, lat p50=93s p99=219s, TTFT p50=89s p99=207s
E2 (KVC v2 + RDMA):
231/1285 succ, lat p50= 7.4s p99=65s, TTFT p50=0.43s p99=8.7s
E2 is 12.4× worse on failure rate but 20× better on TTFT p50 among
the requests that did complete. Both runs leave D2 entirely unused
for the same structural reason: Inferact's shared "permissions
instructions" boilerplate makes overlap dominate the kv-aware lex
score, and v2's migration mechanism only fires on capacity rejects
which never reach D2. The 1054 E2 timeouts are downstream of that
imbalance, not a v2 bug per se.
The doc closes with five concrete follow-ups for the next agent —
cold-D bonus, router-mode admission, default-policy control arm,
TCP-loopback comparison, failure mode forensics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pathological imbalance E1 showed reproduces in E2: D2 has zero
bindings at 33% POSTs in. Root cause is structural, not a KVC v2 bug:
all 50 Inferact sessions begin with identical "permissions
instructions" boilerplate, so the converter assigns them identical
first-block hash_ids. kv-aware policy's overlap term (lex-score
position 0) makes any already-resident D dominate a fresh D
unconditionally, and v2's migration only activates on admission
rejects which never fire because D0/D1 KV pools have headroom. The
H1 conclusion is qualified: KVC v2 helps per-request work (direct-
to-D fast path) but does not rebalance D worker load on workloads
with shared cross-session prefixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
E1 finished 1h29min wall on the 50-session Inferact subset. Headline:
1200/1285 succeeded, latency p50=93s p99=219s, TTFT p50=89s p99=207s,
85 timeouts. Decode-2 was never bound to a single session — all 50
sessions stuck to decode-0/1 by kv-aware policy stickiness with no
migration to rebalance, so effective topology was 1P2D, not 1P3D.
This is exactly the failure mode H1 predicts naive pd-disaggregation
should exhibit, giving E2 (full KVC v2 with migration) a concrete
baseline to improve against.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
