E2 finished 1h33min wall. Headline contrast on the matched Inferact
50-session subset:
E1 (naive 1P3D + kv-aware + RDMA):
1200/1285 succ, lat p50=93s p99=219s, TTFT p50=89s p99=207s
E2 (KVC v2 + RDMA):
231/1285 succ, lat p50= 7.4s p99=65s, TTFT p50=0.43s p99=8.7s
E2 is 12.4× worse on failure rate but 20× better on TTFT p50 among
the requests that did complete. Both runs leave D2 entirely unused
for the same structural reason: Inferact's shared "permissions
instructions" boilerplate makes overlap dominate the kv-aware lex
score, and v2's migration mechanism only fires on capacity rejects
which never reach D2. The 1054 E2 timeouts are downstream of that
imbalance, not a v2 bug per se.
The doc closes with five concrete follow-ups for the next agent —
cold-D bonus, router-mode admission, default-policy control arm,
TCP-loopback comparison, failure mode forensics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pathological imbalance E1 showed reproduces in E2: D2 has zero
bindings at 33% POSTs in. Root cause is structural, not a KVC v2 bug:
all 50 Inferact sessions begin with identical "permissions
instructions" boilerplate, so the converter assigns them identical
first-block hash_ids. kv-aware policy's overlap term (lex-score
position 0) makes any already-resident D dominate a fresh D
unconditionally, and v2's migration only activates on admission
rejects which never fire because D0/D1 KV pools have headroom. The
H1 conclusion is qualified: KVC v2 helps per-request work (direct-
to-D fast path) but does not rebalance D worker load on workloads
with shared cross-session prefixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
E1 finished 1h29min wall on the 50-session Inferact subset. Headline:
1200/1285 succeeded, latency p50=93s p99=219s, TTFT p50=89s p99=207s,
85 timeouts. Decode-2 was never bound to a single session — all 50
sessions stuck to decode-0/1 by kv-aware policy stickiness with no
migration to rebalance, so effective topology was 1P2D, not 1P3D.
This is exactly the failure mode H1 predicts naive pd-disaggregation
should exhibit, giving E2 (full KVC v2 with migration) a concrete
baseline to improve against.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
