V2_DEEP_ANALYSIS §3.1 (execution_mode distribution) and §3.2 (path-level
latency vs DP) had hand-typed tables with approximate latencies (e.g.
"~1.0s") and required readers to mentally compare 5+ rows × 5 columns.
Both sections now reference generated PNG figures derived directly from
the v2 + DP metrics.jsonl files.
§3.1 figure (v2_execution_mode_distribution.png):
Horizontal bar chart, log x-axis. 4076 direct-to-D fast-path requests
(green) dwarf the rest by ~30x; the long tail of slow / fallback /
failure modes is visible at one glance. Counts and percentages
annotated on each bar.
§3.2 figure (v2_path_level_latency.png):
Grouped bar chart, log y-axis. Per-path TTFT p50 / TTFT p99 / Lat p50
with exact numeric labels (no more "~1.0s" approximations). Sample
counts annotated below each path. Quick visual reads:
- KVC fast path TTFT p50 41ms vs DP 92ms (2.2x faster)
- KVC reseed TTFT p99 5.12s vs DP 0.43s (12x slower) -- the cost
- KVC no-d-capacity TTFT p99 7.65s (worst case)
Bundled:
- scripts/analysis/plot_v2_path_breakdown.py -- the script that
generates both figures; rerunable when v2 data changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the critic-agent audit, V2_DEEP_ANALYSIS had drifted into an
audit-grade "5 wins / 1 loss / 3 draws" framing that mistook KVC's
deliberate design motifs (cache concentration via session affinity;
prefill-GPU idle as TTFT-stability trade-off) for "comparison
unfairness." This commit corrects the framing back to a production-
decision lens and adds a paper-track formal specification of the
router algorithm.
V2_DEEP_ANALYSIS_ZH.md changes:
- §0 TL;DR: lead with "online coding agent serving should pick
KVC 1P3D"; the only real cost is TTFT p99 long-tail (3x DP) from
the 8.3% mooncake reseed path, mitigable with real RDMA.
- §4 restructured into three buckets:
real costs (TTFT p99 tail, abort accounting now fixed),
counter-arguments to the critic (cache concentration and idle
prefill GPU are design intent, not deficits),
methodology to-do (naive-1P3D control, v2 N>=2 determinism).
- §6 replaces "5/1/3 rescoring" with production decision rationale:
KVC wins on 6 latency/TTFT metrics + lower failure rate; pays
TTFT p99 tail; lists workloads where DP would reverse the call.
- §8 decision points: D1 recommends Yes (accept v2 as milestone);
D8 added: paper motif "KVC trades P idle for TTFT stability."
KVC_ROUTER_ALGORITHM.md (new, paper-track, Chinese narrative + English
algorithm boxes / variable names / theorems for direct paper reuse):
- Problem formulation, system model, full notation
- Algorithm 1 Route: lexicographic-tuple scoring on
(overlap+alpha*sticky, sticky, -inflight, -assigned)
- Algorithm 2 Admit: D-worker autonomous admission deciding
Direct / Seed / Reseed / reject (with reason)
- Algorithm 3 Dispatch: end-to-end orchestration with reset-on-success
(the v2-specific fix that eliminates v1's self-amplifying thrashing)
- Theorem 1 (no permanent starvation) and Theorem 2 (fast-path
determinism), each with a proof sketch
- Comparison table vs vanilla pd-disagg / DP cache-aware
- Anti-patterns ("what KVC explicitly is NOT")
- Open questions for reviewers
- Suggested paper citation phrasing
- Appendix A: algorithm-step to source-file:line crosswalk
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Post-v2 audit consolidating ts=1 validation + v1 thrashing + v2 win, plus
critic-agent adversarial review of the v2 vs 4DP comparison.
Headline outcomes:
- TEAM_REPORT §1 (session pin starvation) fully fixed by v2 migration +
reset-on-success; direct-to-D 42.8% -> 91.6%.
- TEAM_REPORT §2/§3/§5 (LRU, backpressure, admission RPC) are absorbed by
ts=1 natural drain time, not mechanism-fixed -- will resurface under
ts=10/longer traces/higher concurrency.
- TEAM_REPORT §6 (ts=10 distortion) confirmed and locked as precondition;
TEAM_REPORT §8 (N=1 unreliable) rewritten to "high-pressure N>=3, normal N=1".
Three new problems exposed by adversarial review:
- TTFT p99: KVC 1.285s vs DP 0.427s (KVC 3.0x worse) -- cherry-picked out of
the V2_RESULTS_ZH.md headline table. Root cause: 8.3% non-direct path pays
3-7s mooncake reseed cost on 50-90K-token KV transfer.
- Error accounting asymmetry: DP has 67 fast-aborts (not 5) at ~0.08s each
counted in latency stats; KVC's 5 ReadTimeouts excluded entirely. Root
cause: --max-input-len 87811 (DP) vs 92098 (KVC) + metrics.py:124 filter.
- Topology mismatch: KVC 1P3D's prefill GPU is idle 91.7% of the time
(only ~373/4449 requests use seed/P path); 4DP CA has all 4 GPUs at full
utilization. Plus: no naive 1P3D control exists in the repo -- cannot
isolate KVC-layer contribution from 1P3D-topology contribution.
Re-scored headline: 5 KVC wins / 1 DP win / 3 draws -- still net positive
but not the "7/8 wins" framing the V2_RESULTS_ZH.md claims.
Recommended follow-ups (ROI order):
1. naive 1P3D ts=1 N=1 control (critic's only CRITICAL finding)
2. v2 N=2/N=3 to verify ts=1 determinism with new code paths
3. symmetric error accounting recompute + DP max-input-len = 92098 rerun
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
