Two cleanups:
1. Drop "E1: naive 1P3D default" experiment from the onboarding manual.
GPU hours are precious; naive 1P3D + policy=default has near-certain
loss on multi-turn cache hit (it's round-robin without prefix awareness),
so the comparison doesn't add information vs E1=naive 1P3D kv-aware.
The new manifest has only 2 runs: E1 (naive 1P3D kv-aware) + E2 (KVC
v2 + RDMA). Run-time budget drops from 16.5h serial to 11h serial /
5.5h parallel. Updated:
- §0 TL;DR ("3 组" -> "2 组")
- §2 H1 hypothesis (drop "default and kv-aware each one" -> just kv-aware)
- §3.1 experiment matrix (3 rows -> 2 rows + rationale for the drop)
- §3.2 startup config (drop E1 default section, renumber E2/E3 -> E1/E2)
- §6 decision table + expected-range table
- §7 FAQ ("3 个 E1-E3" -> "2 个 E1-E2")
- §9 deliverables
2. Move 8 deprecated docs to docs/archive/:
AGENTIC_FIT_ANALYSIS_ZH.md (ts=10 era analysis; superseded)
STRUCTURAL_VALIDATION_REPORT_ZH.md (ts=10 era validation; superseded)
KVC_DEBUG_JOURNEY_V1_TO_V5.md (v1-v5 sweep process notes)
V5_PROFILE_INVESTIGATION_ZH.md (v5 1Hz polling investigation)
REFACTOR_PLAN_ZH.md (v0 plan; superseded by V1)
KVCACHE_CENTRIC_PROGRESS_ZH.md (earliest 2026-04-27 progress)
SWEBENCH_EXPERIMENT_PROGRESS.md (early SWE trace setup)
SWEBENCH_EXPERIMENT_RESULTS.md (early SWE result snapshot)
All cross-references in active docs (V2_DEEP_ANALYSIS / V2_RESULTS /
REFACTOR_PLAN_V1 / TEAM_REPORT / ONBOARDING) rewritten from
`docs/FOO.md` to `docs/archive/FOO.md` via sed pass.
Added `docs/archive/README.md` explaining what each archived doc is
and when (if ever) to reopen it. Designed so a new reader hitting
the archive dir immediately knows it's not required reading.
After this commit the active docs in docs/ are 9 files (down from 17),
which should make the onboarding doc's "Level 1 / Level 2 / Level 3"
classification self-evident.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After an independent Opus-agent forensic audit, the previous "(c) 增量
fetch (工程量较大，未实现)" line in V2_DEEP_ANALYSIS §4.2 was understating
the gap. The audit confirmed:
- No D->P KV transfer code exists in the framework at any layer
(agentic_pd_hybrid orchestration, vendored SGLang disaggregation,
or mooncake transport).
- Mooncake MooncakeKVManager has a hard role split: PREFILL = sender,
DECODE = receiver-only loop. `add_transfer_request` asserts the
disaggregation_mode is PREFILL.
- The BaseKVSender / BaseKVReceiver abstraction has no bidirectional slot.
- session_aware_cache.release_session only calls kv_pool_allocator.free()
on eviction -- no serialization, no outbound network call.
- _commit_prefill_backup_residency is only called from the seed/reseed
path (_invoke_kvcache_seeded_router). direct-to-D path never updates
P-side backup state.
- "capacity-backup" policy semantics: it only skips the close on P after
reseed -- the backup is the seed-time static snapshot, never refreshed
by D-side append-prefill activity.
V2_DEEP_ANALYSIS §4.2:
- Decomposed the 3-7s reseed cost into the P-side re-prefill segment
(1.5-3s, dominant) and the P->D mooncake transfer segment (1.5-4s).
- Quantified the realistic effect of enabling RDMA: only the transfer
segment shrinks, reseed reduces to 1.7-3.2s, TTFT p99 ~0.7s, still
loses to DP's 0.43s.
- Replaced the throwaway "(c) incremental fetch" line with a full
paragraph explaining what D->P sync would require, why it's the
largest engineering gap, and that the blocker is SGLang's radix-tree
single-producer assumption, not the network layer.
KVC_ROUTER_ALGORITHM §9:
- Refined Open Question 3 (RDMA) to clarify it only helps the transfer
segment, not the re-prefill segment.
- Added Open Question 4: D->P incremental KV sync as the central
future-work contribution gap, with cited evidence for why it doesn't
currently exist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two figures inserted into V2_DEEP_ANALYSIS §4.5 and §4.4 respectively, to
visually rebut the two critic-agent claims that we argued in prose were
design intent, not deficiencies.
(1) gpu_utilization.png -- §4.5 "P GPU is wasted 90% of the time"
Two-panel side-by-side:
Left (request count view, the naive reading): KVC P = 328 reqs (7.4%),
KVC D = ~1450 each, DP = ~1100 each. P "looks idle."
Right (compute work view, the honest reading): KVC P does 1.07M tokens
of prefill, comparable to each KVC D worker's ~0.80M. P is a
low-frequency high-cost safety net, not idle capacity.
Bonus finding: KVC's total compute (3.47M tokens across 4 GPUs) is 33%
LESS than DP's (5.17M). Same GPUs, less work done. That's the affinity
win.
(2) cache_efficiency.png -- §4.4 "Cache concentration is not policy win"
Two-panel side-by-side. The setup: KVC has 27% LESS total KV pool
(276K vs 351K tokens) yet caches MORE per request.
Left (cache hit rate vs turn number): KVC's session-affinity lets
hit rate accumulate with turns; DP's hash + radix-LRU causes
a mid-turn drift around turns 8-25 where KVC = 97.0% vs DP
= 95.8% (1.24pp gap). Shows mechanism, not just outcome.
Right (ECDF of per-request uncached tokens, log x): KVC's distribution
concentrates near zero (50% < 187 tokens), DP's is spread
(50% < 781 tokens). At uncached = 500 tokens threshold, KVC
has 74% of requests below, DP has 31%.
→ smaller pool, better retention, less per-request work. Direct empirical
rebuttal to "fragmentation is architectural, not policy."
Bundled scripts (rerunable):
- scripts/analysis/plot_gpu_utilization.py
- scripts/analysis/plot_cache_efficiency.py
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a two-panel TTFT PDF comparison plot inserted as a new V2_DEEP_ANALYSIS
§3.4 ("TTFT 概率密度对比: bimodal vs unimodal"). Single-percentile numbers
(p50 / p99) hide the qualitative difference between the two distributions;
the figure makes it visible at a glance.
Left panel (linear x in [0, 0.6]s, body):
KVC has a sharp peak at ~40ms (the direct-to-D fast path).
DP has a broad peak around 50-200ms (full prefill per request).
Annotated with p50 and p90 markers for each side.
Right panel (log x in [10ms, 10s], full range):
KVC is visibly bimodal: a tall fast-path peak plus a small reseed tail
around 1-5s.
DP is unimodal: a single broad peak with shorter tail.
Annotated with p99 callouts pointing to each tail.
KDE: scipy.stats.gaussian_kde, bandwidth=0.15 for the body (Scott's rule
oversmooths the sharp fast-path peak), log10-transformed for the full-range
panel so the bimodal structure is visible.
Bundled:
- scripts/analysis/plot_ttft_pdf.py -- rerunable when v2 / DP data change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V2_DEEP_ANALYSIS §3.1 (execution_mode distribution) and §3.2 (path-level
latency vs DP) had hand-typed tables with approximate latencies (e.g.
"~1.0s") and required readers to mentally compare 5+ rows × 5 columns.
Both sections now reference generated PNG figures derived directly from
the v2 + DP metrics.jsonl files.
§3.1 figure (v2_execution_mode_distribution.png):
Horizontal bar chart, log x-axis. 4076 direct-to-D fast-path requests
(green) dwarf the rest by ~30x; the long tail of slow / fallback /
failure modes is visible at one glance. Counts and percentages
annotated on each bar.
§3.2 figure (v2_path_level_latency.png):
Grouped bar chart, log y-axis. Per-path TTFT p50 / TTFT p99 / Lat p50
with exact numeric labels (no more "~1.0s" approximations). Sample
counts annotated below each path. Quick visual reads:
- KVC fast path TTFT p50 41ms vs DP 92ms (2.2x faster)
- KVC reseed TTFT p99 5.12s vs DP 0.43s (12x slower) -- the cost
- KVC no-d-capacity TTFT p99 7.65s (worst case)
Bundled:
- scripts/analysis/plot_v2_path_breakdown.py -- the script that
generates both figures; rerunable when v2 data changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the critic-agent audit, V2_DEEP_ANALYSIS had drifted into an
audit-grade "5 wins / 1 loss / 3 draws" framing that mistook KVC's
deliberate design motifs (cache concentration via session affinity;
prefill-GPU idle as TTFT-stability trade-off) for "comparison
unfairness." This commit corrects the framing back to a production-
decision lens and adds a paper-track formal specification of the
router algorithm.
V2_DEEP_ANALYSIS_ZH.md changes:
- §0 TL;DR: lead with "online coding agent serving should pick
KVC 1P3D"; the only real cost is TTFT p99 long-tail (3x DP) from
the 8.3% mooncake reseed path, mitigable with real RDMA.
- §4 restructured into three buckets:
real costs (TTFT p99 tail, abort accounting now fixed),
counter-arguments to the critic (cache concentration and idle
prefill GPU are design intent, not deficits),
methodology to-do (naive-1P3D control, v2 N>=2 determinism).
- §6 replaces "5/1/3 rescoring" with production decision rationale:
KVC wins on 6 latency/TTFT metrics + lower failure rate; pays
TTFT p99 tail; lists workloads where DP would reverse the call.
- §8 decision points: D1 recommends Yes (accept v2 as milestone);
D8 added: paper motif "KVC trades P idle for TTFT stability."
KVC_ROUTER_ALGORITHM.md (new, paper-track, Chinese narrative + English
algorithm boxes / variable names / theorems for direct paper reuse):
- Problem formulation, system model, full notation
- Algorithm 1 Route: lexicographic-tuple scoring on
(overlap+alpha*sticky, sticky, -inflight, -assigned)
- Algorithm 2 Admit: D-worker autonomous admission deciding
Direct / Seed / Reseed / reject (with reason)
- Algorithm 3 Dispatch: end-to-end orchestration with reset-on-success
(the v2-specific fix that eliminates v1's self-amplifying thrashing)
- Theorem 1 (no permanent starvation) and Theorem 2 (fast-path
determinism), each with a proof sketch
- Comparison table vs vanilla pd-disagg / DP cache-aware
- Anti-patterns ("what KVC explicitly is NOT")
- Open questions for reviewers
- Suggested paper citation phrasing
- Appendix A: algorithm-step to source-file:line crosswalk
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Post-v2 audit consolidating ts=1 validation + v1 thrashing + v2 win, plus
critic-agent adversarial review of the v2 vs 4DP comparison.
Headline outcomes:
- TEAM_REPORT §1 (session pin starvation) fully fixed by v2 migration +
reset-on-success; direct-to-D 42.8% -> 91.6%.
- TEAM_REPORT §2/§3/§5 (LRU, backpressure, admission RPC) are absorbed by
ts=1 natural drain time, not mechanism-fixed -- will resurface under
ts=10/longer traces/higher concurrency.
- TEAM_REPORT §6 (ts=10 distortion) confirmed and locked as precondition;
TEAM_REPORT §8 (N=1 unreliable) rewritten to "high-pressure N>=3, normal N=1".
Three new problems exposed by adversarial review:
- TTFT p99: KVC 1.285s vs DP 0.427s (KVC 3.0x worse) -- cherry-picked out of
the V2_RESULTS_ZH.md headline table. Root cause: 8.3% non-direct path pays
3-7s mooncake reseed cost on 50-90K-token KV transfer.
- Error accounting asymmetry: DP has 67 fast-aborts (not 5) at ~0.08s each
counted in latency stats; KVC's 5 ReadTimeouts excluded entirely. Root
cause: --max-input-len 87811 (DP) vs 92098 (KVC) + metrics.py:124 filter.
- Topology mismatch: KVC 1P3D's prefill GPU is idle 91.7% of the time
(only ~373/4449 requests use seed/P path); 4DP CA has all 4 GPUs at full
utilization. Plus: no naive 1P3D control exists in the repo -- cannot
isolate KVC-layer contribution from 1P3D-topology contribution.
Re-scored headline: 5 KVC wins / 1 DP win / 3 draws -- still net positive
but not the "7/8 wins" framing the V2_RESULTS_ZH.md claims.
Recommended follow-ups (ROI order):
1. naive 1P3D ts=1 N=1 control (critic's only CRITICAL finding)
2. v2 N=2/N=3 to verify ts=1 determinism with new code paths
3. symmetric error accounting recompute + DP max-input-len = 92098 rerun
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
