docs(kvc): add TTFT probability density figure (KVC v2 vs 4DP)

Adds a two-panel TTFT PDF comparison plot inserted as a new V2_DEEP_ANALYSIS
§3.4 ("TTFT 概率密度对比: bimodal vs unimodal"). Single-percentile numbers
(p50 / p99) hide the qualitative difference between the two distributions;
the figure makes it visible at a glance.

Left panel (linear x in [0, 0.6]s, body):
  KVC has a sharp peak at ~40ms (the direct-to-D fast path).
  DP has a broad peak around 50-200ms (full prefill per request).
  Annotated with p50 and p90 markers for each side.

Right panel (log x in [10ms, 10s], full range):
  KVC is visibly bimodal: a tall fast-path peak plus a small reseed tail
  around 1-5s.
  DP is unimodal: a single broad peak with shorter tail.
  Annotated with p99 callouts pointing to each tail.

KDE: scipy.stats.gaussian_kde, bandwidth=0.15 for the body (Scott's rule
oversmooths the sharp fast-path peak), log10-transformed for the full-range
panel so the bimodal structure is visible.

Bundled:
- scripts/analysis/plot_ttft_pdf.py -- rerunable when v2 / DP data change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
kzlin
2026-05-11 17:46:27 +08:00
parent b5af19583b
commit c5519066de
3 changed files with 219 additions and 0 deletions

View File

@@ -219,6 +219,26 @@ v2 整体跑得快不仅因为 "KVC 机制好",更因为 **91.6% 请求被路
如果工作量做归一化(比如限定都做 2000 token 以上 uncached prefillKVC 应该和 DP 在同一速度量级。
### 3.4 TTFT 概率密度对比bimodal vs unimodal
把 path-level 数据投影到 TTFT 的分布维度,可以更直观看出 KVC 与 DP 是**本质不同的两种分布形状**
![TTFT probability density: KVC v2 vs 4-way DP](figures/ttft_pdf_comparison.png)
左图(线性 x ∈ [0, 0.6s])看 body
- **KVC 的 PDF 在 ~40ms 有一个尖锐峰值**(来自 91.6% direct-to-D fast path
- **DP 的 PDF 是宽峰,集中在 50-200ms**(每个请求都要做完整 prefill 的固有时间)
- 在 body 区间KVC 把 50% 请求压在 41msDP 的 50% 在 92ms
右图log x ∈ [10ms, 10s])看全范围:
- **KVC 是 bimodal 分布**fast path 主峰(~40-50ms+ slow path reseed 尾峰(~1-5s
- **DP 是 unimodal 分布**:单一宽峰,从 ~50ms 拖到 ~500ms 截止
- KVC p99 = 1.28s 来自小尾峰DP p99 = 0.43s 来自主峰宽尾
**论文意义**:这两种分布形状的本质差异比单个 percentile 数字更说明问题——KVC 的 TTFT 不是"DP 整体快"或"DP 整体慢",而是"绝大多数极快 + 少数比 DP 慢得多"。生产决策的判据应该是 **fast path 集中度 vs slow path tail 长度**的权衡,而不是单个 mean 或 p50 数字。
绘图脚本:`scripts/analysis/plot_ttft_pdf.py`(用 `scipy.stats.gaussian_kde`body 用 Scott bandwidth 0.15full range 用 log10 域 KDE
---
## 4. 需要诚实交代的 caveats不是 KVC 的设计缺陷)

Binary file not shown.

After

Width:  |  Height:  |  Size: 315 KiB