docs(kvc): add TTFT probability density figure (KVC v2 vs 4DP)
Adds a two-panel TTFT PDF comparison plot inserted as a new V2_DEEP_ANALYSIS
§3.4 ("TTFT 概率密度对比: bimodal vs unimodal"). Single-percentile numbers
(p50 / p99) hide the qualitative difference between the two distributions;
the figure makes it visible at a glance.
Left panel (linear x in [0, 0.6]s, body):
KVC has a sharp peak at ~40ms (the direct-to-D fast path).
DP has a broad peak around 50-200ms (full prefill per request).
Annotated with p50 and p90 markers for each side.
Right panel (log x in [10ms, 10s], full range):
KVC is visibly bimodal: a tall fast-path peak plus a small reseed tail
around 1-5s.
DP is unimodal: a single broad peak with shorter tail.
Annotated with p99 callouts pointing to each tail.
KDE: scipy.stats.gaussian_kde, bandwidth=0.15 for the body (Scott's rule
oversmooths the sharp fast-path peak), log10-transformed for the full-range
panel so the bimodal structure is visible.
Bundled:
- scripts/analysis/plot_ttft_pdf.py -- rerunable when v2 / DP data change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -219,6 +219,26 @@ v2 整体跑得快不仅因为 "KVC 机制好",更因为 **91.6% 请求被路
|
||||
|
||||
如果工作量做归一化(比如限定都做 2000 token 以上 uncached prefill),KVC 应该和 DP 在同一速度量级。
|
||||
|
||||
### 3.4 TTFT 概率密度对比:bimodal vs unimodal
|
||||
|
||||
把 path-level 数据投影到 TTFT 的分布维度,可以更直观看出 KVC 与 DP 是**本质不同的两种分布形状**:
|
||||
|
||||

|
||||
|
||||
左图(线性 x ∈ [0, 0.6s])看 body:
|
||||
- **KVC 的 PDF 在 ~40ms 有一个尖锐峰值**(来自 91.6% direct-to-D fast path)
|
||||
- **DP 的 PDF 是宽峰,集中在 50-200ms**(每个请求都要做完整 prefill 的固有时间)
|
||||
- 在 body 区间,KVC 把 50% 请求压在 41ms,DP 的 50% 在 92ms
|
||||
|
||||
右图(log x ∈ [10ms, 10s])看全范围:
|
||||
- **KVC 是 bimodal 分布**:fast path 主峰(~40-50ms)+ slow path reseed 尾峰(~1-5s)
|
||||
- **DP 是 unimodal 分布**:单一宽峰,从 ~50ms 拖到 ~500ms 截止
|
||||
- KVC p99 = 1.28s 来自小尾峰;DP p99 = 0.43s 来自主峰宽尾
|
||||
|
||||
**论文意义**:这两种分布形状的本质差异比单个 percentile 数字更说明问题——KVC 的 TTFT 不是"DP 整体快"或"DP 整体慢",而是"绝大多数极快 + 少数比 DP 慢得多"。生产决策的判据应该是 **fast path 集中度 vs slow path tail 长度**的权衡,而不是单个 mean 或 p50 数字。
|
||||
|
||||
绘图脚本:`scripts/analysis/plot_ttft_pdf.py`(用 `scipy.stats.gaussian_kde`,body 用 Scott bandwidth 0.15,full range 用 log10 域 KDE)。
|
||||
|
||||
---
|
||||
|
||||
## 4. 需要诚实交代的 caveats(不是 KVC 的设计缺陷)
|
||||
|
||||
BIN
docs/figures/ttft_pdf_comparison.png
Normal file
BIN
docs/figures/ttft_pdf_comparison.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 315 KiB |
Reference in New Issue
Block a user