f2c: switch to per-instance decode-concurrency view; correct KV pool ceiling

Old f2c plotted per-request KV footprint MiB against an "H20 ~95 GiB
usable" reference line. That ceiling was wrong — a 30B-A3B bf16
deployment burns roughly:
  ~50% HBM for model params (~48 GiB on 96 GiB H20)
  ~10% for runtime activation buffers
  ~40% left for the KV cache pool (~38.4 GiB)
so 95 GiB was overstating the available pool by 2.5×.

New f2c reframes the same data into the answer that actually motivates
the paper: how many concurrent decodes does a single instance hold,
and how does PD-disagg change that? Grouped bars per percentile show
system-wide concurrent decode capacity for three 8-GPU deployments:
  Combined 8C, PD-disagg 4P+4D (N_D=4), PD-disagg 6P+2D (N_D=2)

Key reads off the figure:
  p50 (1.8 GiB/req): 20 fit/inst → 160 / 80 / 40 system-wide
  p90 (8.0 GiB/req):  4 fit/inst →  32 / 16 /  8
  p95 (9.6 GiB/req):  4 fit/inst →  32 / 16 /  8
  p99 (11.5 GiB/req): 3 fit/inst →  24 / 12 /  6

PD-disagg 4P+4D literally halves the decode population at the same
per-request KV pressure — this is the concrete §3.2 "KV memory wall"
penalty stated in terms users care about (concurrency).

- analysis/characterization/render_window1_figures.py:
  fig_kv_footprint_cdf rewritten; reads same kv_footprint_summary.json
  but computes floor(KV_pool / req_size) × N_D and annotates the
  per-instance fit count below each percentile group.
- figs/f2c_kv_footprint_cdf.png: regenerated.
- MEETING.md / PAPER_OUTLINE.md §2.1, §2.4: prose updated with the
  new ceiling and the "3 p99 decodes per instance / halved by PD-disagg"
  framing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-27 11:28:47 +08:00
parent 922d79ac95
commit 555cabcf1f
4 changed files with 62 additions and 14 deletions

View File

@@ -50,7 +50,7 @@ Agentic workload 与 chatbot 的三个本质差异:
- **Prefill-dominated**input/output token ratio **75x**98% 计算在 prefill 阶段chatbot 为 1-10x
- **Skewed sessions**(来自 Qwen3 production tracen=1.3M session / 2.1M req / 7200stop 1% 贡献 **46.5%** input tokentop 5% **66.5%**top 10% **74.6%**top 25% **87.5%**top 50% **96.0%** —— 半数 session 几乎占满全部 input mass
平均 session 长度 TBD turn、TBD 输入 tokenp99 单请求 KV 占用 **11.49 GiB**H20 96GB HBM 的 12%)。
平均 session 长度 TBD turn、TBD 输入 token。Per-request KV footprintQwen3-Coder-30B-A3B, 98304 B/tokenp50 **1.8 GiB**, p90 **8.0 GiB**, p95 **9.6 GiB**, p99 **11.5 GiB**. 单 instance KV pool ≈ 0.4 × 96 GiB = **38.4 GiB**(剩 50% model params bf16 + 10% runtime activation所以 p99 请求一个 instance 只能装 **3 个 concurrent decode**;改 PD-disagg 4P+4D 让系统 decode 容量直接减半(系统并发 24 → 12
### §2.2 KV Cache Reuse Topology
@@ -70,7 +70,7 @@ Trace 上 KV reuse 的分解:
![F2b Session input-token mass CDF — production trace top 1%/5%/10%/25%/50% = 46.5%/66.5%/74.6%/87.5%/96.0% (replay window overlaid for sanity)](figs/f2b_session_skew.png)
![F2c KV footprint CDF — p99 = 11.8 GiB ≈ 12% of H20](figs/f2c_kv_footprint_cdf.png)
![F2c Per-instance decode concurrency vs deployment (KV pool 38.4 GiB; p99 req fits only 3/inst; PD-disagg halves system decode capacity)](figs/f2c_kv_footprint_cdf.png)
> 📝 Layout TBD三张拼成 1×3 还是分散到 §2.1/§2.2/§2.4 各一张。