Gahow Wang dae98c6472 Working-set sizing tool + GLM-5.1-FP8/B300 result ...

Configurable KV working-set analyzer (GPU model x TP/PP/EP x model
config.json with MLA/GQA auto x KV/weight dtype). Computes Denning W(T),
oracle [first,last], and retain-forever footprints vs a per-replica KV
pool, plus the APC captured at each retention window.

GLM-5.1-FP8 (MLA, 43.9 KiB/token) on 1x B300 node (1528 GB KV pool):
live KV fits trivially (~533 GB), but the full 80.4% APC ceiling needs
~14 nodes (oracle) -> long-tail reuse motivates DRAM offload, not HBM.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-28 16:03:25 +08:00

..

mb5

MB5 analysis: per-role KV split proves static-partition mismatch

2026-05-28 12:05:17 +08:00

v2

figs/v2: drop unified_v2 (buggy variant); re-render 4-policy panels

2026-05-27 13:55:10 +08:00

working_set

Working-set sizing tool + GLM-5.1-FP8/B300 result

2026-05-28 16:03:25 +08:00

f2a_reuse_topology.png

EAR outline: copy reusable figures, mark migration sections deferred

2026-05-27 01:44:13 +08:00

f2b_session_skew_prod.png

Add solo production-trace CDF figure (f2b_session_skew_prod.png)

2026-05-27 10:53:30 +08:00

f2b_session_skew.png

f2b: regenerate CDF from production trace (1.3M sessions on dash0)

2026-05-27 10:41:53 +08:00

f2c_kv_footprint_cdf.png

f2c: switch to per-instance decode-concurrency view; correct KV pool ceiling

2026-05-27 11:28:47 +08:00

f3a_inter_turn_gap.png

Add chatbot T_external CDF; overlay on f3a vs agentic

2026-05-27 14:49:44 +08:00

f4a_apc_loss.png

Drop 'capped' from per-policy figures (f4a, f4c×2, f6)

2026-05-27 10:57:43 +08:00

f4b_pdsep_kv_wall.pdf

EAR outline: copy reusable figures, mark migration sections deferred

2026-05-27 01:44:13 +08:00

f4b_pdsep_kv_wall.png

Use PNG for KV memory wall figure; switch outline to inline image embeds

2026-05-27 09:13:26 +08:00

f4c_per_worker_ttft.png

Replace max/median hotspot index with (median, max) absolute pair

2026-05-27 11:07:12 +08:00

f4d_pd_interference.png

EAR outline: copy reusable figures, mark migration sections deferred

2026-05-27 01:44:13 +08:00

f6_e2e_latency_bars.png

Drop 'capped' from per-policy figures (f4a, f4c×2, f6)

2026-05-27 10:57:43 +08:00

f6_e2e_latency_full_grid.png

Add full latency grid (mean/p50/p90/p99 × TTFT/TPOT/E2E) as f6 companion

2026-05-27 11:15:18 +08:00

mb1_interference.png

Correct PD-disagg cost/benefit framing across repo

2026-05-27 22:04:49 +08:00

mb2_transfer_bw_compare.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_bw_inter.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_bw_intra.png

MB2: pure KV-transfer cost on dash1 intra-node — Mooncake ~9.7 GB/s steady

2026-05-27 19:04:03 +08:00

mb2_transfer_time_compare.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_time_inter.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_time_intra.png

MB2: pure KV-transfer cost on dash1 intra-node — Mooncake ~9.7 GB/s steady

2026-05-27 19:04:03 +08:00