Gahow Wang cf812b6264 Workload characterization C1-C3 on full production trace ...

Joint/temporal characterizations of the full 051315 cluster trace (2.11M
req / 1.31M sessions / 2h), beyond the existing single-variable marginals:

- C1 mixture: 90.3% sessions single-turn, but multi-turn (9.7%) = 44% reqs /
  67% prefill mass; continuation hazard rises 10%->94% (Lindy); heaviness
  unpredictable at turn 1 (corr 0.04-0.15) => reactive routing justified.
- C2 resident/delta: resident context 11k->56k while new-prefill 2.7k->~200;
  per-turn reuse ->99.6%; resident/delta ("PD tax") ->~250-450x.
- C3 prefill/decode: token mass 98.7% input / 1.3% output, BUT decode ~70% of
  TIME (robust 68-71%); "decode negligible" is wrong (tokens != time). Correct
  colo argument = roofline complementarity, not "no decode".

Maps each to (1) PD-colocation and (2) routing. compute_chars.py + chars.json
+ figs/workload_chars/. Raw-file exact validation (cached_tokens, real
timings) pending.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-29 18:19:39 +08:00

..

mb5

PD_DISAGG_RESULTS §6.3: producer hot-pinning figure

2026-05-29 00:38:20 +08:00

v2

figs/v2: drop unified_v2 (buggy variant); re-render 4-policy panels

2026-05-27 13:55:10 +08:00

working_set

Working-set figure: extend left panel to ~50 nodes

2026-05-28 17:11:12 +08:00

workload_chars

Workload characterization C1-C3 on full production trace

2026-05-29 18:19:39 +08:00

crossover_pd_advantage.png

PD-disagg crossover: regular synthetic trace + goodput sweep + figure

2026-05-29 18:19:23 +08:00

f2a_reuse_topology.png

EAR outline: copy reusable figures, mark migration sections deferred

2026-05-27 01:44:13 +08:00

f2b_session_skew_prod.png

Add solo production-trace CDF figure (f2b_session_skew_prod.png)

2026-05-27 10:53:30 +08:00

f2b_session_skew.png

f2b: regenerate CDF from production trace (1.3M sessions on dash0)

2026-05-27 10:41:53 +08:00

f2c_kv_footprint_cdf.png

f2c: switch to per-instance decode-concurrency view; correct KV pool ceiling

2026-05-27 11:28:47 +08:00

f3a_inter_turn_gap.png

Add chatbot T_external CDF; overlay on f3a vs agentic

2026-05-27 14:49:44 +08:00

f4a_apc_loss.png

Drop 'capped' from per-policy figures (f4a, f4c×2, f6)

2026-05-27 10:57:43 +08:00

f4b_pdsep_kv_wall.pdf

EAR outline: copy reusable figures, mark migration sections deferred

2026-05-27 01:44:13 +08:00

f4b_pdsep_kv_wall.png

Use PNG for KV memory wall figure; switch outline to inline image embeds

2026-05-27 09:13:26 +08:00

f4c_per_worker_ttft.png

Replace max/median hotspot index with (median, max) absolute pair

2026-05-27 11:07:12 +08:00

f4d_pd_interference.png

EAR outline: copy reusable figures, mark migration sections deferred

2026-05-27 01:44:13 +08:00

f6_e2e_latency_bars.png

Drop 'capped' from per-policy figures (f4a, f4c×2, f6)

2026-05-27 10:57:43 +08:00

f6_e2e_latency_full_grid.png

Add full latency grid (mean/p50/p90/p99 × TTFT/TPOT/E2E) as f6 companion

2026-05-27 11:15:18 +08:00

mb1_interference.png

Correct PD-disagg cost/benefit framing across repo

2026-05-27 22:04:49 +08:00

mb2_transfer_bw_compare.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_bw_inter.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_bw_intra.png

MB2: pure KV-transfer cost on dash1 intra-node — Mooncake ~9.7 GB/s steady

2026-05-27 19:04:03 +08:00

mb2_transfer_time_compare.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_time_inter.png

MB2 inter-node: dash1↔dash2 transfer cost is identical to intra-node

2026-05-27 20:56:08 +08:00

mb2_transfer_time_intra.png

MB2: pure KV-transfer cost on dash1 intra-node — Mooncake ~9.7 GB/s steady

2026-05-27 19:04:03 +08:00