Gahow Wang cf812b6264 Workload characterization C1-C3 on full production trace ...

Joint/temporal characterizations of the full 051315 cluster trace (2.11M
req / 1.31M sessions / 2h), beyond the existing single-variable marginals:

- C1 mixture: 90.3% sessions single-turn, but multi-turn (9.7%) = 44% reqs /
  67% prefill mass; continuation hazard rises 10%->94% (Lindy); heaviness
  unpredictable at turn 1 (corr 0.04-0.15) => reactive routing justified.
- C2 resident/delta: resident context 11k->56k while new-prefill 2.7k->~200;
  per-turn reuse ->99.6%; resident/delta ("PD tax") ->~250-450x.
- C3 prefill/decode: token mass 98.7% input / 1.3% output, BUT decode ~70% of
  TIME (robust 68-71%); "decode negligible" is wrong (tokens != time). Correct
  colo argument = roofline complementarity, not "no decode".

Maps each to (1) PD-colocation and (2) routing. compute_chars.py + chars.json
+ figs/workload_chars/. Raw-file exact validation (cached_tokens, real
timings) pending.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-29 18:19:39 +08:00

..

characterization

Add chatbot T_external CDF; overlay on f3a vs agentic

2026-05-27 14:49:44 +08:00

crossover

PD-disagg crossover: regular synthetic trace + goodput sweep + figure

2026-05-29 18:19:23 +08:00

mb1

Correct PD-disagg cost/benefit framing across repo

2026-05-27 22:04:49 +08:00

mb2

Correct PD-disagg cost/benefit framing across repo

2026-05-27 22:04:49 +08:00

mb5

PD_DISAGG_RESULTS §6.3: producer hot-pinning figure

2026-05-29 00:38:20 +08:00

pd_sep_paper_section

PD-sep matrix results: C2/C3/C4 figures + empirical mechanism refined

2026-05-25 16:23:52 +08:00

working_set

Working-set sizing tool + GLM-5.1-FP8/B300 result

2026-05-28 16:03:25 +08:00

workload_chars

Workload characterization C1-C3 on full production trace

2026-05-29 18:19:39 +08:00

adaptive_prefill_offload_design.md

Design doc: Adaptive Prefill Offload

2026-05-22 00:44:22 +08:00

agentic_pd_unified_story_plan.md

Agentic PD / Unified routing story plan draft

2026-05-26 01:12:42 +08:00

characterization_todo_for_interns.md

Audit package refresh: Window 1 supported claims + risk register

2026-05-25 23:25:27 +08:00

claude_characterization_work_plan.md

Characterization plan: progress snapshot + Claude work plan

2026-05-25 16:18:41 +08:00

elastic_hypotheses.md

Docs: reconcile routing docs with current hybrid direction

2026-05-25 10:47:14 +08:00

elastic_offload_design.md

Elastic P2P offload: TTFT p50 -49% vs baseline (0.551 vs 1.080)

2026-05-22 13:50:25 +08:00

kv_lifecycle_design.md

KV cache lifecycle design + eviction loss analysis

2026-05-22 01:27:22 +08:00

lpwl_5policy_600s.md

Add leastwork_kappa decode-aware ablation (net-negative, documented)

2026-05-29 17:07:23 +08:00

overnight_work_report.md

Update report: adaptive v2 confirms no KV transfer helps single-machine

2026-05-22 10:15:08 +08:00

pd_separation_analysis.md

Invalidate prior A/B results + add proper experiment harness

2026-05-22 17:54:21 +08:00

research_findings.md

Docs: reconcile routing docs with current hybrid direction

2026-05-25 10:47:14 +08:00

unified_routing_fix_review.md

Docs: reconcile routing docs with current hybrid direction

2026-05-25 10:47:14 +08:00