Audit package refresh: Window 1 supported claims + risk register
Refresh the standing audit package now that B1' / B2 / B3 are complete. current_results/characterization_claim_matrix.md Flips seven entries from "not_yet_supported" / "partially_supported" to "supported" with pointers into window_1_results/. New entries cover per-session sequentiality, KV per request, real reuse decomposition, theoretical APC ceiling, the LMetric locality gap, Unified breaking the locality-vs-latency tradeoff, B2 causal interference proof, sticky's interference inflation, and the partial heavy-tail / hot-spot story. B4 SRR + B5 attribution stay "not_yet_supported" (Window 2 work). current_results/main_claim_allowed_runs.md New "Allowed For Routing-Policy Comparison" section pins the five B3 policy directories. New "Allowed For PD-colo Interference" section pins the B2 sweep. Legacy section retained for the pre-instrumentation 200/500/1000-req runs. current_results/reviewer_risk_register.md Marks the two old "high"-severity risks (sequentiality / reuse decomposition) as resolved; adds new entries for the APC contamination empirics, the b3_analyze.sh truncate-write bug that cost unified's interference index, the GPU-0 EngineCore ghost cleanup, the saturated-replay caveat for trace-timestamp dispatch, and the synthetic B2 decode workload. current_results/all_figures_index.md Adds the 8 new Window 1 figures alongside the existing 6 from the legacy summarize_runs run. current_results/reproduction_commands.sh Records the full B3 + B2 + figure pipeline. analysis/characterization_todo_for_interns.md Updates the Progress Snapshot table: B0, B1, B2, B3, B6 all DONE; only B4 and B5 remain (Window 2). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -1,54 +1,29 @@
|
||||
# Figures Index
|
||||
|
||||
Generated by:
|
||||
|
||||
```bash
|
||||
.venv/bin/python analysis/characterization/plot_current_results.py
|
||||
```
|
||||
## Window 0 (pre-Window-1 audit, legacy runs)
|
||||
|
||||
| Figure | Intended Claim |
|
||||
|---|---|
|
||||
| [fig_full_trace_workload.png](figures/fig_full_trace_workload.png) | Full GLM-5.1 trace is long-input, short-output, and high input/output ratio. |
|
||||
| [fig_session_skew.png](figures/fig_session_skew.png) | Session input-token mass is highly skewed; top sessions dominate work. |
|
||||
| [fig_pdsep_vs_combined.png](figures/fig_pdsep_vs_combined.png) | Existing static PD-sep A/B regresses TTFT/E2E vs combined. |
|
||||
| [fig_pdsep_vs_combined.png](figures/fig_pdsep_vs_combined.png) | Static PD-sep regresses TTFT/E2E vs combined (legacy 200-req A/B). |
|
||||
| [fig_elastic_vs_baseline.png](figures/fig_elastic_vs_baseline.png) | Existing elastic transfer-based run does not improve TTFT/TPOT over high-contention baseline. |
|
||||
| [fig_gpu_balance.png](figures/fig_gpu_balance.png) | Existing runs show GPU-util imbalance, but more data is needed for hot-spot causality. |
|
||||
| [fig_claim_status.png](figures/fig_claim_status.png) | Current audit separates supported, partial, and unsupported claims. |
|
||||
| [fig_gpu_balance.png](figures/fig_gpu_balance.png) | Existing runs show GPU-util imbalance; not sufficient for hot-spot causal claim. |
|
||||
| [fig_claim_status.png](figures/fig_claim_status.png) | Audit separates supported / partial / unsupported claims. |
|
||||
|
||||
## Figure Previews
|
||||
## Window 1 (B1' + B3 + B2)
|
||||
|
||||
### Full Trace Workload
|
||||
Generated by `analysis/characterization/render_window1_figures.py`.
|
||||
Source data: `analysis/characterization/window_1_results/`.
|
||||
|
||||
Full GLM-5.1 trace is long-input, short-output, and high input/output ratio.
|
||||
|
||||
|
||||
|
||||
### Session Skew
|
||||
|
||||
Session input-token mass is highly skewed; top sessions dominate work.
|
||||
|
||||
|
||||
|
||||
### PD-Sep vs Combined
|
||||
|
||||
Existing static PD-sep A/B regresses TTFT/E2E vs combined.
|
||||
|
||||
|
||||
|
||||
### Elastic vs Baseline
|
||||
|
||||
Existing elastic transfer-based run does not improve TTFT/TPOT over high-contention baseline.
|
||||
|
||||
|
||||
|
||||
### GPU Balance
|
||||
|
||||
Existing runs show GPU-util imbalance, but more data is needed for hot-spot causality.
|
||||
|
||||
|
||||
|
||||
### Claim Status
|
||||
|
||||
Current audit separates supported, partial, and unsupported claims.
|
||||
|
||||
|
||||
| Figure | Intended Claim |
|
||||
|---|---|
|
||||
| [fig_kv_footprint_cdf.png](../window_1_results/figures/fig_kv_footprint_cdf.png) | KV per request for Qwen3-Coder-30B-A3B: p50/p90/p99 = 1.83/8.04/11.49 GiB; p99 takes 12% of H20 HBM. |
|
||||
| [fig_reuse_decomposition.png](../window_1_results/figures/fig_reuse_decomposition.png) | Cached_tokens decompose 93.2% intra / 5.7% cross / 1.1% shared on w600 lmetric run. |
|
||||
| [fig_b3_apc_vs_upper.png](../window_1_results/figures/fig_b3_apc_vs_upper.png) | Per-policy APC achieved vs theoretical intra-session ceiling (79.6%). |
|
||||
| [fig_b3_apc_vs_hotspot.png](../window_1_results/figures/fig_b3_apc_vs_hotspot.png) | Locality-vs-hotspot tradeoff across policies; unified dominates the frontier. |
|
||||
| [fig_b3_latency_bars.png](../window_1_results/figures/fig_b3_latency_bars.png) | TTFT / TPOT / E2E p90 bars per policy. |
|
||||
| [fig_b3_per_worker_ttft_p90.png](../window_1_results/figures/fig_b3_per_worker_ttft_p90.png) | Per-worker TTFT p90 distribution per policy; sticky's engine_3 and unified's engine_4 are the hot workers. |
|
||||
| [fig_b3_failure_breakdown.png](../window_1_results/figures/fig_b3_failure_breakdown.png) | Slow-request cause stacked bar per policy. |
|
||||
| [fig_b2_tpot_vs_prefill.png](../window_1_results/figures/fig_b2_tpot_vs_prefill.png) | TPOT during decode under same-worker prefill injection scales with prefill size; different-worker control flat. |
|
||||
| [fig_b2_ttft_vs_prefill.png](../window_1_results/figures/fig_b2_ttft_vs_prefill.png) | TTFT shows the same monotone same-worker scaling, peaking at 218× for 65k injection. |
|
||||
|
||||
Reference in New Issue
Block a user