Files

Gahow Wang 4722883903 Audit package refresh: Window 1 supported claims + risk register

Refresh the standing audit package now that B1' / B2 / B3 are complete.

current_results/characterization_claim_matrix.md
  Flips seven entries from "not_yet_supported" / "partially_supported"
  to "supported" with pointers into window_1_results/. New entries
  cover per-session sequentiality, KV per request, real reuse
  decomposition, theoretical APC ceiling, the LMetric locality gap,
  Unified breaking the locality-vs-latency tradeoff, B2 causal
  interference proof, sticky's interference inflation, and the
  partial heavy-tail / hot-spot story. B4 SRR + B5 attribution stay
  "not_yet_supported" (Window 2 work).

current_results/main_claim_allowed_runs.md
  New "Allowed For Routing-Policy Comparison" section pins the five
  B3 policy directories. New "Allowed For PD-colo Interference"
  section pins the B2 sweep. Legacy section retained for the
  pre-instrumentation 200/500/1000-req runs.

current_results/reviewer_risk_register.md
  Marks the two old "high"-severity risks (sequentiality / reuse
  decomposition) as resolved; adds new entries for the APC
  contamination empirics, the b3_analyze.sh truncate-write bug that
  cost unified's interference index, the GPU-0 EngineCore ghost
  cleanup, the saturated-replay caveat for trace-timestamp dispatch,
  and the synthetic B2 decode workload.

current_results/all_figures_index.md
  Adds the 8 new Window 1 figures alongside the existing 6 from the
  legacy summarize_runs run.

current_results/reproduction_commands.sh
  Records the full B3 + B2 + figure pipeline.

analysis/characterization_todo_for_interns.md
  Updates the Progress Snapshot table: B0, B1, B2, B3, B6 all DONE;
  only B4 and B5 remain (Window 2).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-25 23:25:27 +08:00

4.1 KiB

Raw Blame History

Characterization Claim Matrix

Updated 2026-05-25 after Window 1 (B1' KV-footprint + reuse, B3 5-policy sweep, B2 PD-colo interference microbench).

Claim	Status	Supporting Data	Needed Next	Reviewer Risk
Per-session sequentiality is enforced when replayer + proxy carry the new join fields.	`supported`	A1 unix timestamps (t_dispatch/t_first_token/t_finish_unix) and X-Request-Id passthrough; smoke validation 2026-05-25 confirmed 30/30 join coverage.	Use this stack for all Window 2 B4/B5 SRR runs.	Legacy outputs/ without these fields still cannot be re-classified as `online_realistic`.
Agentic workload is long-input / short-output / heavy-tail session mass.	`supported`	Full trace CPU summary (full_trace_summary.json): input p50/p90/p99 = 20k/87.9k/125.5k; top 1% sessions hold 46.5% of input mass. Full trace 2.11M requests, 1.31M sessions.	—	Sample trace (w600) percentiles inherit from this full trace but should not be extrapolated.
KV per request for Qwen3-Coder-30B-A3B is 98304 B/token; p50/p90/p99 footprint = 1.83/8.04/11.49 GiB.	`supported`	window_1_results/kv_footprint_summary.json; computed from model config and full trace input lengths.	—	Assumes bf16; would scale for fp8/int8 quant.
Workload reuse is overwhelmingly intra-session.	`supported`	Real reuse decomposition from w600 lmetric run: intra 93.2%, cross 5.7%, shared 1.1% (window_1_results/lmetric_reuse.json). Theoretical any-vs-intra ceiling gap 0.7 pp.	—	Trace-specific; ChatGPT-style workloads with long system prompts would shift toward shared-prefix.
Theoretical APC ceiling on w600 trace is 79.6% (intra) / 80.3% (any-session).	`supported`	window_1_results/apc_upper_w600.json from block-level trie walk on `hash_ids`.	—	Assumes infinite per-worker cache (no eviction); achieved values must be read as fraction of this ceiling.
Cache-aware LMetric leaves a measurable locality gap (22.7 pp).	`supported`	lmetric achieved 56.9% vs intra-session ceiling 79.6%; B3 sweep window_1_results/b3_policy_comparison.json.	—	sticky data shows the gap can be recovered by harder affinity.
Hybrid affinity (`unified`) breaks the locality-vs-latency tradeoff.	`supported`	unified APC 79.4% (97% of intra ceiling) AND TTFT p90 7.24 s (lmetric is 15.6 s).	—	unified concentrates a single very hot worker (engine_4 at 37.7 s p90); hotspot_index 3.35.
Same-worker prefill-decode interference is causal, not correlation.	`supported`	B2 microbench: different-worker control idx 0.92-1.02 across 32× prefill-size variation; same-worker TTFT idx scales 2.15× (2k) → 218× (65k). window_1_results/b2_sweep_summary.json.	—	Synthetic decode load (256-token prompts at 4 req/s) bounds the realism; production behavior is layered on top of B3.
Hard session affinity (`sticky`) inflates same-worker prefill-decode interference.	`supported`	sticky interference_index 13.65 vs lmetric 6.53; sticky's slow-request breakdown 57% same-worker overlap vs lmetric 23%.	—	Confirms the B2 causal claim observed at the system level.
Heavy-tail sessions are a contributor to hot-spot but not the sole cause.	`supported`	Cap-8 trace (37% requests dropped) reduces hotspot_index only 13% (2.24 → 1.94).	Run capped under unified to see whether unified's hotspot also persists.	Reviewer might counter that cap=8 is too soft; a stricter cap could be tried.
SRR per policy under SLO is not yet measured.	`not_yet_supported`	B3 was driven by trace timestamps with strict session sequentiality; saturation is reached but not parameterized.	Run B4 with the A4 open-loop Poisson loadgen, per-class SLO, 5 policies × λ binary search.	Without B4 the paper cannot claim "policy X sustains higher load than Y".
Failure attribution near SRR boundary is not yet measured.	`not_yet_supported`	B5 protocol exists; no runs.	After B4, rerun each policy at 0.9× / 1.0× / 1.1× of its SRR_max with the same instrumentation, label slow requests.	The current `joined_analysis.label_slow_requests` is the labeler; needs SRR boundaries to point at.

4.1 KiB Raw Blame History Unescape Escape

Characterization Claim Matrix

4.1 KiB

Raw Blame History