After the B3 audit bug fixes (joined_analysis hotspot median +
b3_analyze percentile interp), regenerate b3_policy_comparison.json
and the per-policy hotspot_index.json from the same raw run on
dash0 and re-render the three affected figures (apc-vs-hotspot,
latency-bars, per-worker TTFT).
Key number changes in window_1_results.md:
- hotspot_index magnitudes corrected (all five policies; lmetric
smallest delta at +0.7%, sticky largest at +16.1%)
- "capped reduces hotspot 13%" -> "~10% (2.253 -> 2.020)"
- TTFT/E2E/TPOT percentiles shift by <1% from floor->interp
(unified TTFT p90 7.24 -> 7.35 s)
Restructured "Caveats" into "Limitations (read this before quoting
B3 numbers)":
1. Agentic dispatch coupling is by design — promoted from caveat
to top-level methodology framing, tied to
agentic_dispatch_coupling.md
2. B3 interference_index is binary (not size-graded) — added
3. Hot-sweep cache contamination (<1%) — kept
4. Unified interference unrecoverable — kept with explicit warning
not to read unified's failure attribution as causal
5. w600 is a sample, not full trace — kept
6. Reuse decomposition is per-token in expectation — added
current_results/characterization_claim_matrix.md updates:
- The "heavy-tail not sole cause" claim now cites the corrected
~10% drop with the median bug noted
- New supported claim: "B3 saturated-replay latency gaps include an
agentic dispatch-coupling feedback term, which is intentional and
matches production"; cited against agentic_dispatch_coupling.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The B2 same-worker TPOT p90 idx is non-monotone: 7.89x at 32k drops
to 2.26x at 65k. The naive reading is "interference gets weaker for
huge prefills"; the actual mechanism is a regime shift, and reading
TPOT p90 alone is misleading.
Three superimposed effects:
1. Cost migration TPOT -> TTFT. A 32k prefill is short enough that
chunked-prefill keeps interleaving decode steps, so overlapping
decodes trickle tokens out at painful per-token rates. A 65k
prefill is long enough that overlapping decodes are *fully*
blocked for ~10s; once they break through, the injection is
winding down and subsequent iterations run unobstructed. The
cost lands on the TTFT clock (14s) instead of inflating TPOT.
2. Bimodal TPOT distribution. At 65k overlap, decodes split into
"blocked entire prefill then normal rate" and "trickled slowly
through prefill chunks". p99 sits on the second population and
grows 59 -> 169.5 ms; p90 sits on the first and shrinks.
3. "Clean" stops being clean. With 4x ~10s injections in 60s, the
110 "clean" decodes at 65k are squeezed into 2-3s recovery
pockets. TPOT p90 clean rises 6.9 -> 9.6 ms (40%), shrinking
the denominator of the ratio.
window_1_results.md adds a new B2 subsection laying out the
mechanism with the per-cell data table and the explicit reading
rule: headline interference metric is TTFT idx (monotone); TPOT
p99 is the right tail indicator; TPOT p90 alone is unsafe across
regime shifts. Direct implication: TTFT and TPOT need separate
SLO thresholds under PD-colo, because they measure costs from
different points in the request lifecycle and the cost migration
between them is workload-dependent.
current_results/characterization_claim_matrix.md adds a new
supported claim for the cost migration, listed against the existing
B2 evidence. current_results/reviewer_risk_register.md adds a
low-severity entry warning future readers off TPOT p90 alone.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Refresh the standing audit package now that B1' / B2 / B3 are complete.
current_results/characterization_claim_matrix.md
Flips seven entries from "not_yet_supported" / "partially_supported"
to "supported" with pointers into window_1_results/. New entries
cover per-session sequentiality, KV per request, real reuse
decomposition, theoretical APC ceiling, the LMetric locality gap,
Unified breaking the locality-vs-latency tradeoff, B2 causal
interference proof, sticky's interference inflation, and the
partial heavy-tail / hot-spot story. B4 SRR + B5 attribution stay
"not_yet_supported" (Window 2 work).
current_results/main_claim_allowed_runs.md
New "Allowed For Routing-Policy Comparison" section pins the five
B3 policy directories. New "Allowed For PD-colo Interference"
section pins the B2 sweep. Legacy section retained for the
pre-instrumentation 200/500/1000-req runs.
current_results/reviewer_risk_register.md
Marks the two old "high"-severity risks (sequentiality / reuse
decomposition) as resolved; adds new entries for the APC
contamination empirics, the b3_analyze.sh truncate-write bug that
cost unified's interference index, the GPU-0 EngineCore ghost
cleanup, the saturated-replay caveat for trace-timestamp dispatch,
and the synthetic B2 decode workload.
current_results/all_figures_index.md
Adds the 8 new Window 1 figures alongside the existing 6 from the
legacy summarize_runs run.
current_results/reproduction_commands.sh
Records the full B3 + B2 + figure pipeline.
analysis/characterization_todo_for_interns.md
Updates the Progress Snapshot table: B0, B1, B2, B3, B6 all DONE;
only B4 and B5 remain (Window 2).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
