aituner/docs/opprof/phase5-results.md

OpProf Phase 5 dash0 results

Status: FINAL — INCONCLUSIVE MECHANISM LEDGER; NO DOMINANCE CALL.

Date: 2026-07-12. All 15 registered P10 primary runs completed with three replicates per arm and passed A-P5-1. The fresh A3 bridge passed, so the frozen P3 P03/P04 controls were reused. The ledger is nevertheless inconclusive: P04's reused-control denominator is unstable, and A2, A3, and A4 fail their predeclared manipulation checks. The protocol therefore permits one official share (A1 under P03), requires the other lines to be N/A, and forbids a dual-control dominance conclusion.

The machine result is runs/opprof-phase5/phase5/metrics.json (SHA-256 fe16934866806ac542f7974cac061beeb1f1c58f5bfb36698efd8aeaa99b8cb7). The final analyzer SHA-256 is 8b0ff8658614227ee00cf8d10c82c4b5edf6f6210db172fd6efc9cc149e51e10; its no-GPU tools and analysis tests pass.

This remains explicitly a recorded-arrival P5 bridge ledger anchored to P3 controls, not a literal decomposition of P3's already-uniform P10 gap.

Data-sanity result first

RED FLAG: the P04 control denominator is unstable. P5 recorded-arrival base has E_token=3.013752, while reused P3 P04 has E_token=3.054704, leaving only a 0.040952 gap with bootstrap 95% CI [-0.649096, 0.752594]; 45.353% of denominator draws are nonpositive. This violates every registered stability form: the CI contains zero and the nonpositive fraction exceeds 5%.

P03 remains stable: its gap is 1.721848, CI [1.459092, 1.988772], with zero nonpositive draws. All execution-validity checks pass: 15/15 primary runs, three replicates per arm, exact 240-second clean windows, zero clean failures, offered rates within 5%, continuous/balanced Layer-1 accounting, identical request IDs and token sums, all A-P5-1 cold-start gates, no drain quarantine, and GPU spend below the cap. The red flag is reported before the ledger and no conclusion is built on P04's raw ratio draws.

Bridge and absolute efficiency

A3 reproduces the P3 uniform-arrival P10 base: delta_E=+0.007041, 95% CI [-0.329799, 0.354859], or 0.269% absolute relative difference. The CI contains zero and the point difference is below 3%, so the bridge passes and P3 control reuse is valid under the frozen rule. Conditional control reruns were not launched.

Arm	E_token	Bootstrap 95% CI	Delta from P5 base
Base, recorded	3.013752	[2.754462, 3.267964]	—
A1 length-binned	3.078999	[2.801659, 3.349307]	+0.065247
A2 capture sizes	3.062128	[2.802812, 3.314276]	+0.048376
A3 uniform arrival	2.626154	[2.451852, 2.792684]	-0.387598
A4 prefix cache off	3.020959	[2.760494, 3.274105]	+0.007207

Manipulation checks and Holm family

Arm	Manipulation result	Efficiency delta 95% CI	Raw p	Holm p
A1	PASS: R16 0.641744 -> 0.473409; max delay 62.743 s; prefix-hit-ratio delta -0.296 pp	[-0.309461, 0.442565]	0.73108	1.00000
A2	FAIL: support coverage 98.879% <99%; padding reduction 82.541% <90%	[-0.315361, 0.411446]	0.79022	1.00000
A3	FAIL: decode-B CV falls 0.8706 -> 0.6551, but waiting CV rises 6.0145 -> 6.8557	[-0.693444, -0.078141]	0.01384	0.05536
A4	FAIL: disabled arm still reports 3,093,324 local prefix queries and 52,896 hits, not zero	[-0.357637, 0.369309]	0.97230	1.00000

Per protocol, a failed manipulation makes that mechanism's official share N/A even if its raw efficiency contrast is nonzero. A3's uncorrected contrast is negative, and it also misses family-wise significance after Holm correction.

A-P5-1 arrival evidence

The four originally invalidated arms supplied the evidence for A-P5-1. Under the superseded throughput-drift calculation, recorded base/A1/A4 span 190.55%-216.50% normalized drift, while uniform A3 is 13.23%. This gap is retained as direct evidence that the old gate measured arrival shape in a rate-following experiment; it is not converted into an efficiency share.

All 15 replacement/continued primary runs pass the amended cold-start gate: 25-28 warm-up completions, 12-16 completions with input at least 8192 tokens, all compile/capture first occurrences before clean, zero clean-window first-capture events, and zero uncovered Layer-1 graph descriptors. The 10% drift gate was retained unchanged for the P06 saturation background arm.

Operational findings

• The detached controller completed three cache burn-ins, four four-way waves, all drains, validation, budget accounting, and cleanup. The four invalidated first-wave arms were rerun exactly; none of their superseded measurements enters the ledger.
• The first exact rerun produced complete clean artifacts but hit a post-hoc validator NameError from a missing math import. Immutable artifacts were CPU-re-adjudicated under A-P5-1, the import was repaired, and no third GPU run was charged.
• CPU analysis repairs admitted legitimate idle fixed-time blocks as [0,0], supported the older P3 control schema, converted one NumPy boolean at the JSON boundary, and explicitly separated diagnostic from reportable shares. These changes alter no GPU data, estimator, seed, threshold, or resample count.
• P10 drains range from 0.676 to 6.004 seconds. The separate P06 saturation background run drained naturally in 87.903 seconds, below the 600-second gate, with 564/564 clean completions and zero failures.
• Public Phase-5 artifacts occupy 286,205,365 bytes (304 MiB), below the 3 GB stop threshold; CPFS retained 1,286 GB free.
• MoE routing skew remains content-bound and receives no causal share. The optional routed-expert analysis-only run was not required for this primary ledger and was not launched.

Launch echo

LAUNCH_ECHO utc=2026-07-12T12:07:54Z host=dash0 gpus=0-3 cpus=0-79 source=/home/admin/cpfs/wjh/opprof-phase2-dash0-20260711/vllm-v0.24.0@4b253fd manifests=/home/admin/cpfs/wjh/opprof-phase5-private/manifests outputs=/home/admin/cpfs/wjh/opprof-phase3-dash0-20260712/runs/phase5 runs=3burnin+15primary+1background rate=0.4725 warmup=60s clean=240s drain=600s est_wall=35-60min est_gpu=1.7-2.1_H20h hard_cap=6.0_H20h conditional_controls=6_if_bridge_fails

Resume after A-P5-1 reused the completed burn-ins and compile/cudagraph caches. The first launch through final controller completion took approximately 57.9 minutes wall time, including the adjudication/resume interval.

Run completion stats

Item	Count/result
Valid primary P10 runs	15/15
Replicates	3 per arm
Valid P06 background runs	1
Completed burn-ins	3
Superseded pre-amendment arms	4
Conditional fresh controls	0; bridge passed, P3 reused
Optional secondary/routed-expert runs	0
A-P5-1 primary passes	15/15
Clean failures	0
Drain quarantines	0

Mechanism ledger

Official shares follow the frozen N/A rules. Values in parentheses are raw diagnostic ratios retained for audit, not causal/reportable shares.

Mechanism	P03 share (95% CI)	P04 share (95% CI)	Disposition
A1 length raggedness	0.0379 [-0.2011, 0.2344]	N/A; unstable denominator (diagnostic 1.5933 [-6.8925, 7.0636])	Manipulation passes; no significant recovery
A2 capture mismatch	N/A; manipulation failed (diagnostic 0.0281 [-0.2049, 0.2170])	N/A; unstable denominator (diagnostic 1.1813 [-6.2871, 6.4924])	Config did not meet 99%/90% removal gates
A3 arrival dynamics	N/A; manipulation failed (diagnostic -0.2251 [-0.4625, -0.0403])	N/A; unstable denominator (diagnostic -9.4647 [-17.2450, 17.4102])	Queue-variance gate failed; Holm p=0.05536
A4 prefix structure	N/A; manipulation failed (diagnostic 0.0042 [-0.2340, 0.1948])	N/A; unstable denominator (diagnostic 0.1760 [-6.2590, 6.4925])	Zero-query/hit gate failed

Shares are neither clipped nor renormalized. P04's extreme raw ratios are the registered consequence of dividing by a near-zero, sign-unstable denominator, not evidence of large mechanisms.

Dominance verdicts

Mechanism	P03 rule	P04 rule	Dual-control verdict
A1	Fails share/CI/Holm thresholds	Not evaluable	NOT EVALUABLE
A2	Not evaluable: manipulation failed	Not evaluable	NOT EVALUABLE
A3	Not evaluable: manipulation failed	Not evaluable	NOT EVALUABLE
A4	Not evaluable: manipulation failed	Not evaluable	NOT EVALUABLE

No mechanism may be called dominant because the frozen rule requires both control denominators. This is an inconclusive verdict, not evidence that every mechanism is small.

Residual + interaction line

The official residual is N/A under both controls because the official share ledger is incomplete. The unchanged arithmetic over all raw diagnostic shares is reported only for transparency:

Control	Diagnostic share sum	Diagnostic residual + interaction (95% CI)	Status
P03	-0.1549	1.1549 [0.5536, 1.9397]	Diagnostic only
P04	-6.5142	7.5142 [-21.3965, 22.8767]	Diagnostic only; unstable denominator

Nothing is forced to 100%. The residual can include interactions, unablated MoE routing, chunked-prefill effects, double-counting, and measurement error.

Config-tier A2 measured delta

A2 added the P3-derived sizes {3,5,6,7} to the full default capture list. Its measured delta_E is +0.048376, 95% CI [-0.315361, 0.411446], or +1.605% relative to base. Pure-decode padding falls from 13.7405% to 2.3989%, an 82.541% reduction. New recorded-arrival runs expose decode-B support 1-13, so uncaptured 9-13 leave coverage at 98.879%; both this and the sub-90% padding reduction invalidate the causal A2 share.

The config cost is reproducible: base graph capture is 11 seconds and 0.80 GiB in all three runs; A2 is 12-14 seconds and 0.84 GiB, a 1-3 second and 0.04 GiB increase. The measured config-tier result is therefore a small, statistically uncertain efficiency gain with a failed preregistered removal gate.

GPU total

New Phase-5 spend is 2.4388167443 H20-hours of the 6.0-hour cap, including the superseded first wave, exact reruns, continued primary matrix, burn-ins, and background arm. 3.5611832557 H20-hours remain unspent. Final state on all eight H20s is 0 MiB and 0% utilization with zero compute processes.

Sanity block

Anomaly first: P04's gap CI contains zero and 45.353% of its denominator draws are nonpositive; the dual-control ledger is inconclusive and inferential claims stop there.

Numeric family	n	Finite	Missing	Min	Max	Distinct	Checked invariant/result
Primary run E_token	15	15	0	2.623534	3.083263	15	Positive; per-arm results not all identical
Clean Layer-1 steps	15	15	0	11,178	18,400	15	Non-negative; continuous/balanced per run
Clean duration (s)	15	15	0	240	240	1 expected	Exact 3*80; zero clean failures
Offered rate (req/s)	15	15	0	0.470833	0.487500	2	Every run within 5% of 0.4725
Warm-up completions	15	15	0	25	28	2	Every run >=16
Warm-up long completions	15	15	0	12	16	3	Every run >=1 with input >=8192
Clean first-capture events	15	15	0	0	0	1 expected	Zero events and zero uncovered descriptors
P10 drain duration (s)	15	15	0	0.676181	6.003959	15	Non-negative; all below 600
Control gap E_token	2	2	0	0.040952	1.721848	2	P03 stable; P04 unstable red flag
Raw diagnostic shares	8	8	0	-9.464746	1.593268	8	Finite; intentionally not constrained to [0,1]
Diagnostic residuals	2	2	0	1.154931	7.514211	2	Finite; never forced to close
New H20-hours	1	1	0	2.438817	2.438817	1	Non-negative; below 6.0

Checked invariants: source/helper/protocol provenance; identical request IDs, content/token totals, and prompt-preserving manifests; C00-TP1 four-way placement; detached controller ownership; exact clean windows; zero failures; offered-rate tolerance; A-P5-1 compile/capture ordering and coverage; Layer-1 footer/sidecar balance, continuous step indices, token composition, and zero drops; non-negative counters; drain gates; bridge decision; no public prompt or generated-text leakage; public-disk/free-space limits; GPU hard-cap compliance; and zero-process/zero-memory cleanup. Manipulation failures are A2, A3, and A4; the sole machine sanity red flag is P04 denominator instability.