aituner/docs/opprof/phase5-results.md

# OpProf Phase 5 dash0 results

Status: **FINAL — INCONCLUSIVE MECHANISM LEDGER; NO DOMINANCE CALL**.

Date: 2026-07-12. All 15 registered P10 primary runs completed with three
replicates per arm and passed A-P5-1. The fresh A3 bridge passed, so the frozen
P3 P03/P04 controls were reused. The ledger is nevertheless inconclusive:
P04's reused-control denominator is unstable, and A2, A3, and A4 fail their
predeclared manipulation checks. The protocol therefore permits one official
share (A1 under P03), requires the other lines to be N/A, and forbids a
dual-control dominance conclusion.

The machine result is `runs/opprof-phase5/phase5/metrics.json` (SHA-256
`fe16934866806ac542f7974cac061beeb1f1c58f5bfb36698efd8aeaa99b8cb7`).
The final analyzer SHA-256 is
`8b0ff8658614227ee00cf8d10c82c4b5edf6f6210db172fd6efc9cc149e51e10`;
its no-GPU tools and analysis tests pass.

This remains explicitly a **recorded-arrival P5 bridge ledger anchored to P3
controls**, not a literal decomposition of P3's already-uniform P10 gap.

## Data-sanity result first

**RED FLAG: the P04 control denominator is unstable.** P5 recorded-arrival
base has `E_token=3.013752`, while reused P3 P04 has `E_token=3.054704`, leaving
only a `0.040952` gap with bootstrap 95% CI `[-0.649096, 0.752594]`; 45.353% of
denominator draws are nonpositive. This violates every registered stability
form: the CI contains zero and the nonpositive fraction exceeds 5%.

P03 remains stable: its gap is `1.721848`, CI `[1.459092, 1.988772]`, with zero
nonpositive draws. All execution-validity checks pass: 15/15 primary runs,
three replicates per arm, exact 240-second clean windows, zero clean failures,
offered rates within 5%, continuous/balanced Layer-1 accounting, identical
request IDs and token sums, all A-P5-1 cold-start gates, no drain quarantine,
and GPU spend below the cap. The red flag is reported before the ledger and no
conclusion is built on P04's raw ratio draws.

## Bridge and absolute efficiency

A3 reproduces the P3 uniform-arrival P10 base: `delta_E=+0.007041`, 95% CI
`[-0.329799, 0.354859]`, or 0.269% absolute relative difference. The CI contains
zero and the point difference is below 3%, so the bridge passes and P3 control
reuse is valid under the frozen rule. Conditional control reruns were not
launched.

| Arm | `E_token` | Bootstrap 95% CI | Delta from P5 base |
|---|---:|---:|---:|
| Base, recorded | 3.013752 | [2.754462, 3.267964] | — |
| A1 length-binned | 3.078999 | [2.801659, 3.349307] | +0.065247 |
| A2 capture sizes | 3.062128 | [2.802812, 3.314276] | +0.048376 |
| A3 uniform arrival | 2.626154 | [2.451852, 2.792684] | -0.387598 |
| A4 prefix cache off | 3.020959 | [2.760494, 3.274105] | +0.007207 |

## Manipulation checks and Holm family

| Arm | Manipulation result | Efficiency delta 95% CI | Raw p | Holm p |
|---|---|---:|---:|---:|
| A1 | **PASS:** R16 0.641744 -> 0.473409; max delay 62.743 s; prefix-hit-ratio delta -0.296 pp | [-0.309461, 0.442565] | 0.73108 | 1.00000 |
| A2 | **FAIL:** support coverage 98.879% <99%; padding reduction 82.541% <90% | [-0.315361, 0.411446] | 0.79022 | 1.00000 |
| A3 | **FAIL:** decode-B CV falls 0.8706 -> 0.6551, but waiting CV rises 6.0145 -> 6.8557 | [-0.693444, -0.078141] | 0.01384 | 0.05536 |
| A4 | **FAIL:** disabled arm still reports 3,093,324 local prefix queries and 52,896 hits, not zero | [-0.357637, 0.369309] | 0.97230 | 1.00000 |

Per protocol, a failed manipulation makes that mechanism's official share N/A
even if its raw efficiency contrast is nonzero. A3's uncorrected contrast is
negative, and it also misses family-wise significance after Holm correction.

## A-P5-1 arrival evidence

The four originally invalidated arms supplied the evidence for A-P5-1. Under
the superseded throughput-drift calculation, recorded base/A1/A4 span
190.55%-216.50% normalized drift, while uniform A3 is 13.23%. This gap is
retained as direct evidence that the old gate measured arrival shape in a
rate-following experiment; it is not converted into an efficiency share.

All 15 replacement/continued primary runs pass the amended cold-start gate:
25-28 warm-up completions, 12-16 completions with input at least 8192 tokens,
all compile/capture first occurrences before clean, zero clean-window
first-capture events, and zero uncovered Layer-1 graph descriptors. The 10%
drift gate was retained unchanged for the P06 saturation background arm.

## Operational findings

- The detached controller completed three cache burn-ins, four four-way waves,
  all drains, validation, budget accounting, and cleanup. The four invalidated
  first-wave arms were rerun exactly; none of their superseded measurements
  enters the ledger.
- The first exact rerun produced complete clean artifacts but hit a post-hoc
  validator `NameError` from a missing `math` import. Immutable artifacts were
  CPU-re-adjudicated under A-P5-1, the import was repaired, and no third GPU run
  was charged.
- CPU analysis repairs admitted legitimate idle fixed-time blocks as `[0,0]`,
  supported the older P3 control schema, converted one NumPy boolean at the
  JSON boundary, and explicitly separated diagnostic from reportable shares.
  These changes alter no GPU data, estimator, seed, threshold, or resample
  count.
- P10 drains range from 0.676 to 6.004 seconds. The separate P06 saturation
  background run drained naturally in 87.903 seconds, below the 600-second
  gate, with 564/564 clean completions and zero failures.
- Public Phase-5 artifacts occupy 286,205,365 bytes (304 MiB), below the 3 GB
  stop threshold; CPFS retained 1,286 GB free.
- MoE routing skew remains content-bound and receives no causal share. The
  optional routed-expert analysis-only run was not required for this primary
  ledger and was not launched.

## Launch echo

```text
LAUNCH_ECHO utc=2026-07-12T12:07:54Z host=dash0 gpus=0-3 cpus=0-79 source=/home/admin/cpfs/wjh/opprof-phase2-dash0-20260711/vllm-v0.24.0@4b253fd manifests=/home/admin/cpfs/wjh/opprof-phase5-private/manifests outputs=/home/admin/cpfs/wjh/opprof-phase3-dash0-20260712/runs/phase5 runs=3burnin+15primary+1background rate=0.4725 warmup=60s clean=240s drain=600s est_wall=35-60min est_gpu=1.7-2.1_H20h hard_cap=6.0_H20h conditional_controls=6_if_bridge_fails
```

Resume after A-P5-1 reused the completed burn-ins and compile/cudagraph caches.
The first launch through final controller completion took approximately 57.9
minutes wall time, including the adjudication/resume interval.

## Run completion stats

| Item | Count/result |
|---|---:|
| Valid primary P10 runs | **15/15** |
| Replicates | **3 per arm** |
| Valid P06 background runs | 1 |
| Completed burn-ins | 3 |
| Superseded pre-amendment arms | 4 |
| Conditional fresh controls | 0; bridge passed, P3 reused |
| Optional secondary/routed-expert runs | 0 |
| A-P5-1 primary passes | 15/15 |
| Clean failures | 0 |
| Drain quarantines | 0 |

## Mechanism ledger

Official shares follow the frozen N/A rules. Values in parentheses are raw
diagnostic ratios retained for audit, not causal/reportable shares.

| Mechanism | P03 share (95% CI) | P04 share (95% CI) | Disposition |
|---|---|---|---|
| A1 length raggedness | **0.0379 [-0.2011, 0.2344]** | N/A; unstable denominator (diagnostic 1.5933 [-6.8925, 7.0636]) | Manipulation passes; no significant recovery |
| A2 capture mismatch | N/A; manipulation failed (diagnostic 0.0281 [-0.2049, 0.2170]) | N/A; unstable denominator (diagnostic 1.1813 [-6.2871, 6.4924]) | Config did not meet 99%/90% removal gates |
| A3 arrival dynamics | N/A; manipulation failed (diagnostic -0.2251 [-0.4625, -0.0403]) | N/A; unstable denominator (diagnostic -9.4647 [-17.2450, 17.4102]) | Queue-variance gate failed; Holm p=0.05536 |
| A4 prefix structure | N/A; manipulation failed (diagnostic 0.0042 [-0.2340, 0.1948]) | N/A; unstable denominator (diagnostic 0.1760 [-6.2590, 6.4925]) | Zero-query/hit gate failed |

Shares are neither clipped nor renormalized. P04's extreme raw ratios are the
registered consequence of dividing by a near-zero, sign-unstable denominator,
not evidence of large mechanisms.

## Dominance verdicts

| Mechanism | P03 rule | P04 rule | Dual-control verdict |
|---|---|---|---|
| A1 | Fails share/CI/Holm thresholds | Not evaluable | **NOT EVALUABLE** |
| A2 | Not evaluable: manipulation failed | Not evaluable | **NOT EVALUABLE** |
| A3 | Not evaluable: manipulation failed | Not evaluable | **NOT EVALUABLE** |
| A4 | Not evaluable: manipulation failed | Not evaluable | **NOT EVALUABLE** |

No mechanism may be called dominant because the frozen rule requires both
control denominators. This is an inconclusive verdict, not evidence that every
mechanism is small.

## Residual + interaction line

The official residual is N/A under both controls because the official share
ledger is incomplete. The unchanged arithmetic over all raw diagnostic shares
is reported only for transparency:

| Control | Diagnostic share sum | Diagnostic residual + interaction (95% CI) | Status |
|---|---:|---:|---|
| P03 | -0.1549 | 1.1549 [0.5536, 1.9397] | Diagnostic only |
| P04 | -6.5142 | 7.5142 [-21.3965, 22.8767] | Diagnostic only; unstable denominator |

Nothing is forced to 100%. The residual can include interactions, unablated
MoE routing, chunked-prefill effects, double-counting, and measurement error.

## Config-tier A2 measured delta

A2 added the P3-derived sizes `{3,5,6,7}` to the full default capture list.
Its measured `delta_E` is **+0.048376**, 95% CI
`[-0.315361, 0.411446]`, or **+1.605%** relative to base. Pure-decode padding
falls from 13.7405% to 2.3989%, an 82.541% reduction. New recorded-arrival
runs expose decode-B support 1-13, so uncaptured 9-13 leave coverage at 98.879%;
both this and the sub-90% padding reduction invalidate the causal A2 share.

The config cost is reproducible: base graph capture is 11 seconds and 0.80 GiB
in all three runs; A2 is 12-14 seconds and 0.84 GiB, a 1-3 second and 0.04 GiB
increase. The measured config-tier result is therefore a small, statistically
uncertain efficiency gain with a failed preregistered removal gate.

## GPU total

New Phase-5 spend is **2.4388167443 H20-hours** of the 6.0-hour cap, including
the superseded first wave, exact reruns, continued primary matrix, burn-ins,
and background arm. **3.5611832557 H20-hours remain unspent.** Final state on
all eight H20s is 0 MiB and 0% utilization with zero compute processes.

## Sanity block

**Anomaly first:** P04's gap CI contains zero and 45.353% of its denominator
draws are nonpositive; the dual-control ledger is inconclusive and inferential
claims stop there.

| Numeric family | n | Finite | Missing | Min | Max | Distinct | Checked invariant/result |
|---|---:|---:|---:|---:|---:|---:|---|
| Primary run `E_token` | 15 | 15 | 0 | 2.623534 | 3.083263 | 15 | Positive; per-arm results not all identical |
| Clean Layer-1 steps | 15 | 15 | 0 | 11,178 | 18,400 | 15 | Non-negative; continuous/balanced per run |
| Clean duration (s) | 15 | 15 | 0 | 240 | 240 | 1 expected | Exact `3*80`; zero clean failures |
| Offered rate (req/s) | 15 | 15 | 0 | 0.470833 | 0.487500 | 2 | Every run within 5% of 0.4725 |
| Warm-up completions | 15 | 15 | 0 | 25 | 28 | 2 | Every run >=16 |
| Warm-up long completions | 15 | 15 | 0 | 12 | 16 | 3 | Every run >=1 with input >=8192 |
| Clean first-capture events | 15 | 15 | 0 | 0 | 0 | 1 expected | Zero events and zero uncovered descriptors |
| P10 drain duration (s) | 15 | 15 | 0 | 0.676181 | 6.003959 | 15 | Non-negative; all below 600 |
| Control gap `E_token` | 2 | 2 | 0 | 0.040952 | 1.721848 | 2 | P03 stable; **P04 unstable red flag** |
| Raw diagnostic shares | 8 | 8 | 0 | -9.464746 | 1.593268 | 8 | Finite; intentionally not constrained to [0,1] |
| Diagnostic residuals | 2 | 2 | 0 | 1.154931 | 7.514211 | 2 | Finite; never forced to close |
| New H20-hours | 1 | 1 | 0 | 2.438817 | 2.438817 | 1 | Non-negative; below 6.0 |

Checked invariants: source/helper/protocol provenance; identical request IDs,
content/token totals, and prompt-preserving manifests; C00-TP1 four-way
placement; detached controller ownership; exact clean windows; zero failures;
offered-rate tolerance; A-P5-1 compile/capture ordering and coverage; Layer-1
footer/sidecar balance, continuous step indices, token composition, and zero
drops; non-negative counters; drain gates; bridge decision; no public prompt or
generated-text leakage; public-disk/free-space limits; GPU hard-cap compliance;
and zero-process/zero-memory cleanup. Manipulation failures are A2, A3, and A4;
the sole machine sanity red flag is P04 denominator instability.