# Active intervention v0: held-out trace-13 result

Date: 2026-07-15 (Asia/Singapore)

Decision: **close the passive-telemetry treatment-effect route**.  The held-out
campaign produced no telemetry-induced action change, measurement reduction, or
GPU-cost reduction.  It did show that the engine state contained the correct
action-specific mechanism; the current feature model failed to use it.

## Headline result

The outcome-only and telemetry policies both measured the source for 300
seconds, selected `joint=(MNS64,MBBT8192)`, and produced the same complete
acquisition order.  Both reached the exact finite-surface oracle after the
first intervention at a reconstructed all-in lower-bound cost of 2.4284
H20-hours.  Telemetry GPU-cost reduction was therefore exactly 0%, below the
10% confirmation trigger and 20% contribution gate.  No actual early-stop
confirmation was launched.

The complete annotation campaign cost 5.0379 H20-hours, below the 6.0 H20-hour
hard cap.  It ran 12 uncensored real-GPU outcomes: four configs, three disjoint
request partitions, and a fresh server per config.

## Exact response surface

Median normalized SLO-goodput was:

| Config | Rep values | Median |
|---|---|---:|
| `MNS32,MBBT4096` source | 0.40091 / 0.39788 / 0.42061 | 0.40091 |
| `MNS64,MBBT4096` | 1.00000 / 0.99970 / 1.00000 | 1.00000 |
| `MNS32,MBBT8192` | 0.44394 / 0.41515 / 0.42606 | 0.42606 |
| `MNS64,MBBT8192` joint | 1.00000 / 1.00000 / 1.00000 | 1.00000 |

Increasing MNS alone was sufficient and joint was redundant.  Increasing MBBT
alone improved the median by only 0.02515, versus 0.59909 for MNS.  This is a
strong non-additive action response, not a setting where independently tuning
the knobs and merging their improvements is valid.

## What the telemetry actually said

Across 41,086 source scheduler records, 93.12% of steps had waiting work,
85.36% were MNS-exclusive binding, 1.11% were MBBT-exclusive, mean running-slot
utilization was 97.39%, mean token-budget utilization was 15.69%, mean KV usage
was 2.75%, and there were no preemptions.

The intervention transition agreed with that state:

| Config | Waiting | MNS-exclusive | MBBT-exclusive | Median goodput |
|---|---:|---:|---:|---:|
| source | 93.12% | 85.36% | 1.11% | 0.40091 |
| MNS only | 5.38% | 0% | 5.38% | 1.00000 |
| MBBT only | 91.19% | 91.09% | 0.04% | 0.42606 |
| joint | 0.89% | 0% | 0.89% | 1.00000 |

Thus this experiment does **not** support the claim that engine telemetry lacks
tuning information.  It rejects the narrower claim that adding passive state
summaries to the current small-data ridge policy converts that information into
lower tuning cost.

## Why the learned policy failed

At 300 seconds, the telemetry model predicted joint, MNS, and MBBT effects of
0.35190, 0.26118, and 0.09686.  The actual median effects were 0.59909,
0.59909, and 0.02515.  Telemetry therefore made the nonexistent joint-over-MNS
gap larger: 0.09072 predicted versus 0 actual; the outcome-only model predicted
0.03188.

The failure has three concrete causes:

1. The six training decisions contain no joint intervention.  The
   `delta_product` feature has no support, so joint ranking is extrapolation.
2. Passive raw summaries do not represent the counterfactual scheduler work
   unlocked by each action.  Capacity-normalized MNS pressure was visible, but
   the model was not structurally required to map it to MNS marginal value.
3. The policy maximizes predicted effect.  It does not identify the smallest
   epsilon-optimal intervention or price unsupported action complexity.

## Research implication

Do not retain the harness or the passive telemetry model as a contribution.
The next defensible route is engine-native, action-conditional counterfactual
instrumentation: at a real scheduling state, shadow-replay the exact scheduler
decision under an MNS relaxation, MBBT relaxation, and their joint relaxation,
then expose the incremental queued work admitted by each action.  Real paired
interventions calibrate how those one-step shadow effects map to E2E SLO
goodput.  This is distinct from a hand-written cap-to-knob rule and from a
full-system simulator: it reuses the exact live queue, scheduler, and cache
state while simulating only the local decision boundary.

That route should be evaluated against outcome-only search, the present passive
telemetry model, a cap-hit expert rule, and a full simulator.  The paper-level
gate remains at least 20% measured H20-hour reduction to a 2%-oracle config on
task-held-out workloads with at most 2% regret.

## Sanity

Surface outcomes: n=12, min=0.39788, max=1.0, distinct=8.  Session costs: n=4,
min=1.1702, max=1.3566 H20-hours, distinct=4.  Scheduler-record counts: n=4,
min=37,001, max=41,348, distinct=4.  All counters and costs were non-negative;
all ratios were in `[0,1]`; request hashes matched; all 12 runs were uncensored;
the controller and four sessions completed; and config outcomes were not all
identical.  No red flags were found.

Machine-readable summary: `runs/active-intervention-v0/trace13-results.json`.
Raw immutable root: `/home/admin/cpfs/wjh/active-intervention-prospective-20260715`.
