# Active intervention + measurement v0 protocol

Date: 2026-07-15 (Asia/Singapore)

Status: **FROZEN BEFORE THE `chat_w20260313_1000` GPU RUN**.

## Research question

This experiment asks whether a tuner conditioned on direct engine-state
trajectories can choose both a measurement horizon and a coupled configuration
intervention with lower real-GPU cost than the same tuner using only external
prefix outcomes.

The contribution is not the controller, legality checks, telemetry collection,
or the ridge model.  The route remains open only if engine state changes an
actual decision and reduces cost-to-near-oracle on unseen workloads.

## Development result that motivates, but does not pass, the route

The frozen trace-12 dataset contains 72 examples: six source decisions, four
measurement checkpoints, and `noop/MNS/MBBT` actions.  Features are direct
continuous Layer-1 state summaries; cap-exclusive and bottleneck labels are
excluded.  Leave-one-repetition-out sequential replay uses the same model,
candidate set, confidence rule, and checkpoint set for both modes.

The external-outcome policy and telemetry policy both put all six decisions
within 2% regret.  Outcome-only selected a mean 262.5-second source measurement
and cost 3.750 replay H20-hours across the six replayed decisions; telemetry
selected 275 seconds and cost 3.833 H20-hours.  Telemetry therefore increased
the replay lower-bound cost by 2.22%, with no regret reduction.  This is a
negative result.  It does not settle the question because the dataset has only
two source regimes, one source is at the offered ceiling, and there is no joint
MNS+MBBT action.

Sanity: n=6 decisions; regret min=0, max=0.009412, distinct=3; source cutoff
min=150s, max=300s, distinct=3 across the two policies; all costs are
non-negative, regrets are in `[0,1]`, target results are not all identical, and
the six decisions are complete exact-workload pairs.

## Frozen prospective setup

- Host: `dash0`, 8 NVIDIA H20 GPUs available; each TP4 server runs alone on
  GPUs 0-3.  Co-location is prohibited for SLO verdicts.
- Engine: patched vLLM `0.24.1.dev3+g668cfb7e2` from clean source commit
  `4b253fd8619764b6971a7f2e3a3aa7545f6ace05` at
  `/home/admin/cpfs/wjh/opprof-phase2-dash0-20260711/vllm-v0.24.0`, using
  `/tmp/wjh-opprof-phase2-dash0-20260711/.venv`.
- Model: `/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B`, BF16.
- Workload: unseen `chat_w20260313_1000`; input 0-8192; output exactly 128;
  replay scale 0.5; 300-second arrival window.
- Three disjoint repetitions: source rows are assigned by a deterministic
  SHA-256 modulo-3 partition before input filtering.  Each repetition selects
  approximately 3300 requests, or 2.75 requests/s/GPU at TP4.
- SLO: at least 95% pass; stepped TTFT 2/4/6 seconds; TPOT at most 50 ms.
- Checkpoints: 75, 150, 225, and 300 seconds.
- Full 2x2 surface:
  - source: `MNS=32, MBBT=4096`;
  - MNS action: `64,4096`;
  - MBBT action: `32,8192`;
  - joint action: `64,8192`;
  - `noop` retains the source.
- Four config sessions are serialized.  Each session uses a fresh server,
  warm-up, burn-in, and counter-rotated repetition order.
- Expected campaign cost: 4.6-5.5 H20-hours; hard cap: 6.0 H20-hours;
  expected wall time: 75-100 minutes.

The source is executed first.  The frozen telemetry policy selects the next
real config session; all remaining cells are then measured only to construct
the exact finite-surface oracle.  Oracle annotation after the selected action
is reported separately from tuner cost.

## Frozen policies

Both policies fit the paired treatment effect

```text
target normalized SLO-goodput - source normalized SLO-goodput
```

from source config, full config delta, offered load, and external prefix
outcomes.  The telemetry policy additionally receives fixed direct Layer-1
summaries and their interactions with `delta_log2(MNS)` and
`delta_log2(MBBT)`.  It does not receive a bottleneck label or a
diagnosis-to-knob rule.

At each checkpoint, jackknife models produce an effect distribution for
`noop`, MNS, MBBT, and joint actions.  Measurement stops at the earliest second
consecutive checkpoint with the same confident best action; otherwise it uses
the full 300 seconds.  Confidence requires a predicted margin of at least 0.02
and the best lower bound to exceed the second-best upper bound.  If the final
choice is not confident, the next run is the positive-UCB action, explicitly
marked as a diagnostic intervention.  The exact same rule is used for the
outcome-only baseline.

## Hypotheses and gates

### H1: action value

Engine state must change the selected intervention or its ranking and reduce
real action regret.  Prediction error or bottleneck-label accuracy is not a
success metric.

### H2: measurement value

Engine state must select a shorter stable source measurement without increasing
action regret.  A shorter reconstructed prefix is only a trigger; it is not an
actual GPU-cost claim until an early-terminated confirmation run measures
startup, warm-up, drain, and cleanup.

### H3: end-to-end cost

Primary development metric is H20-hours to first reach a configuration within
2% of the exact median-goodput oracle.  The outcome-only and telemetry policies
use the same measured config costs and differ only in source information.

- At least 10% prospective replay cost reduction, telemetry regret at most 2%,
  and no outcome-only-to-telemetry harm triggers an actual early-stop
  confirmation.
- At least 20% measured all-in H20-hour reduction is required for a contribution
  claim.  This one task can only establish development feasibility; a paper
  claim additionally requires task-held-out replication.
- Source median normalized goodput at or above 0.98 stops the surface before
  target runs because the workload has no material improvement headroom.
- Any hash mismatch, missing/censored result, telemetry drop, non-monotonic
  phase, negative cost, ratio outside `[0,1]`, or all-identical config outcomes
  is a red flag and stops analysis.

If the 10% trigger fails, this route is closed for the current engine-state
representation.  The experimental control plane is not retained as a fallback
research contribution.

Pre-run provenance amendment: the first controller dry-run on 2026-07-15
rejected two stale engine paths before starting a server.  The paths and exact
runtime version above were recovered from the accepted trace-12 campaign and
corrected before any trace-13 GPU work.  No scientific treatment or gate was
changed.
