aituner/docs/action-aware-constraint-pilot-v0-protocol-20260714.md

# Action-aware constraint pilot v0 protocol

Status: **FROZEN BEFORE NEW GPU RUNS**.

Date: 2026-07-14 (Asia/Singapore).

## Headline question

Can telemetry from one complete initial-config benchmark identify which of two
competing knob families should be changed, before either target configuration
is evaluated?

This pilot tests a narrow prerequisite, not an end-to-end tuner claim.  It
uses fields already present in the per-step OpProf stream to reconstruct exact
zero-slack conditions for `max_num_seqs` (MNS) and
`max_num_batched_tokens` (MBBT).  No new vLLM instrumentation is justified
unless those action-conditioned conditions predict crossed real-system
intervention responses.

## Hypothesis

I believe config-normalized scheduler constraints provide a stronger tuning
signal than an aggregate queue symptom because the same waiting queue can be
blocked by different admission limits.

I will verify it by holding model, hardware, TP, request bands, arrival times,
and offered load fixed while constructing two source configurations with
different binding constraints.  From each source run alone, the larger
exclusive binding fraction predicts the action family.  Both candidate
actions are then measured on the same requests for the full 300-second replay.

## Frozen platform and workload

- Host: `dash0`, solo placement on GPUs 0-3, four NVIDIA H20 GPUs.
- Model: Qwen3-30B-A3B BF16.
- Engine: patched vLLM `0.24.1.dev3+opprof`, TP=4.
- Workload: the three disjoint `mid` bands from
  `chat_w20260312_1000`, 2.125 requests/s/GPU, 300-second arrival window,
  exactly 128 output tokens.
- SLO: the unchanged study TTFT/TPOT thresholds and 0.95 pass-rate target.
- Every config starts one fresh server, performs the accepted 16-request
  warm-up and the existing burn-in, then runs all three disjoint measured
  bands in its frozen order.
- SLO early stopping is disabled.  A measured run must drain all selected
  requests and finish within the 450-second client deadline.

## Frozen configuration and action matrix

| ID | MNS | MBBT | Role |
|---|---:|---:|---|
| `b_base` | 64 | 256 | token-budget-bound source; operational gate runs first |
| `a_base` | 16 | 8192 | MNS-bound source |
| `shared` | 64 | 8192 | MNS action from A; MBBT action from B |
| `b_mns` | 128 | 256 | competing MNS action from B |
| `a_mbbt` | 16 | 16384 | competing MBBT action from A |

The two decisions are therefore:

```text
Regime A: a_base -> {shared (increase MNS), a_mbbt (increase MBBT)}
Regime B: b_base -> {b_mns (increase MNS), shared (increase MBBT)}
```

The candidate magnitudes are intentionally large in this feasibility pilot so
that a missing crossed response is not explained by an imperceptibly small
intervention.  This does not establish that these are production step sizes.

Frozen config order is `b_base`, `a_base`, `shared`, `b_mns`, `a_mbbt`.
Frozen repetition orders are respectively `123`, `231`, `312`, `132`, and
`213`, reducing band/time alignment without reusing a server across configs.

## Pre-action signal

For each source run, let `waiting` include the normal and deferred waiting
queues, and let `scheduled_tokens = prefill_tokens + decode_tokens`.

```text
mns_exclusive = waiting > 0
                and running == configured MNS
                and scheduled_tokens < configured MBBT

mbbt_exclusive = waiting > 0
                 and scheduled_tokens == configured MBBT
                 and running < configured MNS

both = waiting > 0
       and running == configured MNS
       and scheduled_tokens == configured MBBT
```

Each score is the fraction of all scheduler records in the measured interval
that satisfies the condition.  The predicted action is the family with the
larger exclusive fraction.  This uses no target telemetry or target outcome.
KV usage and preemptions are reported as possible alternative constraints but
are not silently reassigned to either score.

These conditions reproduce two scheduler loop boundaries, but they are still
a Level-0 proxy: they do not expose the exact request rejected at the boundary
or run a shadow schedule.  The pilot explicitly tests whether that additional
engine patch is warranted.

## Outcomes and baselines

Primary intervention outcome:

```text
SLO-goodput = full-run SLO pass count / 300-second arrival window
```

Also report pass rate, TTFT p50/p95/p99, TPOT p50/p95/p99, drain elapsed time,
KV usage, preemptions, queue area, and CUDA-graph padding.

Required decision baselines:

1. always choose the MNS family;
2. always choose the MBBT family;
3. queue-pressure-only, which has no candidate-specific score and therefore
   must use one frozen family for both regimes;
4. the pre-action exclusive-binding prediction.

This is a mechanism ablation.  It does not compare against a trained black-box
tuner because two regimes are not a valid training surface.

## Gates and failure meanings

Data validity requires 15 uncensored measured runs, exact request/arrival/input
hashes across each repetition, full request accounting, one continuous OpProf
stream per config, zero dropped records, monotonic timestamps and step indices,
nonnegative counters, bounded ratios, clean GPU placement, and config values in
the result matching the server command.

The crossed-response gate passes only if, in all three repetitions:

- the MNS target has higher SLO-goodput than the MBBT target in Regime A;
- the MBBT target has higher SLO-goodput than the MNS target in Regime B;
- each winning target exceeds its competing target by at least 10% of the
  source SLO-goodput.  A source with zero goodput makes the run invalid for
  this relative gate rather than changing the denominator.

The binding gate passes only if, in both regimes:

- the predicted family matches the measured winning family in all three
  repetitions;
- the median winning-family exclusive fraction is at least 0.10;
- it is at least 5x the median competing-family exclusive fraction;
- the direction is unchanged under cumulative 25%, 50%, 75%, and 100%
  checkpoints after the 25% checkpoint.

Decision meanings:

- `STOP_WORKLOAD_NOT_CROSSED`: candidate outcomes do not have different
  winners; the experiment cannot test action selection.
- `STOP_BINDING_NOT_PREDICTIVE`: outcomes cross but source-only constraint
  scores do not select them; do not implement shadow scheduling from this
  hypothesis.
- `STOP_NO_NEW_INSTRUMENTATION_NEEDED`: the signal works but every required
  field was already present; keep it as an analysis/tuner feature and do not
  claim a new engine-instrumentation contribution.
- `OPEN_EXACT_ATTRIBUTION_ABLATION`: the signal works but unresolved/both/KV
  cases are material enough that exact rejection reasons could change a
  decision.  Only this result authorizes a minimal vLLM attribution patch.

Ambiguity is material only when, in either regime, the median
`both + waiting_unresolved` fraction is at least the median absolute gap
between the two exclusive fractions, or when any source run records a
preemption or median source KV maximum is at least 0.90.  Otherwise all fields
needed for the observed decision were already present and the result is
`STOP_NO_NEW_INSTRUMENTATION_NEEDED`.

No result from this development pilot is a paper-level E2E tuning claim.

## Cost and stopping discipline

- Hard cap: 8.0 H20-hours, including failed sessions.
- Expected: 6.0-7.2 H20-hours and 90-110 minutes wall time.
- `b_base` runs first.  If its first measured band cannot drain by 450 seconds,
  the controller stops before any comparative analysis; MBBT=256 is then an
  operationally invalid source, not negative evidence.
- Any data red flag stops analysis before computing a tuning conclusion.