9.5 KiB
Static-policy oracle-gap results
Status: FINAL — REFUTED WITHIN THE FROZEN TP1 MNS/MBT POLICY SPACE.
Date: 2026-07-13. The registered experiment completed 104 valid trials on
dash0 and returned REFUTED. Even a phase-perfect oracle with zero detection,
switching, and state-transfer cost is bounded below the registered 10%
SLO-goodput contribution gate.
The machine result is
runs/opprof-oracle-gap/metrics.json (SHA-256
250ba4c1657a8830795ee06392eea4e21c62d958fea11701ba60581ef0266543).
All 104 trial-level measurements are in
runs/opprof-oracle-gap/trials.csv (SHA-256
60cd901f18bbf107eb130f0095e867c3b6d47a78a42e83bcbe57fecf23cc5f9c).
The final resumable controller state is
runs/opprof-oracle-gap/controller-state.json (SHA-256
27ec5e9a3cd32a871d583ba8eb6d7d3fe2a338a8fd42369233ada57cd4da6436).
Decision
The tested motivation was:
One static batching policy leaves at least 10% end-to-end SLO-goodput on the table across temporally heterogeneous phases, and a phase-aware runtime can recover it by switching MNS/MBT policies.
The registered 10,001-point mixture scan finds a worst conservative gap of
8.333219%, at P01 time weight 0.0244. Equal phase time gives
7.260726%. An independent implementation that evaluates every exact
static-policy crossover gives a slightly more conservative exact maximum of
8.333333% = 1/12, at P01 weight 1/41. This leaves 1.666667 percentage
points below the 10% gate.
The negative result is stronger than a failed online prototype. The oracle already knows the current phase and pays no switching cost. A realizable controller over the same four policies cannot exceed this oracle bound.
This is a scoped refutation, not a universal claim about adaptive serving. It rules out the current contribution based on phase-aware selection among the four MNS/MBT configurations for the frozen P01/P06 pair, Qwen3-30B-A3B, TP1, H20, vLLM 0.24, and the registered SLO. It does not rule out a new scheduler, KV/cache policy, routing-aware mechanism, topology change, or a different workload family.
Fixed setup
| Item | Value |
|---|---|
| Placement | dash0, serialized on physical GPU0; GPUs 1-7 idle |
| GPU | NVIDIA H20, driver 580.95.05 |
| Model | Qwen3-30B-A3B, BF16 |
| Runtime | vLLM 0.24.1.dev3+g668cfb7e2, source 4b253fd8619764b6971a7f2e3a3aa7545f6ace05 |
| Topology | TP1; one server and one client |
| Fixed mechanisms | chunked prefill on; prefix caching on |
| P01 | input U[128,512], exactly 64 output tokens, deterministic steady arrivals |
| P06 | 50/50 input U[128,512] / U[4096,8192], exactly 512 output tokens, deterministic bursts of eight |
| P01 timeline | 60 s warm-up + 60 s clean measurement |
| P06 timeline | 60 s warm-up + 120 s clean measurement |
| SLO feasibility | at least 95% of clean-admitted requests pass both TTFT and TPOT |
| TTFT SLO | <=2 s for input <=4096; <=4 s for input <=32768; otherwise <=6 s |
| TPOT SLO | <=50 ms/token |
The four policies were:
| Config | MNS | MBT |
|---|---|---|
| C00 | 1024 | 8192 |
| C10 | 64 | 8192 |
| C01 | 1024 | 2048 |
| C11 | 64 | 2048 |
Final SLO frontiers
L is the highest majority-feasible offered rate. U is the next higher
majority-infeasible rate. Both sides below have at least three trials, and all
accepted rate sequences are monotone.
| Config | P01 L |
P01 U |
P06 L |
P06 U |
|---|---|---|---|---|
| C00 | 28.0 | 30.0 | 2.3 | 2.4 |
| C10 | 24.0 | 26.0 | 2.4 | 2.5 |
| C01 | 28.0 | 30.0 | 2.2 | 2.3 |
| C11 | 24.0 | 26.0 | 2.2 | 2.3 |
The optimistic oracle upper bound chooses C00/C01 at U=30 for P01 and C10
at U=2.5 for P06. The static lower bound chooses C10 below P01 weight 1/41
and C00 above it; they tie at the exact worst point.
| Mixture | Oracle upper | Best-static lower | Conservative gap |
|---|---|---|---|
| Pure P06 | 2.500000 | 2.400000 | 4.166667% |
Exact worst, P01 weight 1/41 |
3.170732 | 2.926829 | 8.333333% |
| Equal phase time | 16.250000 | 15.150000 | 7.260726% |
| Pure P01 | 30.000000 | 28.000000 | 7.142857% |
The specialization exists but is too small. Reducing MNS from 1024 to 64
raises the conservative P06 lower bound from 2.3 to 2.4 req/s, while lowering
P01 from 28 to 24 req/s. Even after using infeasible U values for the oracle
and feasible L values for the static baseline, the best possible phase-aware
selection cannot reach 10%.
Robustness finding
The strongest system finding is not config specialization but a repeat-level
mode flip at P01/26 rps. Both C10 and C11 show the identical verdict sequence
rep0=0% pass, rep1=100% pass, rep2=0% pass. For C10, TTFT p50 is
3864.62, 252.71, and 3994.88 ms; for C11 it is 6533.62, 593.44, and
5812.71 ms. TPOT remains below the 50 ms SLO in these trials.
These trials are measurement-valid: exact outputs, offered-rate tolerance, schedule lag, timestamps, and client invariants all pass. The majority rule therefore classifies 26 rps as infeasible for both configs. The result does not change the oracle-gap decision because neither config supplies the P01 oracle maximum or the relevant best-static P01 lower bound.
Token-domain seed and server execution history change together across repetitions, so this experiment cannot attribute the flip to MoE routing, cache/compiler state, or another source. It does motivate a narrower factorial study that crosses token seed with fresh/reused server state and randomizes order, while recording routed-expert and per-step scheduler telemetry. That is a mechanism question; it should not be presented as evidence for a phase-aware MNS/MBT controller.
Execution and audit history
The 104 scored trials comprise 52 base-grid primaries, 18 registered upward or
downward extensions, 32 boundary confirmations, and two boundary-closure
trials. The final closure moved C00/P06 from the provisional [2.4,2.5) to
[2.3,2.4) and repeated 2.3 rps to 3/3 feasible.
Four frozen amendments are recorded in the protocol:
- A-OG-1 extended only the P06 upward anchors after C00 remained feasible through the original 2.3-rps limit.
- A-OG-2 quarantined one P06/C01/2.2 local
ClientOSErrorattempt and retried the identical logical trial on a fresh server. - A-OG-3 generalized the same pre-score transport rule after one P06/C10/1.9 attempt showed the identical signature.
- A-OG-4, frozen before any confirmation score, added closure for a majority-shifted boundary without adding new anchors.
The two transport-invalid attempts and one pre-A-OG-4 interrupted attempt
created no score and do not enter any metric. Their retained tree hashes are,
respectively,
434863ba90513cbc54534ffbc1a13c980b3ef7d567190a0aa3f97b55650acbb2,
a57b1ac5f090680bb70c16b5d709eb2b8ac47dce57c7a03b8077cd9b6d80d831,
and bc3feb53514601e641ef1db204c74cffe3d282b24ad797e5b161468f5d15de5c.
Ten deliberately overloaded primary anchors exceed both the 1 s schedule-lag gate and the 5% offered-rate tolerance. They are correctly classified as infeasible and are not used as final boundary points. All 48 final boundary trials pass both client-side gates; their maximum schedule lag is 573.59 ms.
The final experiment fingerprint uses repository commit 16177b0, analyzer
SHA-256
d86ecb1f077472906cbb729bd2c9d4b3a82ac6dfdc90838a17ae300d0767110d,
controller SHA-256
f7c5c2f74f2002f1e4b097e608d165a3a2e9374fbf07935a4d6d6a7c5d45a83a,
and protocol SHA-256
173f969a4428643cf6c4b950413aa82bef25cafd163e20d7944370a5d87af435.
The archived launch log SHA-256 is
54e2f4804a5efee072b670043c4092cf41f1a27664faa05ea71bfc1412c3e9db.
GPU accounting and cleanup
The campaign used 4.8877033845 H20-hours, below the 6.0-hour cap, across 5 h 41 min 51 s wall time including amendment, audit, restart, and server startup intervals. At completion, all eight H20s report 0 MiB, 0% utilization, and zero compute processes.
Sanity block
There are no data-sanity red flags. The P01/26 all-or-none mode flip is a scientific robustness finding, not an invalid trial signature; it repeats in two configs and all measurement invariants pass.
| Numeric family | n | Min | Max | Distinct | Checked invariant/result |
|---|---|---|---|---|---|
| Score-row indicator | 104 | 1 | 1 | 1 expected | 104 score files; no overwrite |
| Target rate (req/s) | 104 | 1.4 | 36.0 | 19 | Non-negative; fixed grid/extensions only |
| Clean cohort per trial | 104 | 168 | 2160 | 30 | Non-empty and non-negative |
| Pass rate | 104 | 0.0 | 1.0 | 57 | All ratios in [0,1] |
| SLO-goodput (req/s) | 104 | 0.0 | 27.816667 | 59 | Non-negative; per-cell results not all identical |
| Boundary pass rate | 48 | 0.0 | 1.0 | 33 | Both final sides have three trials |
| Boundary max schedule lag (ms) | 48 | 2.145861 | 573.593768 | 48 | All below the 1000 ms gate |
| Exact-output indicator | 85,402 | 1 | 1 | 1 expected | Every clean request produced the requested token count |
| Frontier cells | 8 | 8 | 8 | 1 expected | 8/8 bracketed and monotone |
| Registered weight scan | 10,001 | 0.0 | 1.0 | 10,001 | Continuous 0.0001 increments, endpoints included |
| Campaign H20-hours | 1 | 4.887703 | 4.887703 | 1 | Non-negative and below 6.0 |
| Final GPU memory/utilization | 8 | 0 MiB / 0% | 0 MiB / 0% | 1 expected | Zero compute processes |
Checked invariants: fixed model, runtime, manifests, SLOs, config values, seeds, and serialized placement; exact output work; nondecreasing timestamps; non-negative counters and latencies; pass ratios in range; offered-rate and schedule gates; majority-of-three final boundaries; monotone accepted frontiers; complete 10,001-point scan; independent exact-crossover recomputation; transport-attempt quarantine; GPU hard-cap compliance; and complete GPU cleanup.