# Frontier multi-case sufficiency protocol v0

## Claim under test

Can Frontier select a low-regret configuration for the frozen candidate grid
under the objective **maximum SLO-feasible offered request throughput per GPU**?

The allowed conclusion is scoped to the tested model/runtime/hardware/workload
family. Three matches cannot establish universal simulator fidelity.

## Cases

1. Qwen3-30B-A3B mixed chat, H20, TP x MNS grid (existing data).
2. Qwen3-235B-A22B prefill-only, H20, TP x MNS x MBT grid.
3. Qwen3-235B-A22B decode-only, H20, TP/DP/EP x MNS x MBT grid.

For Qwen235B, keep the real trace window, request filtering, SLO, model build,
runtime flags, and candidate cells recorded by the original trials. Frontier is
evaluated at the same offered-load anchors; no capacity extrapolation beyond
the common anchors is allowed.

## Two separate simulator readings

- **Zero-shot/profile-only:** operator profiles and documented hardware/model
  inputs are allowed; no end-to-end measurement from the evaluated workload is
  used to scale Frontier.
- **Calibrated:** every real measurement used to fit a scalar or residual is
  charged and reported. Calibration is fitted on a declared train fixture and
  evaluated on a disjoint workload/config holdout.

These readings must never be merged. A per-TP scale fitted from the same
workload does not count as zero-shot simulator accuracy.

## Ground-truth gate

Before a match decision:

- every result must be a completed primary search without probe failure;
- capacity is an interval from the largest feasible anchor to the next
  infeasible anchor, not merely the feasible lower bound;
- refine probes when the possibly-optimal set induced by those intervals is too
  broad to distinguish the simulator's selected cell;
- record the random top-set hit rate and the number of informative (non-tied)
  pairs.

## Primary metrics and predeclared decision rule

Per case, report:

1. selected-config real regret (point estimate and interval-robust bound);
2. Kendall tau-b and exact pair-direction accuracy with real ties preserved;
3. top-set hit and its random-hit baseline;
4. anchor-level SLO feasibility confusion;
5. profile/calibration cost and all right-censored cells.

The profile-only simulator is considered sufficient as a **config ranker for
the tested family** only if every case has:

- worst selected-config real regret <= 5%;
- Kendall tau-b >= 0.8 on a response surface with enough informative pairs;
- no unresolved ground-truth interval that can reverse the selected decision;
- no per-case or per-action end-to-end calibration.

SLO-oracle sufficiency is a stronger claim and additionally requires low false
feasible/false infeasible rates; ranker success alone does not establish it.

## Current representational gaps to disclose

- The checked-in Frontier Qwen235B model config is BF16 and has no H20
  Qwen235B operator profiles, while the real runtime uses FP8 weights and FP8 KV.
- Prefill uses internal vLLM/BLADNN paths and PIECEWISE CUDA graphs.
- Decode uses TP/DP with EP8, DeepEP/NVSHMEM, EAGLE3, DecodeBenchConnector, and
  FULL_DECODE_ONLY CUDA graphs.
- Frontier currently rejects the faithful combination of speculative decoding
  and decode CUDA-graph modeling unless using a diagnostic opt-in; its public
  examples disable decode CUDA graphs for speculative decoding.

Therefore an as-is Frontier run and an upgraded/semantically aligned run, if
implemented, must be reported separately.
