# Frontier multi-case sufficiency: current finding

## Bottom line

The existing evidence does **not** show that Frontier already solves config
selection. The strongest Qwen30B match is an action-conditioned calibrated
result, while the zero-shot/profile-only result selects the wrong TP family.
The Qwen235B surfaces are useful follow-up cases, but decode ground truth and
Frontier's execution semantics must be repaired before a match claim is valid.

## Qwen30B: calibration is decision-bearing

Under the aligned maximum-SLO-feasible-throughput objective:

| reading | Frontier selection | worst real regret | Kendall tau-b |
|---|---|---:|---:|
| profile-only | TP4/MNS32 or TP4/MNS64 | 25.63% | 0.0000 |
| per-TP calibrated | TP2/MNS32 or TP2/MNS64 | 0.76% | 0.9668 |

The calibrated run multiplies all simulated execution times by a separately
fitted factor for each TP: TP1=0.723481, TP2=0.468089, TP4=0.352137. These
factors were fitted to real end-to-end throughput on the same model/workload
family. The large, TP-dependent residual is therefore evidence that the
profile-only simulator misses action-dependent execution behavior; it is not
evidence that the unmodified simulator predicts the action correctly.

## Qwen235B real response surfaces

### Prefill-only

- 8/8 cells are complete primary searches.
- The point-estimate top set contains all four TP8 cells, so a random cell has a
  50% top-set hit probability.
- Capacity brackets still separate all TP8 cells from all TP4 cells. Thus this
  case can test the **TP4 versus TP8** decision, but the current six probes do
  not distinguish MNS/MBT within TP8.

### Decode-only

- Only 7/8 cells are fully valid; `TP2/DP4/EP8, MNS128, MBT384` is marked
  `partial_probe_before_failure` after an engine restart/port failure.
- There are only two point-estimate score levels, and the top set contains 5/8
  cells (62.5% random hit probability).
- More importantly, all eight binary-search capacity brackets overlap the best
  observed lower bound. The current data cannot rule out any cell as optimal.

Consequently, a decode top-set hit on these results is not a match. After the
simulator selects a cell, the cheapest rigorous next step is to refine only
that cell and one competing topology/batching cell until their capacity
intervals separate or remain statistically indistinguishable.

## Frontier semantic coverage for Qwen235B

The topology itself is expressible: Frontier exposes attention TP/DP and MoE
TP/EP separately, so TP4/DP2/EP8 and TP2/DP4/EP8 need not be decomposed into
independent scheduling and execution problems.

The current execution model is not yet aligned, however:

- no checked-in H20 Qwen235B `linear_op.csv`, `attention.csv`, or `moe.csv`;
- checked-in Qwen235B config is BF16 with max position 40960, whereas the real
  run uses FP8 weights, FP8 KV, and max model length 262144;
- real prefill uses FlashAttention and internal BLADNN kernels, while Frontier's
  attention profiler exposes only FlashInfer and NO-OP backends;
- real decode starts with a dummy-filled external KV cache through
  `DecodeBenchConnector`; Frontier's trace request generator requires positive
  prefill tokens and has no equivalent initial-KV trace contract;
- real decode combines DeepEP/NVSHMEM, EAGLE3, and FULL_DECODE_ONLY CUDA graphs;
  Frontier explicitly treats speculative decode plus decode CUDA graphs as a
  conflicting/diagnostic combination.

Running dummy profiles or silently substituting FlashInfer/eager decode would
produce a number, but it would not test whether Frontier matches this system.

## Community-vLLM Qwen235B smoke

The controlled community stack is feasible: vLLM 0.10.2 loaded the
Qwen3-235B-A22B-FP8 checkpoint on TP4, allocated a BF16 KV cache, and completed
a real request. Representative FP8 linear/MoE, FlashInfer attention, and TP4
NCCL paths also executed successfully.

The smoke nevertheless found a profiler/runtime mismatch before any scheduler
model was involved. Frontier's MoE wrapper calls `get_config_dtype_str` without
`use_fp8_w8a8=True`, so it misses vLLM's tuned H20 block-FP8 Triton config and
uses a default. At TP4/EP1 with 16 tokens, a paired five-routing-seed factorial
measured:

| variant | grouped-GEMM mean | paired delta vs original |
|---|---:|---:|
| original default config + FP16 compute type | 0.3100 ms | 0.00% |
| FP8 config key only | 0.2508 ms | -19.12% |
| BF16 compute type only | 0.3088 ms | -0.40% |
| both aligned | 0.2512 ms | -18.99% |

The config-key-only routing-seed 95% interval is [-21.26%, -16.98%]; the
compute-type-only interval crosses zero. This localizes the dominant error at
this point to kernel tuning-config selection. The original MoE CSV must not be
used for a formal Frontier ranking until this path and the related shuffling
block-size lookup are aligned and re-profiled. Full evidence and hashes are in
`results/community-qwen235b-smoke/`.

## Research implication

The most interesting observation is already visible: Frontier's error is not a
single global time bias. It changes strongly with the execution action (TP in
Qwen30B), and a per-action residual can reverse the selected config. Qwen235B
prefill and decode offer held-out tests of whether those residuals are explained
by measurable execution-state features (kernel family, communication mode,
graph mode, speculative width, and KV initial state) or require case-specific
end-to-end fitting.

That is a systems question: **which execution-state transitions make operator
profile composition non-invariant across configurations, and what is the
minimum real evidence needed to recover the counterfactual ordering?**