# Community vLLM Qwen235B versus Frontier protocol v0

## Hypothesis

I believe that collecting compute and communication profiles from the same
community vLLM execution stack used for serving removes the current
internal-runtime confounder. I will verify this by freezing Frontier's
profile-only predictions before collecting the corresponding serving response
surface, then measuring config-selection regret and rank agreement.

This experiment tests Frontier inside a declared compatibility envelope. It
does not claim fidelity for the previous internal vLLM, EAGLE3, DeepEP, or
external-KV setup.

## Frozen system boundary

- Host: `dash0`, 8 NVIDIA H20 GPUs, driver `580.95.05`.
- Model: `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8`.
- Model config SHA256:
  `702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e`.
- Model index SHA256:
  `e03d4abec9611fca05844d2c8b5a08318ddc3c86acfb9481a0ef5692bf76e6d6`.
- Frontier: NetX-lab/Frontier commit
  `d9cfeb6d8791fbf2f295dd9744c56a666171776e`.
- vLLM target: exact community release `0.10.2`, installed in a new isolated
  environment with wheel/source hash recorded. Frontier declares
  `vllm>=0.10,<0.11`; the model README declares `vllm>=0.8.5`.
- Transformers is pinned to `4.55.2` and tokenizers resolves to `0.21.4`.
  vLLM 0.10.2 only declares a lower bound on Transformers, while current
  Transformers 5.x removes a tokenizer API used by this vLLM release.
- Do not use the shared `/usr/local` vLLM. Its imported version and package
  metadata disagree, so it is not a reproducible community baseline.

The profiler and serving process must import the same vLLM installation. The
following execution choices are fixed for the first validation pass:

- FlashInfer attention in both profiler and serving;
- eager execution, with CUDA graphs disabled;
- no speculative decoding;
- no external KV connector;
- no prefix reuse;
- chunked prefill enabled;
- BF16 KV cache (`auto` for this model), while the checkpoint's block-wise FP8
  weight quantization remains enabled;
- community vLLM default MoE implementation unless a backend is explicitly
  frozen and supported on both sides.

The SM90 path cannot use `CutlassBlockScaledGroupedGemm` (the vLLM 0.10.2
implementation gates that path on SM100), so this system uses Triton block-FP8
MoE. However, the two sides do not yet select the same Triton tuning config.
Serving resolves the checked-in H20 config named with
`dtype=fp8_w8a8`, while Frontier's standalone MoE profiler requests a filename
without that FP8 dtype component and falls back to a default config. Formal
comparison is gated on repairing this profiler/runtime tuning-config mismatch
and re-profiling; recording the same vLLM package version is not sufficient.

These controls isolate operator composition and scheduling. CUDA graphs,
FlashAttention, speculative decoding, external KV, and optimized expert
communication become separate stress cases after the controlled pass.

## Case P: prefill-only

Reuse the original trace window and SLO because the community model's 40,960
token limit covers the filtered input plus the one-token completion.

- Window: `thinking_w20260327_1000`.
- Input filter: 0--32,768 tokens.
- Output override: 1 token.
- Replay scale: 1.0.
- SLO pass rate: at least 0.95.
- TTFT SLO: 1 s for input <=8,191; 2 s otherwise.
- Objective: maximum SLO-feasible offered requests/s/GPU.
- Candidate grid: TP `{4,8}` x MNS `{64,128}` x MBT `{8192,16384}`.
- DP=1 and expert parallel disabled.

This case can test the TP4-versus-TP8 decision and batching effects. It must not
reuse performance values from the internal 256k model as ground truth.

## Case D: decode-dominant

Do not initially reproduce the previous strict decode-only case. It depends on
`DecodeBenchConnector`, EAGLE3, FP8 KV, DeepEP/NVSHMEM, and decode CUDA graphs,
which are outside the controlled Frontier profile contract.

Construct a community-only decode-dominant case that both systems can express:

- same trace window and timestamp/sampling fields;
- input filter: 1--512 tokens;
- output override: 512 tokens with EOS ignored;
- SLO pass rate: at least 0.95;
- TPOT SLO: 40 ms;
- objective: maximum SLO-feasible offered requests/s/GPU;
- topology grid: `(TP=4, DP=2, EP=8)` and `(TP=2, DP=4, EP=8)`;
- batching grid: MNS `{64,128}` x MBT `{256,384}`.

This is deliberately named decode-dominant, not decode-only. A strict
decode-only claim requires an initial-KV state contract in Frontier.

## Required profile closure

Profile data are measurement inputs, not end-to-end calibration. No serving
throughput or latency from either case may scale the profiles.

1. Linear/operator profiles for the TP degrees consumed by the two grids.
2. FlashInfer attention profiles covering the observed prefill/decode batch,
   context-length, and chunk-size ranges.
3. FP8 MoE profiles for the actually consumed parallel pairs:
   `(MoE TP=4, EP=1)`, `(MoE TP=8, EP=1)`, and `(MoE TP=1, EP=8)`.
4. H20 intra-node collective profiles for TP all-reduce at world sizes 2, 4,
   and 8 and the EP8 all-to-all path.

Frontier already provides an H20 device description but no checked-in H20
network profiles. Its public collective profiler covers all-reduce and
send/recv, not all-to-all. Therefore Case P may proceed after H20 all-reduce
closure; Case D remains blocked until EP8 all-to-all is either measured and
consumed or the selected communication model is independently validated
against those measurements.

## Blind run order

1. Build the isolated environment and record package/binary hashes.
2. Run one-row compute and collective smokes, then one TP4 server-load/request
   smoke. A smoke failure stops the campaign.
3. Collect profiles and validate CSV metadata/coverage.
4. Run Frontier for every candidate and offered-load anchor.
5. Freeze simulator outputs and their SHA256 checksums.
6. Only then collect community-vLLM serving ground truth. Randomize the first
   trial order and reverse it for the second trial.
7. Refine only decision-relevant capacity intervals that still overlap.

## Metrics and decision rule

Report per case:

- absolute simulated and real TTFT/TPOT/throughput values;
- anchor-level SLO feasibility confusion;
- selected-config real regret, including interval-robust regret;
- Kendall tau-b with ties preserved;
- informative-pair direction accuracy;
- top-set hit and random top-set hit probability;
- profile and real-GPU measurement cost.

Frontier is sufficient as a config ranker for this controlled family only if
every completed case has:

- worst selected-config real regret <=5%;
- Kendall tau-b >=0.8 on enough informative pairs;
- no unresolved ground-truth interval capable of reversing the decision;
- no per-case or per-action end-to-end calibration.

If the controlled pass succeeds, enable one omitted mechanism at a time. If it
fails, use operator/communication/stage residuals to localize which composition
assumption reverses the ranking before proposing a new tuner mechanism.

## Launch gates and initial cost cap

The first authorized GPU action should be smoke-only:

- isolated vLLM import and Qwen235B TP4 load;
- one request through the community server;
- one representative FP8 linear, attention, MoE, and all-reduce profile point;
- expected wall time: 20--40 minutes;
- hard GPU budget: 2 H20-GPU-hours.

No full response-surface sweep is authorized by this protocol. Its cost and
anchor count must be resolved from the smoke timings and echoed separately.
