Record Qwen235 portability experiment gate
This commit is contained in:
@@ -0,0 +1,41 @@
|
||||
# EXP-SIMFID-Q235-V020-PORTABILITY: large FP8 MoE boundary case
|
||||
|
||||
> Status: compatibility gate prepared. No Qwen235 vLLM 0.20/deadc4a profile or
|
||||
> real/simulator latency result exists yet; historical vLLM 0.10.2 SLO data is
|
||||
> explicitly excluded.
|
||||
|
||||
## Scope
|
||||
|
||||
This is the requested Qwen3-235B-A22B-FP8 case, kept separate from the Qwen30
|
||||
surface. Its purpose is first to establish whether the current Frontier stack
|
||||
can even represent the same large-FP8-MoE runtime; it is not evidence that a
|
||||
Qwen30 profile generalizes.
|
||||
|
||||
| Item | Frozen proposal |
|
||||
|---|---|
|
||||
| model | Qwen3-235B-A22B-FP8: FP8 e4m3/dynamic-block checkpoint; BF16 compute/KV contract recorded at runtime |
|
||||
| engine | community vLLM 0.20.0 (`88d34c640…`) |
|
||||
| simulator | Frontier `deadc4a…`, with `frontier.__file__` recorded by each profiler |
|
||||
| candidate plans | TP4/EP1 and TP8/EP8, each MNS 64/128, MBT=8192 |
|
||||
| first workload | Fixed-P: ISL=2048, OSL=1, prefix off, provisional global QPS=1.6 |
|
||||
| objective | mean/p90 TTFT and E2E; TPOT=N/A |
|
||||
|
||||
TP1/TP2 do not fit the 239-GB checkpoint. TP8/EP1 is not assumed legal: only
|
||||
the two plans above have existing historical memory/topology evidence.
|
||||
|
||||
## Gates
|
||||
|
||||
1. A fresh vLLM 0.20 TP4 and TP8 server must load the checkpoint and return an
|
||||
exact 2048→1 response without FP8 fallback. This is a compatibility smoke,
|
||||
not a latency sample.
|
||||
2. New model-specific vLLM 0.20/deadc4a profiles must cover linear, router,
|
||||
FP8 MoE, prefill attention, and TP collectives for TP4/TP8. Old vLLM 0.10.2
|
||||
CUDA-event CSVs are forbidden inputs.
|
||||
3. The profile manifest binds model config/quantization, vLLM, exact Frontier
|
||||
import path/commit, hardware, and finite shape coverage. CPU replay must
|
||||
emit legal request metrics before a real surface is launched.
|
||||
4. Only then run the four-plan real/simulator Fixed-P surface with three fresh
|
||||
trials; a missing simulator cell is coverage failure, not a poor config.
|
||||
|
||||
This ordering prevents a profile-runtime mismatch from being misreported as a
|
||||
simulator tuning failure.
|
||||
Reference in New Issue
Block a user