Add Qwen30 prefill fidelity experiment
This commit is contained in:
54
runs/frontier-phase-factorial-v0/experiment-card.md
Normal file
54
runs/frontier-phase-factorial-v0/experiment-card.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# 实验 EXP-SIMFID-PHASE-FACTORIAL：prefill-only 是否是 simulator ranking 的容易区间？
|
||||
|
||||
> **状态：** 已批准，运行中（用户于 2026-07-17 明确要求先完成 235B mixed 与 30B prefill-only）
|
||||
|
||||
## Claim 与决策
|
||||
|
||||
- **Parent claim：** Frontier 的 config-ranking fidelity 由 workload execution phase 决定；prefill-only 可能比 decode/mixed 更容易由 isolated operator profiles 组合。
|
||||
- **目的：** 用跨 model 的 phase factorial 区分 phase-complexity explanation 与 model/runtime/profile-specific explanation。
|
||||
- **Competing hypotheses：** H-phase：prefill-only 是低状态反馈的 compatibility envelope，因此 30B prefill-only 也能达到低-regret ranking，而 decode/mixed 更容易失真；H-stack：30B 的失败主要来自 FA3/CUDA-graph/routing/profile composition 等 stack-specific mismatch，因此即使 prefill-only 也可能失败；H-margin：mixed 的 differential 可以错误，但只要 topology margin 足够仍会选对 top set。
|
||||
- **事前预测：** 若 H-phase 成立，30B prefill-only 满足 regret ≤5%、Kendall τ-b ≥0.8，并明显优于同模型 mixed；若 H-stack 成立，30B prefill-only 仍过不了 gate；若 H-margin 成立，235B mixed 可保持 top set，但会漏掉 MNS/MBT pair directions。
|
||||
- **判定规则：** 30B prefill-only fail → “prefill-only 是充分条件”被否证，下一步优先做 same-model execution-context ablation；30B prefill-only pass 且新增 decode-heavy mixed fail → 支持 phase hypothesis；30B prefill-only 与 235B mixed 都 pass → phase 不能单独解释，转向 margin-aware compatibility envelope。
|
||||
|
||||
## Setup
|
||||
|
||||
- **自变量：** model×phase：已有 Qwen3-235B-A22B-FP8 mixed；新增 Qwen3-30B-A3B BF16 prefill-only。
|
||||
- **控制变量：** dash0 H20、community vLLM 与各自 frozen Frontier profile、同一 config 内 real/sim 的 request shape、arrival lattice、SLO、prefix policy、MNS/MBT/TP 与随机种子。
|
||||
- **30B system context：** community vLLM 0.20.0+cu129，BF16 weights/activation/KV，TP∈{1,2,4}，MNS∈{8,16,32,64}，MBT=8192，chunked prefill on，prefix off；real 保留 runtime 默认 CUDA graph，Frontier profile-only 不做 E2E calibration。
|
||||
- **30B workload：** fixed ISL=2048、OSL=1，64 个不同 token-chain prompts，uniform open-loop QPS；fresh server per `(config, rate, round)`，target-rate warmup 与 measured requests 分离。
|
||||
- **30B SLO：** TTFT≤1256 ms，至少 61/64 requests 通过；primary score 为最大共同 tested feasible req/s / 实际 TP GPUs。
|
||||
- **235B baseline：** 已冻结 `ISL=2048, OSL=128`、8 configs、68 个 fresh-server anchors；primary sensitivity TTFT≤1256 ms、TPOT≤150 ms。
|
||||
- **Baselines：** real community vLLM；Frontier same-stack profile-only；historical Qwen30 mixed profile-only；historical frozen per-TP calibration 只作为 upper bound，不参与本 case 拟合。
|
||||
- **Metrics：** top set、worst tie-break regret、Kendall τ-b、exact/non-tied pair direction、anchor confusion、absolute capacity、TTFT p50/p95、real trial variance与GPU-hour。
|
||||
|
||||
## 预期产物与 review
|
||||
|
||||
- **预期数据：** 30B frozen simulator surface；real config-rate anchors；两模型 phase comparison table；failure mechanism breakdown。
|
||||
- **Figure prototype：** `mock-phase-factorial.png`；x=model×phase，左轴=worst regret，右侧 annotation=τ-b；虚线是 5% regret gate。mock 只表达可区分趋势，不进入结论。
|
||||
- **人工 review：** 已批准。用户要求 smoke 通过后推进实验；先做这两个 case，再根据 hypothesis verdict 扩展。
|
||||
- **Review 意见：** 不把 existing 235B mixed top-set match 隐藏掉；不把“全 config 并列”算作成功 hit；30B prefill-only 必须使用相同 primary ranking objective。
|
||||
|
||||
## Benchmark design audit
|
||||
|
||||
| Risk | Verdict | 处理 |
|
||||
|---|---|---|
|
||||
| Selective benchmarking | PASS for initial screen | 同时报已有 235B mixed success和内部 pairwise failure；后续 expansion 由预注册 verdict 触发 |
|
||||
| Simplified workload | NEEDS EVIDENCE | fixed-shape 只用于 phase isolation，不外推 trace-faithful mixed |
|
||||
| Calibration=evaluation | PASS | 新 case 不用 serving E2E 数据拟合 scale |
|
||||
| Missing significance | NEEDS EVIDENCE until run | boundary anchors做独立 fresh-server repeat，保留 disagreement |
|
||||
| Relative-only result | PASS by design | 同时报 req/s/GPU、TTFT distribution、rank/regret |
|
||||
|
||||
## 复现信息
|
||||
|
||||
- **Code：** AITuner branch `codex/fidelity-prefix-pilot-20260714`；Frontier upstream `d9cfeb6d8791fbf2f295dd9744c56a666171776e` + frozen known patches。
|
||||
- **Environment：** 只使用 dash0 8×H20；Qwen30 venv `/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1`；model `/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B`。
|
||||
- **产物路径：** local/remote `runs/frontier-phase-factorial-v0/`；raw GPU artifacts 由 fleet harvest，condensed JSON/CSV 进入结果目录。
|
||||
- **已知 deviation：** 235B 为 FP8/vLLM0.10.2/FlashInfer eager，30B 为 BF16/vLLM0.20/FA3/default CUDA graph；因此跨模型只检验 hypothesis consistency，causal phase claim 最终仍需 same-model phase pair。
|
||||
|
||||
## 结果
|
||||
|
||||
- **观察事实：** 235B fixed-shape mixed 已完成：real/sim TP4 top set exact match，worst regret=0、τ-b=0.8944；但 20 个 real non-tie pairs 只保持 16 个，10/34 anchors false-infeasible，TP8 MNS×MBT interaction 被漏掉。30B prefill-only 待运行。
|
||||
- **异常：** 无。
|
||||
- **Interpretation 与剩余 alternatives：** 强版本“mixed 必然失败”已被 235B top-set result 削弱；仍可能存在 phase-dependent error magnitude，由 topology margin 掩盖。
|
||||
- **Claim update：** unchanged，等待 30B prefill-only。
|
||||
- **下一步：** freeze 30B simulator surface → guided real anchors → joint verdict；只有判定需要时扩展 decode-heavy 235B 或 trace-shaped 30B prefill。
|
||||
24
runs/frontier-phase-factorial-v0/fleet.toml
Normal file
24
runs/frontier-phase-factorial-v0/fleet.toml
Normal file
@@ -0,0 +1,24 @@
|
||||
version = 1
|
||||
|
||||
[paths]
|
||||
state_dir = "runs/frontier-phase-factorial-v0/fleet-state"
|
||||
artifacts_dir = "runs/frontier-phase-factorial-v0/fleet-artifacts"
|
||||
|
||||
[ssh]
|
||||
connect_timeout_sec = 10
|
||||
|
||||
[scheduler]
|
||||
gpu_free_memory_mb = 1024
|
||||
gpu_free_utilization_pct = 10
|
||||
prefer_pack = true
|
||||
|
||||
[sync]
|
||||
mode = "scp"
|
||||
local_path = "runs/frontier-phase-factorial-v0/remote-sync-marker"
|
||||
|
||||
[[hosts]]
|
||||
name = "dash0"
|
||||
ssh_alias = "dash0"
|
||||
enabled = true
|
||||
sync_remote_path = "/home/admin/cpfs/wjh/aituner/phase-factorial-sync-marker"
|
||||
fleet_root = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0"
|
||||
241
runs/frontier-phase-factorial-v0/jobs_full.toml
Normal file
241
runs/frontier-phase-factorial-v0/jobs_full.toml
Normal file
@@ -0,0 +1,241 @@
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp1-mns8-20260717-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp1-mns8-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "1"
|
||||
MNS = "8"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8720"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns8-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp1-mns16-20260717-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp1-mns16-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "1"
|
||||
MNS = "16"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8721"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns16-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp1-mns32-20260717-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp1-mns32-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "1"
|
||||
MNS = "32"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8722"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns32-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp1-mns64-20260717-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp1-mns64-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "1"
|
||||
MNS = "64"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8723"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns64-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp2-mns8-20260717-v1"
|
||||
gpus = 2
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp2-mns8-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "2"
|
||||
MNS = "8"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8724"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns8-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp2-mns16-20260717-v1"
|
||||
gpus = 2
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp2-mns16-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "2"
|
||||
MNS = "16"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8725"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns16-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp2-mns32-20260717-v1"
|
||||
gpus = 2
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp2-mns32-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "2"
|
||||
MNS = "32"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8726"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns32-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp2-mns64-20260717-v1"
|
||||
gpus = 2
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp2-mns64-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "2"
|
||||
MNS = "64"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8727"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns64-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp4-mns8-20260717-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp4-mns8-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "4"
|
||||
MNS = "8"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8728"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns8-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp4-mns16-20260717-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp4-mns16-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "4"
|
||||
MNS = "16"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8729"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns16-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp4-mns32-20260717-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp4-mns32-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "4"
|
||||
MNS = "32"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8730"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns32-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-tp4-mns64-20260717-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-tp4-mns64-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
RATES = "4 8 16 32 64"
|
||||
SERVER_PORT = "8731"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns64-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
21
runs/frontier-phase-factorial-v0/jobs_smoke.toml
Normal file
21
runs/frontier-phase-factorial-v0/jobs_smoke.toml
Normal file
@@ -0,0 +1,21 @@
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen30-prefill-real-smoke-tp1-mns8-20260717-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 1800 bash run_qwen30_prefill_real_config.sh"
|
||||
artifacts = ["artifacts/real-smoke-tp1-mns8-v1"]
|
||||
|
||||
[jobs.env]
|
||||
HOME = "/tmp/wjh"
|
||||
XDG_CACHE_HOME = "/tmp/wjh/.cache"
|
||||
VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm"
|
||||
TP = "1"
|
||||
MNS = "8"
|
||||
RATES = "4"
|
||||
SERVER_PORT = "8718"
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-smoke-tp1-mns8-v1"
|
||||
VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
|
||||
BIN
runs/frontier-phase-factorial-v0/mock-phase-factorial.png
Normal file
BIN
runs/frontier-phase-factorial-v0/mock-phase-factorial.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 116 KiB |
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Render the preregistered phase-factorial hypothesis schematic (mock data)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import matplotlib
|
||||
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
|
||||
def main() -> None:
|
||||
labels = ("235B\nprefill-only", "235B\nmixed", "30B\nprefill-only", "30B\nmixed")
|
||||
x = range(len(labels))
|
||||
phase_hypothesis = (1.0, 18.0, 2.0, 28.0)
|
||||
stack_hypothesis = (1.0, 4.0, 24.0, 30.0)
|
||||
|
||||
fig, ax = plt.subplots(figsize=(8.6, 4.8), constrained_layout=True)
|
||||
ax.plot(x, phase_hypothesis, marker="o", lw=2, label="H-phase (mock)")
|
||||
ax.plot(x, stack_hypothesis, marker="s", lw=2, label="H-stack (mock)")
|
||||
ax.axhline(5, color="black", ls="--", lw=1.2, label="5% regret gate")
|
||||
ax.set_xticks(list(x), labels)
|
||||
ax.set_ylabel("Worst selected-config regret (%) — MOCK DATA")
|
||||
ax.set_title("Schematic only: predictions that distinguish phase vs stack explanations")
|
||||
ax.set_ylim(0, 35)
|
||||
ax.grid(axis="y", alpha=0.25)
|
||||
ax.legend()
|
||||
ax.text(
|
||||
0.01,
|
||||
0.98,
|
||||
"MOCK DATA / NOT A RESULT",
|
||||
transform=ax.transAxes,
|
||||
va="top",
|
||||
color="crimson",
|
||||
weight="bold",
|
||||
)
|
||||
output = Path(__file__).with_name("mock-phase-factorial.png")
|
||||
fig.savefig(output, dpi=180)
|
||||
print(output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
216
runs/frontier-phase-factorial-v0/qwen30_prefill_client.py
Normal file
216
runs/frontier-phase-factorial-v0/qwen30_prefill_client.py
Normal file
@@ -0,0 +1,216 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Open-loop fixed-shape prefill-only workload for one real offered-load anchor."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import hashlib
|
||||
import http.client
|
||||
import json
|
||||
import math
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
TTFT_SLO_MS = 1256.0
|
||||
TARGET_PASS_RATE = 0.95
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, required=True)
|
||||
parser.add_argument("--served-model", required=True)
|
||||
parser.add_argument("--model-path", type=Path, required=True)
|
||||
parser.add_argument("--rate", type=float, required=True)
|
||||
parser.add_argument("--requests", type=int, default=64)
|
||||
parser.add_argument("--input-tokens", type=int, default=2048)
|
||||
parser.add_argument("--timeout-seconds", type=float, default=900.0)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def percentile(values: list[float], fraction: float) -> float | None:
|
||||
if not values:
|
||||
return None
|
||||
ordered = sorted(values)
|
||||
index = min(len(ordered) - 1, max(0, math.ceil(fraction * len(ordered)) - 1))
|
||||
return ordered[index]
|
||||
|
||||
|
||||
def run_request(
|
||||
*,
|
||||
request_index: int,
|
||||
scheduled_at: float,
|
||||
benchmark_start: float,
|
||||
args: argparse.Namespace,
|
||||
prompt_ids: list[int],
|
||||
) -> dict[str, Any]:
|
||||
delay = scheduled_at - time.perf_counter()
|
||||
if delay > 0:
|
||||
time.sleep(delay)
|
||||
admitted = time.perf_counter()
|
||||
record: dict[str, Any] = {
|
||||
"request_index": request_index,
|
||||
"scheduled_s": scheduled_at - benchmark_start,
|
||||
"admitted_s": admitted - benchmark_start,
|
||||
"admission_lag_ms": (admitted - scheduled_at) * 1000.0,
|
||||
"success": False,
|
||||
}
|
||||
connection = http.client.HTTPConnection(args.host, args.port, timeout=args.timeout_seconds)
|
||||
body = {
|
||||
"model": args.served_model,
|
||||
"prompt": prompt_ids,
|
||||
"min_tokens": 1,
|
||||
"max_tokens": 1,
|
||||
"ignore_eos": True,
|
||||
"temperature": 0,
|
||||
"stream": True,
|
||||
"stream_options": {"include_usage": True},
|
||||
"return_token_ids": True,
|
||||
}
|
||||
try:
|
||||
started = time.perf_counter()
|
||||
connection.request(
|
||||
"POST",
|
||||
"/v1/completions",
|
||||
body=json.dumps(body, separators=(",", ":")).encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
response = connection.getresponse()
|
||||
if response.status != 200:
|
||||
raise RuntimeError(
|
||||
f"HTTP {response.status}: {response.read().decode(errors='replace')}"
|
||||
)
|
||||
first_token_at = None
|
||||
streamed_tokens = 0
|
||||
usage = None
|
||||
while True:
|
||||
raw = response.readline()
|
||||
if not raw:
|
||||
break
|
||||
line = raw.decode(errors="replace").strip()
|
||||
if not line.startswith("data:"):
|
||||
continue
|
||||
data = line[5:].strip()
|
||||
if data == "[DONE]":
|
||||
break
|
||||
payload = json.loads(data)
|
||||
if payload.get("usage"):
|
||||
usage = payload["usage"]
|
||||
emitted = 0
|
||||
for choice in payload.get("choices") or []:
|
||||
token_ids = choice.get("token_ids") or []
|
||||
emitted += len(token_ids) if token_ids else int(bool(choice.get("text")))
|
||||
if emitted:
|
||||
first_token_at = first_token_at or time.perf_counter()
|
||||
streamed_tokens += emitted
|
||||
finished = time.perf_counter()
|
||||
if first_token_at is None or usage is None:
|
||||
raise RuntimeError("missing streaming token or usage")
|
||||
prompt_tokens = int(usage["prompt_tokens"])
|
||||
completion_tokens = int(usage["completion_tokens"])
|
||||
if prompt_tokens != args.input_tokens or completion_tokens != 1:
|
||||
raise RuntimeError(f"usage mismatch: {prompt_tokens}+{completion_tokens}")
|
||||
ttft = (first_token_at - started) * 1000.0
|
||||
record.update(
|
||||
{
|
||||
"success": True,
|
||||
"prompt_tokens": prompt_tokens,
|
||||
"completion_tokens": completion_tokens,
|
||||
"streamed_token_count": streamed_tokens,
|
||||
"ttft_ms": ttft,
|
||||
"e2e_ms": (finished - started) * 1000.0,
|
||||
"slo_pass": ttft <= TTFT_SLO_MS,
|
||||
}
|
||||
)
|
||||
except Exception as error: # Failed requests remain in the SLO denominator.
|
||||
record["error"] = f"{type(error).__name__}: {error}"
|
||||
record["slo_pass"] = False
|
||||
finally:
|
||||
connection.close()
|
||||
return record
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
if args.rate <= 0 or args.requests <= 0 or args.input_tokens <= 0:
|
||||
raise ValueError("rate, requests, and input tokens must be positive")
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(args.model_path, trust_remote_code=True)
|
||||
excluded = set(tokenizer.all_special_ids)
|
||||
candidates = [
|
||||
token_id for token_id in range(tokenizer.vocab_size) if token_id not in excluded
|
||||
]
|
||||
if len(candidates) < args.requests + 1:
|
||||
raise RuntimeError("tokenizer has too few non-special token IDs")
|
||||
base_id = candidates[0]
|
||||
prompts = [
|
||||
[candidates[index + 1], *([base_id] * (args.input_tokens - 1))]
|
||||
for index in range(args.requests)
|
||||
]
|
||||
prompt_hash = hashlib.sha256(
|
||||
"\n".join(",".join(map(str, prompt)) for prompt in prompts).encode()
|
||||
).hexdigest()
|
||||
|
||||
benchmark_start = time.perf_counter() + 2.0
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=args.requests) as pool:
|
||||
futures = [
|
||||
pool.submit(
|
||||
run_request,
|
||||
request_index=index,
|
||||
scheduled_at=benchmark_start + index / args.rate,
|
||||
benchmark_start=benchmark_start,
|
||||
args=args,
|
||||
prompt_ids=prompts[index],
|
||||
)
|
||||
for index in range(args.requests)
|
||||
]
|
||||
requests = [future.result() for future in futures]
|
||||
requests.sort(key=lambda row: int(row["request_index"]))
|
||||
completed = [row for row in requests if row["success"]]
|
||||
passed = sum(bool(row["slo_pass"]) for row in requests)
|
||||
ttfts = [float(row["ttft_ms"]) for row in completed]
|
||||
pass_rate = passed / len(requests)
|
||||
payload = {
|
||||
"schema": "qwen30-prefill-rate-anchor-v1",
|
||||
"workload": {
|
||||
"offered_request_rate": args.rate,
|
||||
"request_count": args.requests,
|
||||
"input_tokens": args.input_tokens,
|
||||
"output_tokens": 1,
|
||||
"prefix_caching": False,
|
||||
"arrival": "open_loop_uniform",
|
||||
"last_scheduled_arrival_s": (args.requests - 1) / args.rate,
|
||||
"prompt_vector_sha256": prompt_hash,
|
||||
},
|
||||
"summary": {
|
||||
"completed": len(completed),
|
||||
"failed": len(requests) - len(completed),
|
||||
"ttft_p50_ms": percentile(ttfts, 0.50),
|
||||
"ttft_p95_ms": percentile(ttfts, 0.95),
|
||||
"ttft_max_ms": max(ttfts) if ttfts else None,
|
||||
"admission_lag_max_ms": max(
|
||||
float(row["admission_lag_ms"]) for row in requests
|
||||
),
|
||||
"slo": {
|
||||
"ttft_threshold_ms": TTFT_SLO_MS,
|
||||
"passed": passed,
|
||||
"pass_rate": pass_rate,
|
||||
"feasible": pass_rate >= TARGET_PASS_RATE,
|
||||
},
|
||||
},
|
||||
"requests": requests,
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
print(json.dumps(payload["summary"], sort_keys=True), flush=True)
|
||||
if len(completed) != args.requests:
|
||||
raise SystemExit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
# Phase-factorial fleet sync marker
|
||||
|
||||
Experiment source is synchronized through Git into the clean dash0 checkout
|
||||
`/home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1`. This small
|
||||
directory exists only to satisfy the fleet orchestrator's explicit sync phase
|
||||
without copying local simulator caches or raw metrics to the GPU host.
|
||||
@@ -0,0 +1,400 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Freeze the Qwen30 fixed-shape prefill-only Frontier surface."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODEL = "qwen3-a3b-30b-moe"
|
||||
RATES = (4.0, 8.0, 16.0, 32.0, 64.0)
|
||||
TTFT_SLO_MS = 1256.0
|
||||
TARGET_PASS_RATE = 0.95
|
||||
NUM_BLOCKS = {1: 20080, 2: 76537, 4: 191727}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Config:
|
||||
tp: int
|
||||
mns: int
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return f"tp{self.tp}_mns{self.mns}"
|
||||
|
||||
|
||||
GRID = tuple(Config(tp, mns) for tp in (1, 2, 4) for mns in (8, 16, 32, 64))
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-source", type=Path, required=True)
|
||||
parser.add_argument("--replayserve-root", type=Path, required=True)
|
||||
parser.add_argument("--profile-root", type=Path, required=True)
|
||||
parser.add_argument("--python-deps", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
parser.add_argument("--requests", type=int, default=64)
|
||||
parser.add_argument("--rate", type=float, action="append")
|
||||
parser.add_argument("--config", action="append")
|
||||
parser.add_argument("--timeout-seconds", type=float, default=900.0)
|
||||
parser.add_argument("--resume", action="store_true")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_suffix(path.suffix + ".tmp")
|
||||
temporary.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
os.replace(temporary, path)
|
||||
|
||||
|
||||
def load_module(name: str, path: Path):
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError(path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def write_trace(path: Path, *, requests: int, rate: float) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fields = [
|
||||
"arrived_at",
|
||||
"num_prefill_tokens",
|
||||
"num_decode_tokens",
|
||||
"session_id",
|
||||
"block_hash_ids",
|
||||
]
|
||||
with path.open("w", newline="") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=fields)
|
||||
writer.writeheader()
|
||||
for request_id in range(requests):
|
||||
writer.writerow(
|
||||
{
|
||||
"arrived_at": f"{request_id / rate:.12f}",
|
||||
"num_prefill_tokens": 2048,
|
||||
"num_decode_tokens": 1,
|
||||
"session_id": request_id,
|
||||
"block_hash_ids": "|".join(
|
||||
str(request_id * 128 + block + 1) for block in range(128)
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def profile_paths(root: Path) -> dict[str, Path]:
|
||||
paths = {
|
||||
"linear": root / "linear_op.csv",
|
||||
"attention": root / "attention.csv",
|
||||
"moe": root / "moe.csv",
|
||||
"manifest": root / "manifest.json",
|
||||
}
|
||||
missing = [str(path) for path in paths.values() if not path.is_file()]
|
||||
if missing:
|
||||
raise FileNotFoundError(missing)
|
||||
return paths
|
||||
|
||||
|
||||
def validate_profile(paths: dict[str, Path]) -> dict[str, Any]:
|
||||
manifest = json.loads(paths["manifest"].read_text())
|
||||
expected = manifest["outputs"]
|
||||
for filename in ("linear_op.csv", "attention.csv", "moe.csv"):
|
||||
actual = sha256(paths[{"linear_op.csv": "linear", "attention.csv": "attention", "moe.csv": "moe"}[filename]])
|
||||
if actual != expected[filename]:
|
||||
raise ValueError(f"profile hash mismatch for {filename}")
|
||||
with paths["attention"].open(newline="") as source:
|
||||
rows = list(csv.DictReader(source))
|
||||
coverage = {}
|
||||
for tp in (1, 2, 4):
|
||||
exact = [
|
||||
row
|
||||
for row in rows
|
||||
if int(row["num_tensor_parallel_workers"]) == tp
|
||||
and row["is_prefill"].lower() == "true"
|
||||
and row.get("is_true_mixed_batch", "").lower() != "true"
|
||||
and int(float(row["total_tokens"])) == 2048
|
||||
]
|
||||
if len(exact) != 1:
|
||||
raise ValueError(f"expected one exact TP{tp} 2048-token prefill row, got {len(exact)}")
|
||||
coverage[str(tp)] = {"exact_prefill_2048_rows": len(exact), "profile_batch_size": int(exact[0]["batch_size"])}
|
||||
return {"manifest": manifest, "attention": coverage}
|
||||
|
||||
|
||||
def knobs(config: Config, paths: dict[str, Path], cache: Path) -> dict[str, Any]:
|
||||
return {
|
||||
"simulation_mode": "online",
|
||||
"sys_arch": "co-location",
|
||||
"num_replicas": 1,
|
||||
"cluster_scheduler": "sticky_round_robin",
|
||||
"model_name": MODEL,
|
||||
"device": "h20",
|
||||
"network_device": "h20_dgx",
|
||||
"attn_tensor_parallel_size": config.tp,
|
||||
"attn_data_parallel_size": 1,
|
||||
"moe_tensor_parallel_size": config.tp,
|
||||
"moe_expert_parallel_size": 1,
|
||||
"num_pipeline_stages": 1,
|
||||
"replica_scheduler": "vllm_v1",
|
||||
"decode_cuda_graph_mode": "none",
|
||||
"batch_size_cap": config.mns,
|
||||
"max_tokens_in_batch": 8192,
|
||||
"long_prefill_token_threshold": 0,
|
||||
"block_size": 16,
|
||||
"num_blocks_mode": "explicit",
|
||||
"num_blocks": NUM_BLOCKS[config.tp],
|
||||
"gpu_memory_utilization": 0.92,
|
||||
"non_kv_cache_overhead_bytes": 0,
|
||||
"trace_max_tokens": 40960,
|
||||
"cache_dir": str(cache),
|
||||
"enable_dummy_mode": False,
|
||||
"linear_op_input_file": str(paths["linear"]),
|
||||
"atten_input_file": str(paths["attention"]),
|
||||
"moe_input_file": str(paths["moe"]),
|
||||
"prediction_max_prefill_chunk_size": 18000,
|
||||
"prediction_max_batch_size": 128,
|
||||
"prediction_max_tokens_per_request": 32768,
|
||||
"no_cache": False,
|
||||
"skip_cpu_overhead_modeling": True,
|
||||
"enable_prefix_caching": False,
|
||||
"enable_chunked_prefill": True,
|
||||
}
|
||||
|
||||
|
||||
def find_metrics(run_dir: Path) -> tuple[Path, Path]:
|
||||
systems = list((run_dir / "metrics").rglob("system_metrics.json"))
|
||||
requests = list((run_dir / "metrics").rglob("request_metrics.csv"))
|
||||
if len(systems) != 1 or len(requests) != 1:
|
||||
raise RuntimeError(f"metric pair mismatch: {len(systems)}/{len(requests)}")
|
||||
return systems[0], requests[0]
|
||||
|
||||
|
||||
def score(system_path: Path, request_path: Path, expected_requests: int) -> dict[str, Any]:
|
||||
system = json.loads(system_path.read_text())
|
||||
metadata = system["simulation_metadata"]
|
||||
if int(metadata["completed_requests"]) != expected_requests:
|
||||
raise ValueError("Frontier completion count mismatch")
|
||||
with request_path.open(newline="") as source:
|
||||
rows = list(csv.DictReader(source))
|
||||
if len(rows) != expected_requests:
|
||||
raise ValueError("request metric count mismatch")
|
||||
values = []
|
||||
passed = 0
|
||||
for row in rows:
|
||||
if int(float(row["request_num_prefill_tokens"])) != 2048:
|
||||
raise ValueError("prefill shape drift")
|
||||
if int(float(row["request_num_decode_tokens"])) != 1:
|
||||
raise ValueError("decode shape drift")
|
||||
ttft = float(row["ttft"])
|
||||
if not math.isfinite(ttft) or ttft < 0:
|
||||
raise ValueError("invalid TTFT")
|
||||
values.append(ttft)
|
||||
passed += int(ttft <= TTFT_SLO_MS)
|
||||
ordered = sorted(values)
|
||||
pass_rate = passed / expected_requests
|
||||
return {
|
||||
"ttft_p50_ms": ordered[math.ceil(0.50 * len(ordered)) - 1],
|
||||
"ttft_p95_ms": ordered[math.ceil(0.95 * len(ordered)) - 1],
|
||||
"ttft_max_ms": max(ordered),
|
||||
"passed": passed,
|
||||
"pass_rate": pass_rate,
|
||||
"feasible": pass_rate >= TARGET_PASS_RATE,
|
||||
"throughput_requests_per_second": float(system["throughput_metrics"]["requests_per_second"]),
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
args.frontier_source = args.frontier_source.resolve()
|
||||
args.replayserve_root = args.replayserve_root.resolve()
|
||||
args.profile_root = args.profile_root.resolve()
|
||||
args.python_deps = args.python_deps.resolve()
|
||||
args.output_root = args.output_root.resolve()
|
||||
rates = tuple(args.rate or RATES)
|
||||
selected = list(GRID)
|
||||
if args.config:
|
||||
wanted = set(args.config)
|
||||
selected = [config for config in GRID if config.name in wanted]
|
||||
if {config.name for config in selected} != wanted:
|
||||
raise ValueError(f"unknown configs: {wanted - {config.name for config in selected}}")
|
||||
paths = profile_paths(args.profile_root)
|
||||
coverage = validate_profile(paths)
|
||||
builder = load_module(
|
||||
"qwen30_prefill_frontier_builder",
|
||||
args.replayserve_root / "tools/run_frontier_sweep.py",
|
||||
)
|
||||
frontier_head = subprocess.run(
|
||||
["git", "-C", str(args.frontier_source), "rev-parse", "HEAD"],
|
||||
check=True,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
).stdout.strip()
|
||||
traces = {}
|
||||
for rate in rates:
|
||||
trace = args.output_root / "traces" / f"r{rate:g}.csv"
|
||||
write_trace(trace, requests=args.requests, rate=rate)
|
||||
traces[rate] = trace
|
||||
|
||||
config_results = []
|
||||
for config in selected:
|
||||
loads = []
|
||||
config_knobs = knobs(config, paths, args.output_root / "cache")
|
||||
for rate in rates:
|
||||
run_dir = args.output_root / "runs" / config.name / f"r{rate:g}"
|
||||
result_path = run_dir / "result.json"
|
||||
if args.resume and result_path.is_file():
|
||||
loads.append(json.loads(result_path.read_text()))
|
||||
continue
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
command = builder.build_frontier_command(
|
||||
python_bin="/usr/bin/python3",
|
||||
trace_file=traces[rate],
|
||||
metrics_root=run_dir / "metrics",
|
||||
run_id=f"qwen30_prefill_{config.name}_r{rate:g}",
|
||||
knobs=config_knobs,
|
||||
)
|
||||
write_json(run_dir / "command.json", command)
|
||||
environment = os.environ.copy()
|
||||
pythonpath = [str(args.python_deps), str(args.frontier_source)]
|
||||
if environment.get("PYTHONPATH"):
|
||||
pythonpath.append(environment["PYTHONPATH"])
|
||||
environment.update(
|
||||
{
|
||||
"PYTHONPATH": ":".join(pythonpath),
|
||||
"CUDA_VISIBLE_DEVICES": "",
|
||||
"NVIDIA_VISIBLE_DEVICES": "void",
|
||||
"WANDB_DISABLED": "true",
|
||||
"VIDUR_DISABLE_WANDB": "1",
|
||||
"FRONTIER_LOG_LEVEL": "WARNING",
|
||||
"PYTHONDONTWRITEBYTECODE": "1",
|
||||
}
|
||||
)
|
||||
started = time.time()
|
||||
with (run_dir / "stdout.log").open("w") as stdout, (
|
||||
run_dir / "stderr.log"
|
||||
).open("w") as stderr:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
cwd=args.frontier_source,
|
||||
env=environment,
|
||||
stdout=stdout,
|
||||
stderr=stderr,
|
||||
timeout=args.timeout_seconds,
|
||||
check=False,
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Frontier failed for {config.name} rate={rate}: {completed.returncode}"
|
||||
)
|
||||
system_path, request_path = find_metrics(run_dir)
|
||||
result = {
|
||||
"status": "completed",
|
||||
"config": asdict(config) | {"name": config.name},
|
||||
"offered_request_rate": rate,
|
||||
"offered_request_rate_per_gpu": rate / config.tp,
|
||||
"request_count": args.requests,
|
||||
"elapsed_seconds": time.time() - started,
|
||||
"trace_sha256": sha256(traces[rate]),
|
||||
"request_metrics_sha256": sha256(request_path),
|
||||
"score": score(system_path, request_path, args.requests),
|
||||
}
|
||||
write_json(result_path, result)
|
||||
loads.append(result)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"config": config.name,
|
||||
"rate": rate,
|
||||
"pass_rate": result["score"]["pass_rate"],
|
||||
"feasible": result["score"]["feasible"],
|
||||
},
|
||||
sort_keys=True,
|
||||
),
|
||||
flush=True,
|
||||
)
|
||||
config_results.append({"config": asdict(config) | {"name": config.name}, "loads": loads})
|
||||
|
||||
capacities = []
|
||||
for item in config_results:
|
||||
feasible = [
|
||||
load["offered_request_rate"]
|
||||
for load in item["loads"]
|
||||
if load["score"]["feasible"]
|
||||
]
|
||||
capacity = max(feasible) if feasible else None
|
||||
capacities.append(
|
||||
{
|
||||
"config": item["config"],
|
||||
"maximum_tested_feasible_request_rate": capacity,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": (
|
||||
capacity / item["config"]["tp"] if capacity is not None else None
|
||||
),
|
||||
"lower_censored": capacity is None,
|
||||
"upper_censored": capacity == max(rates) if capacity is not None else False,
|
||||
}
|
||||
)
|
||||
capacities.sort(
|
||||
key=lambda row: (
|
||||
-(row["maximum_tested_feasible_request_rate_per_gpu"] or -1),
|
||||
row["config"]["name"],
|
||||
)
|
||||
)
|
||||
full = selected == list(GRID) and rates == RATES and args.requests == 64
|
||||
manifest = {
|
||||
"schema": "frontier-qwen30-prefill-surface-v1",
|
||||
"status": "frozen_before_real" if full else "partial_not_decision_bearing",
|
||||
"contract": {
|
||||
"rates": rates,
|
||||
"requests_per_anchor": args.requests,
|
||||
"input_tokens": 2048,
|
||||
"output_tokens": 1,
|
||||
"ttft_slo_ms": TTFT_SLO_MS,
|
||||
"target_pass_rate": TARGET_PASS_RATE,
|
||||
"prefix_caching": False,
|
||||
"arrival": "open_loop_uniform",
|
||||
},
|
||||
"frontier": {
|
||||
"source": str(args.frontier_source),
|
||||
"git_head": frontier_head,
|
||||
"git_status_short": subprocess.run(
|
||||
["git", "-C", str(args.frontier_source), "status", "--short"],
|
||||
check=True,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
).stdout,
|
||||
},
|
||||
"profiles": {
|
||||
"root": str(args.profile_root),
|
||||
"coverage": coverage,
|
||||
"sha256": {name: sha256(path) for name, path in paths.items()},
|
||||
},
|
||||
"config_results": config_results,
|
||||
"capacity": capacities,
|
||||
}
|
||||
write_json(args.output_root / "frontier_surface_frozen.json", manifest)
|
||||
print(args.output_root / "frontier_surface_frozen.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:?OUTPUT_ROOT is required}"
|
||||
TP="${TP:?TP is required}"
|
||||
MNS="${MNS:?MNS is required}"
|
||||
RATES="${RATES:-4 8 16 32 64}"
|
||||
SERVER_PORT="${SERVER_PORT:?SERVER_PORT is required}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1}"
|
||||
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}"
|
||||
SERVED_MODEL="qwen3-30b-prefill-only"
|
||||
SERVER_PID=""
|
||||
|
||||
mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/provenance"
|
||||
exec > >(tee -a "${OUTPUT_ROOT}/logs/controller.log") 2>&1
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
for _ in $(seq 1 30); do
|
||||
kill -0 "${SERVER_PID}" 2>/dev/null || break
|
||||
sleep 1
|
||||
done
|
||||
kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
fi
|
||||
SERVER_PID=""
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES:-}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne "${TP}" ]]; then
|
||||
echo "ERROR: expected ${TP} allocated GPUs, got ${CUDA_VISIBLE_DEVICES:-unset}" >&2
|
||||
exit 1
|
||||
fi
|
||||
read -r -a RATE_ARRAY <<< "${RATES}"
|
||||
|
||||
echo "QWEN30_PREFILL_REAL_LAUNCH_ECHO host=$(hostname) gpus=${CUDA_VISIBLE_DEVICES} model=${MODEL_ROOT} runtime=vLLM-0.20.0+cu129 dtype=BF16 config=TP${TP}_MNS${MNS}_MBT8192 rates=${RATES// /,} rounds=2 requests=64 shape=ISL2048_OSL1 arrivals=uniform prefix=off cuda_graph=runtime_default isolation=fresh_server_per_anchor output=${OUTPUT_ROOT}"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
sha256sum run_qwen30_prefill_real_config.sh qwen30_prefill_client.py \
|
||||
> "${OUTPUT_ROOT}/provenance/source.sha256"
|
||||
"${VENV_ROOT}/bin/python" - "${TP}" "${MNS}" "${RATES}" \
|
||||
> "${OUTPUT_ROOT}/provenance/contract.json" <<'PY'
|
||||
import importlib.metadata as metadata
|
||||
import json
|
||||
import platform
|
||||
import sys
|
||||
|
||||
tp, mns, rates = sys.argv[1:]
|
||||
print(json.dumps({
|
||||
"python": platform.python_version(),
|
||||
"torch": metadata.version("torch"),
|
||||
"transformers": metadata.version("transformers"),
|
||||
"vllm": metadata.version("vllm"),
|
||||
"config": {"tp": int(tp), "mns": int(mns), "mbt": 8192},
|
||||
"rates": [float(value) for value in rates.split()],
|
||||
"rounds": 2,
|
||||
"requests_per_anchor": 64,
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"target_rate_warmup_requests": "min(32, max(4, ceil(rate * 2)))",
|
||||
"input_tokens": 2048,
|
||||
"output_tokens": 1,
|
||||
"ttft_slo_ms": 1256.0,
|
||||
"target_pass_rate": 0.95,
|
||||
}, indent=2, sort_keys=True))
|
||||
PY
|
||||
nvidia-smi --query-gpu=index,name,uuid,driver_version --format=csv,noheader \
|
||||
> "${OUTPUT_ROOT}/provenance/gpus.csv"
|
||||
sha256sum "${MODEL_ROOT}/config.json" > "${OUTPUT_ROOT}/provenance/model.sha256"
|
||||
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export VLLM_USE_V1=1
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
export HF_HUB_OFFLINE=1
|
||||
export TRANSFORMERS_OFFLINE=1
|
||||
|
||||
for ROUND in 1 2; do
|
||||
ROUND_ROOT="${OUTPUT_ROOT}/round${ROUND}"
|
||||
mkdir -p "${ROUND_ROOT}/logs" "${ROUND_ROOT}/results"
|
||||
ORDERED_RATES=("${RATE_ARRAY[@]}")
|
||||
if [[ "${ROUND}" -eq 2 ]]; then
|
||||
ORDERED_RATES=()
|
||||
for ((index=${#RATE_ARRAY[@]}-1; index>=0; index--)); do
|
||||
ORDERED_RATES+=("${RATE_ARRAY[index]}")
|
||||
done
|
||||
fi
|
||||
for RATE in "${ORDERED_RATES[@]}"; do
|
||||
KEY="$(printf 'r%.2f' "${RATE}" | tr '.' 'p')"
|
||||
SERVER_LOG="${ROUND_ROOT}/logs/server_${KEY}.log"
|
||||
setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
|
||||
--host 127.0.0.1 --port "${SERVER_PORT}" --served-model-name "${SERVED_MODEL}" \
|
||||
--tensor-parallel-size "${TP}" --gpu-memory-utilization 0.92 \
|
||||
--max-model-len 40960 --max-num-batched-tokens 8192 --max-num-seqs "${MNS}" \
|
||||
--no-enable-prefix-caching --enable-chunked-prefill --no-enable-log-requests \
|
||||
> "${SERVER_LOG}" 2>&1 &
|
||||
SERVER_PID=$!
|
||||
READY=0
|
||||
for _ in $(seq 1 120); do
|
||||
if curl -fsS --max-time 2 "http://127.0.0.1:${SERVER_PORT}/v1/models" \
|
||||
> "${ROUND_ROOT}/results/models_${KEY}.json" 2>/dev/null; then
|
||||
READY=1
|
||||
break
|
||||
fi
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
tail -200 "${SERVER_LOG}"
|
||||
exit 1
|
||||
fi
|
||||
sleep 3
|
||||
done
|
||||
if [[ "${READY}" -ne 1 ]]; then
|
||||
tail -200 "${SERVER_LOG}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
WARMUP_REQUESTS="$("${VENV_ROOT}/bin/python" - "${RATE}" <<'PY'
|
||||
import math
|
||||
import sys
|
||||
print(min(32, max(4, math.ceil(float(sys.argv[1]) * 2.0))))
|
||||
PY
|
||||
)"
|
||||
"${VENV_ROOT}/bin/python" qwen30_prefill_client.py --port "${SERVER_PORT}" \
|
||||
--served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" --rate "${RATE}" \
|
||||
--requests "${WARMUP_REQUESTS}" \
|
||||
--output "${ROUND_ROOT}/results/warmup_${KEY}.json"
|
||||
"${VENV_ROOT}/bin/python" qwen30_prefill_client.py --port "${SERVER_PORT}" \
|
||||
--served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" --rate "${RATE}" \
|
||||
--requests 64 --output "${ROUND_ROOT}/results/${KEY}.json"
|
||||
cleanup
|
||||
done
|
||||
done
|
||||
|
||||
find "${OUTPUT_ROOT}" -type f ! -path '*/provenance/artifacts.sha256' -print0 \
|
||||
| sort -z | xargs -0 sha256sum > "${OUTPUT_ROOT}/provenance/artifacts.sha256"
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "QWEN30_PREFILL_REAL_CONFIG_COMPLETE"
|
||||
@@ -0,0 +1,647 @@
|
||||
{
|
||||
"capacity": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 8.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 8.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 8.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 8.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
}
|
||||
],
|
||||
"config_results": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 20.451046466827393,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.0197002243768605,
|
||||
"ttft_max_ms": 171.58529929215405,
|
||||
"ttft_p50_ms": 171.58529929215405,
|
||||
"ttft_p95_ms": 171.58529929215405
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 22.0544171333313,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 7.742175153637853,
|
||||
"ttft_max_ms": 570.3133207139617,
|
||||
"ttft_p50_ms": 391.4107605377559,
|
||||
"ttft_p95_ms": 516.4107605377559
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 18.859471321105957,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.3125,
|
||||
"passed": 20,
|
||||
"throughput_requests_per_second": 8.916224898348922,
|
||||
"ttft_max_ms": 3318.8404181099845,
|
||||
"ttft_p50_ms": 1814.2305264490028,
|
||||
"ttft_p95_ms": 3193.8404181099845
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 16.640390872955322,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.203125,
|
||||
"passed": 13,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 5147.666503402199,
|
||||
"ttft_p50_ms": 2739.3537469047046,
|
||||
"ttft_p95_ms": 4962.713638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp1_mns8",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 18.142091274261475,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 64.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.140625,
|
||||
"passed": 9,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 6100.791503402199,
|
||||
"ttft_p50_ms": 3208.1037469047046,
|
||||
"ttft_p95_ms": 5853.338638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 19.96323037147522,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.0197002243768605,
|
||||
"ttft_max_ms": 171.58529929215405,
|
||||
"ttft_p50_ms": 171.58529929215405,
|
||||
"ttft_p95_ms": 171.58529929215405
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 18.554351568222046,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 7.742175153637853,
|
||||
"ttft_max_ms": 570.3133207139617,
|
||||
"ttft_p50_ms": 391.4107605377559,
|
||||
"ttft_p95_ms": 516.4107605377559
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 25.70284128189087,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.3125,
|
||||
"passed": 20,
|
||||
"throughput_requests_per_second": 8.916224898348922,
|
||||
"ttft_max_ms": 3318.8404181099845,
|
||||
"ttft_p50_ms": 1814.2305264490028,
|
||||
"ttft_p95_ms": 3193.8404181099845
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 20.056548833847046,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.203125,
|
||||
"passed": 13,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 5147.666503402199,
|
||||
"ttft_p50_ms": 2739.3537469047046,
|
||||
"ttft_p95_ms": 4962.713638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp1_mns16",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 11.15293574333191,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 64.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.140625,
|
||||
"passed": 9,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 6100.791503402199,
|
||||
"ttft_p50_ms": 3208.1037469047046,
|
||||
"ttft_p95_ms": 5853.338638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.411258935928345,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.0197002243768605,
|
||||
"ttft_max_ms": 171.58529929215405,
|
||||
"ttft_p50_ms": 171.58529929215405,
|
||||
"ttft_p95_ms": 171.58529929215405
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.355234384536743,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 7.742175153637853,
|
||||
"ttft_max_ms": 570.3133207139617,
|
||||
"ttft_p50_ms": 391.4107605377559,
|
||||
"ttft_p95_ms": 516.4107605377559
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.098335266113281,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.3125,
|
||||
"passed": 20,
|
||||
"throughput_requests_per_second": 8.916224898348922,
|
||||
"ttft_max_ms": 3318.8404181099845,
|
||||
"ttft_p50_ms": 1814.2305264490028,
|
||||
"ttft_p95_ms": 3193.8404181099845
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 9.8981294631958,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.203125,
|
||||
"passed": 13,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 5147.666503402199,
|
||||
"ttft_p50_ms": 2739.3537469047046,
|
||||
"ttft_p95_ms": 4962.713638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp1_mns32",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 9.754717826843262,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 64.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.140625,
|
||||
"passed": 9,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 6100.791503402199,
|
||||
"ttft_p50_ms": 3208.1037469047046,
|
||||
"ttft_p95_ms": 5853.338638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 9.84917163848877,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.0197002243768605,
|
||||
"ttft_max_ms": 171.58529929215405,
|
||||
"ttft_p50_ms": 171.58529929215405,
|
||||
"ttft_p95_ms": 171.58529929215405
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.197595357894897,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 7.742175153637853,
|
||||
"ttft_max_ms": 570.3133207139617,
|
||||
"ttft_p50_ms": 391.4107605377559,
|
||||
"ttft_p95_ms": 516.4107605377559
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.00463604927063,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.3125,
|
||||
"passed": 20,
|
||||
"throughput_requests_per_second": 8.916224898348922,
|
||||
"ttft_max_ms": 3318.8404181099845,
|
||||
"ttft_p50_ms": 1814.2305264490028,
|
||||
"ttft_p95_ms": 3193.8404181099845
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.153925895690918,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.203125,
|
||||
"passed": 13,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 5147.666503402199,
|
||||
"ttft_p50_ms": 2739.3537469047046,
|
||||
"ttft_p95_ms": 4962.713638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp1_mns64",
|
||||
"tp": 1
|
||||
},
|
||||
"elapsed_seconds": 10.055675268173218,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 64.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.140625,
|
||||
"passed": 9,
|
||||
"throughput_requests_per_second": 9.07297385348011,
|
||||
"ttft_max_ms": 6100.791503402199,
|
||||
"ttft_p50_ms": 3208.1037469047046,
|
||||
"ttft_p95_ms": 5853.338638565687
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"contract": {
|
||||
"arrival": "open_loop_uniform",
|
||||
"input_tokens": 2048,
|
||||
"output_tokens": 1,
|
||||
"prefix_caching": false,
|
||||
"rates": [
|
||||
4.0,
|
||||
8.0,
|
||||
16.0,
|
||||
32.0,
|
||||
64.0
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"target_pass_rate": 0.95,
|
||||
"ttft_slo_ms": 1256.0
|
||||
},
|
||||
"frontier": {
|
||||
"git_head": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"git_status_short": " M frontier/config/config.py\n M frontier/entities/request.py\n M frontier/events/cluster_schedule_event.py\n M frontier/execution_time_predictor/sklearn_execution_time_predictor.py\n M frontier/metrics/constants.py\n M frontier/metrics/metrics_store.py\n M frontier/profiling/common/layers/rotary_embedding.py\n M frontier/profiling/moe/moe_impl.py\n M frontier/profiling/moe/moe_vllm_kernel.py\n M frontier/scheduler/cluster_scheduler/__init__.py\n M frontier/scheduler/cluster_scheduler/base_cluster_scheduler.py\n M frontier/scheduler/cluster_scheduler/cluster_scheduler_registry.py\n M frontier/scheduler/cluster_scheduler/sticky_lor_cluster_scheduler.py\n M frontier/scheduler/replica_scheduler/base_replica_scheduler.py\n M frontier/scheduler/replica_scheduler/vllm_v1_engine_replica_scheduler.py\n M frontier/scheduler/replica_stage_scheduler/replica_stage_schduler.py\n M frontier/simulator.py\n M frontier/types/cluster_scheduler_type.py\n?? data/profiling/compute/h20/\n?? frontier/scheduler/cluster_scheduler/prefix_lor_cluster_scheduler.py\n?? runs/\n?? tests/unit/test_attn_prefill_prediction_fallback.py\n",
|
||||
"source": "/tmp/replayserve-frontier-rs1b"
|
||||
},
|
||||
"profiles": {
|
||||
"coverage": {
|
||||
"attention": {
|
||||
"1": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
},
|
||||
"2": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
},
|
||||
"4": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
}
|
||||
},
|
||||
"manifest": {
|
||||
"attention_tp_coverage": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
],
|
||||
"environment_contract": {
|
||||
"dtype": "bfloat16",
|
||||
"frontier_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"hardware": "NVIDIA H20",
|
||||
"model": "Qwen3-30B-A3B",
|
||||
"tensor_parallel_sizes": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
],
|
||||
"vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
|
||||
"vllm_version": "0.20.0"
|
||||
},
|
||||
"inputs": {
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp2-20260716-v1-dispatch-aware-20260716T140743025781Z/artifacts/artifacts/allreduce-full-tp2-v1/raw/allreduce-tp2.json": "97c3c76b5a04e95bd9192423c2b891667c668f39cc0dfecbd097d749939f2d0a",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp4-20260716-v1-dispatch-aware-20260716T141106009788Z/artifacts/artifacts/allreduce-full-tp4-v1/raw/allreduce-tp4.json": "809df9baa6f468cf12bf0c99827475acc67894dd9f3f948976590b665fac0e76",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp1-20260716-v2-20260716T135132587012Z/artifacts/artifacts/flashattn-kv-full-v2-tp1/raw/flashattn-tp1.json": "dcb4c1bf7e76b9c765f78ddd2b8a734f2d7ba2adac13ce017689a8a77fe69a27",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp2-20260716-v2-20260716T135134194295Z/artifacts/artifacts/flashattn-kv-full-v2-tp2/raw/flashattn-tp2.json": "43ce042556ba887c8860614b43ccf0f564e5cebc1a0cffbce299d0acb9fa8d07",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp4-20260716-v2-20260716T135135197200Z/artifacts/artifacts/flashattn-kv-full-v2-tp4/raw/flashattn-tp4.json": "84eef31bcad0f556907a093318a420959d14fdc94474823d11f659704bdfec73",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-frontier-linear-full-20260716-v2-max-tokens-20260716T144444676943Z/artifacts/artifacts/frontier-linear-full-v2/profiles/compute/h20/qwen3-a3b-30b-moe/linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-moe-full-20260716-v1-local-shard-20260716T141334565164Z/artifacts/artifacts/moe-full-v1/raw/moe-full.json": "588f6ad0d69c9636d1b852e3df0a12d13cfe731f050ea7ec7aea457cceefbde8",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-router-full-20260716-v3-tp-context-20260716T145446098505Z/artifacts/artifacts/router-full-v3/raw/router.json": "1962972e983bff3e06a721ef4ae4ec65728ff669681497a4a7e7f769b88b4931"
|
||||
},
|
||||
"outputs": {
|
||||
"allreduce.json": "b38d14f990578d668523d25b107aceed433da5020d8ada3b6e44d3562261a3b3",
|
||||
"attention.csv": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d",
|
||||
"attention_true_mixed_fused.csv": "43ef4be90bddc9aeac6dbbe339feec24162cd1f2129a08fbd959e6ee4eaf5f60",
|
||||
"linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"moe.csv": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0"
|
||||
},
|
||||
"profile_id": "qwen3-30b-a3b-bf16-vllm020-h20-tp1-2-4-fused-mixed-total-conserving",
|
||||
"projection_contract": {
|
||||
"allreduce": "Frozen exact runtime measurements; base profile-only comparison keeps the historical Frontier CC backend fixed to isolate compute profile fidelity",
|
||||
"attention": "Pure prefill/extend/decode FA3 core plus separately measured KV update; input/output reshape assumed zero; exported mean is used as median target; true mixed rows use a total-conserving compatibility projection",
|
||||
"attention_true_mixed": "The directly measured fused total is preserved in diagnostics. Frontier's two targets are projected by the same-TP pure prefill/decode reference ratio, with projected prefill + decode exactly equal to the fused total; the split is a schema compatibility attribution, not an observation",
|
||||
"linear": "Frontier profiler using vLLM 0.20 CUDA operators",
|
||||
"moe": "Replicated gate and fused top-k plus TP-local modular expert kernel; expert measurement already includes prepare/finalize so shuffling is zero"
|
||||
},
|
||||
"row_counts": {
|
||||
"allreduce": 24,
|
||||
"attention_frontier_compatible": 132,
|
||||
"attention_true_mixed_fused_diagnostic": 30,
|
||||
"linear": 36,
|
||||
"moe": 72
|
||||
},
|
||||
"schema_version": "frontier_qwen30_vllm020_frozen_profile.v2"
|
||||
}
|
||||
},
|
||||
"root": "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/frozen/profile-v2",
|
||||
"sha256": {
|
||||
"attention": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d",
|
||||
"linear": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"manifest": "af40545e75aff55c6333cd2d5379ccf042a5a0b7d7fc7df4f745ce256cb290eb",
|
||||
"moe": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0"
|
||||
}
|
||||
},
|
||||
"schema": "frontier-qwen30-prefill-surface-v1",
|
||||
"status": "partial_not_decision_bearing"
|
||||
}
|
||||
@@ -0,0 +1,647 @@
|
||||
{
|
||||
"capacity": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 8.0,
|
||||
"upper_censored": false
|
||||
}
|
||||
],
|
||||
"config_results": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 264.1678559780121,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.032196934357252,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 10.15904188156128,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.00276032449276,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 10.447325944900513,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 13.174042777348715,
|
||||
"ttft_max_ms": 1048.2973400621365,
|
||||
"ttft_p50_ms": 643.9195307403329,
|
||||
"ttft_p95_ms": 962.8278877316862
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 10.053350687026978,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 2851.9297930156335,
|
||||
"ttft_p50_ms": 1510.0519836938302,
|
||||
"ttft_p95_ms": 2758.1797930156335
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp2_mns8",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.744416952133179,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.265625,
|
||||
"passed": 17,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 3773.8047930156335,
|
||||
"ttft_p50_ms": 1995.3684369244447,
|
||||
"ttft_p95_ms": 3622.636520081023
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.402071952819824,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.032196934357252,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.398229360580444,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.00276032449276,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.70562195777893,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 13.174042777348715,
|
||||
"ttft_max_ms": 1048.2973400621365,
|
||||
"ttft_p50_ms": 643.9195307403329,
|
||||
"ttft_p95_ms": 962.8278877316862
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.598989009857178,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 2851.9297930156335,
|
||||
"ttft_p50_ms": 1510.0519836938302,
|
||||
"ttft_p95_ms": 2758.1797930156335
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp2_mns16",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.697861671447754,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.265625,
|
||||
"passed": 17,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 3773.8047930156335,
|
||||
"ttft_p50_ms": 1995.3684369244447,
|
||||
"ttft_p95_ms": 3622.636520081023
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.397430419921875,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.032196934357252,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.3956880569458,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.00276032449276,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.647802591323853,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 13.174042777348715,
|
||||
"ttft_max_ms": 1048.2973400621365,
|
||||
"ttft_p50_ms": 643.9195307403329,
|
||||
"ttft_p95_ms": 962.8278877316862
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.694743156433105,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 2851.9297930156335,
|
||||
"ttft_p50_ms": 1510.0519836938302,
|
||||
"ttft_p95_ms": 2758.1797930156335
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp2_mns32",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.594672203063965,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.265625,
|
||||
"passed": 17,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 3773.8047930156335,
|
||||
"ttft_p50_ms": 1995.3684369244447,
|
||||
"ttft_p95_ms": 3622.636520081023
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.398636817932129,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.032196934357252,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.495105981826782,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.00276032449276,
|
||||
"ttft_max_ms": 122.24062760264154,
|
||||
"ttft_p50_ms": 122.24062760264154,
|
||||
"ttft_p95_ms": 122.24062760264154
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.54387378692627,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 13.174042777348715,
|
||||
"ttft_max_ms": 1048.2973400621365,
|
||||
"ttft_p50_ms": 643.9195307403329,
|
||||
"ttft_p95_ms": 962.8278877316862
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.192227840423584,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 2851.9297930156335,
|
||||
"ttft_p50_ms": 1510.0519836938302,
|
||||
"ttft_p95_ms": 2758.1797930156335
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp2_mns64",
|
||||
"tp": 2
|
||||
},
|
||||
"elapsed_seconds": 9.294084072113037,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 32.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.265625,
|
||||
"passed": 17,
|
||||
"throughput_requests_per_second": 13.539443740959392,
|
||||
"ttft_max_ms": 3773.8047930156335,
|
||||
"ttft_p50_ms": 1995.3684369244447,
|
||||
"ttft_p95_ms": 3622.636520081023
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"contract": {
|
||||
"arrival": "open_loop_uniform",
|
||||
"input_tokens": 2048,
|
||||
"output_tokens": 1,
|
||||
"prefix_caching": false,
|
||||
"rates": [
|
||||
4.0,
|
||||
8.0,
|
||||
16.0,
|
||||
32.0,
|
||||
64.0
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"target_pass_rate": 0.95,
|
||||
"ttft_slo_ms": 1256.0
|
||||
},
|
||||
"frontier": {
|
||||
"git_head": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"git_status_short": " M frontier/config/config.py\n M frontier/entities/request.py\n M frontier/events/cluster_schedule_event.py\n M frontier/execution_time_predictor/sklearn_execution_time_predictor.py\n M frontier/metrics/constants.py\n M frontier/metrics/metrics_store.py\n M frontier/profiling/common/layers/rotary_embedding.py\n M frontier/profiling/moe/moe_impl.py\n M frontier/profiling/moe/moe_vllm_kernel.py\n M frontier/scheduler/cluster_scheduler/__init__.py\n M frontier/scheduler/cluster_scheduler/base_cluster_scheduler.py\n M frontier/scheduler/cluster_scheduler/cluster_scheduler_registry.py\n M frontier/scheduler/cluster_scheduler/sticky_lor_cluster_scheduler.py\n M frontier/scheduler/replica_scheduler/base_replica_scheduler.py\n M frontier/scheduler/replica_scheduler/vllm_v1_engine_replica_scheduler.py\n M frontier/scheduler/replica_stage_scheduler/replica_stage_schduler.py\n M frontier/simulator.py\n M frontier/types/cluster_scheduler_type.py\n?? data/profiling/compute/h20/\n?? frontier/scheduler/cluster_scheduler/prefix_lor_cluster_scheduler.py\n?? runs/\n?? tests/unit/test_attn_prefill_prediction_fallback.py\n",
|
||||
"source": "/tmp/replayserve-frontier-rs1b"
|
||||
},
|
||||
"profiles": {
|
||||
"coverage": {
|
||||
"attention": {
|
||||
"1": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
},
|
||||
"2": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
},
|
||||
"4": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
}
|
||||
},
|
||||
"manifest": {
|
||||
"attention_tp_coverage": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
],
|
||||
"environment_contract": {
|
||||
"dtype": "bfloat16",
|
||||
"frontier_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"hardware": "NVIDIA H20",
|
||||
"model": "Qwen3-30B-A3B",
|
||||
"tensor_parallel_sizes": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
],
|
||||
"vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
|
||||
"vllm_version": "0.20.0"
|
||||
},
|
||||
"inputs": {
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp2-20260716-v1-dispatch-aware-20260716T140743025781Z/artifacts/artifacts/allreduce-full-tp2-v1/raw/allreduce-tp2.json": "97c3c76b5a04e95bd9192423c2b891667c668f39cc0dfecbd097d749939f2d0a",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp4-20260716-v1-dispatch-aware-20260716T141106009788Z/artifacts/artifacts/allreduce-full-tp4-v1/raw/allreduce-tp4.json": "809df9baa6f468cf12bf0c99827475acc67894dd9f3f948976590b665fac0e76",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp1-20260716-v2-20260716T135132587012Z/artifacts/artifacts/flashattn-kv-full-v2-tp1/raw/flashattn-tp1.json": "dcb4c1bf7e76b9c765f78ddd2b8a734f2d7ba2adac13ce017689a8a77fe69a27",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp2-20260716-v2-20260716T135134194295Z/artifacts/artifacts/flashattn-kv-full-v2-tp2/raw/flashattn-tp2.json": "43ce042556ba887c8860614b43ccf0f564e5cebc1a0cffbce299d0acb9fa8d07",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp4-20260716-v2-20260716T135135197200Z/artifacts/artifacts/flashattn-kv-full-v2-tp4/raw/flashattn-tp4.json": "84eef31bcad0f556907a093318a420959d14fdc94474823d11f659704bdfec73",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-frontier-linear-full-20260716-v2-max-tokens-20260716T144444676943Z/artifacts/artifacts/frontier-linear-full-v2/profiles/compute/h20/qwen3-a3b-30b-moe/linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-moe-full-20260716-v1-local-shard-20260716T141334565164Z/artifacts/artifacts/moe-full-v1/raw/moe-full.json": "588f6ad0d69c9636d1b852e3df0a12d13cfe731f050ea7ec7aea457cceefbde8",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-router-full-20260716-v3-tp-context-20260716T145446098505Z/artifacts/artifacts/router-full-v3/raw/router.json": "1962972e983bff3e06a721ef4ae4ec65728ff669681497a4a7e7f769b88b4931"
|
||||
},
|
||||
"outputs": {
|
||||
"allreduce.json": "b38d14f990578d668523d25b107aceed433da5020d8ada3b6e44d3562261a3b3",
|
||||
"attention.csv": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d",
|
||||
"attention_true_mixed_fused.csv": "43ef4be90bddc9aeac6dbbe339feec24162cd1f2129a08fbd959e6ee4eaf5f60",
|
||||
"linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"moe.csv": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0"
|
||||
},
|
||||
"profile_id": "qwen3-30b-a3b-bf16-vllm020-h20-tp1-2-4-fused-mixed-total-conserving",
|
||||
"projection_contract": {
|
||||
"allreduce": "Frozen exact runtime measurements; base profile-only comparison keeps the historical Frontier CC backend fixed to isolate compute profile fidelity",
|
||||
"attention": "Pure prefill/extend/decode FA3 core plus separately measured KV update; input/output reshape assumed zero; exported mean is used as median target; true mixed rows use a total-conserving compatibility projection",
|
||||
"attention_true_mixed": "The directly measured fused total is preserved in diagnostics. Frontier's two targets are projected by the same-TP pure prefill/decode reference ratio, with projected prefill + decode exactly equal to the fused total; the split is a schema compatibility attribution, not an observation",
|
||||
"linear": "Frontier profiler using vLLM 0.20 CUDA operators",
|
||||
"moe": "Replicated gate and fused top-k plus TP-local modular expert kernel; expert measurement already includes prepare/finalize so shuffling is zero"
|
||||
},
|
||||
"row_counts": {
|
||||
"allreduce": 24,
|
||||
"attention_frontier_compatible": 132,
|
||||
"attention_true_mixed_fused_diagnostic": 30,
|
||||
"linear": 36,
|
||||
"moe": 72
|
||||
},
|
||||
"schema_version": "frontier_qwen30_vllm020_frozen_profile.v2"
|
||||
}
|
||||
},
|
||||
"root": "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/frozen/profile-v2",
|
||||
"sha256": {
|
||||
"attention": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d",
|
||||
"linear": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"manifest": "af40545e75aff55c6333cd2d5379ccf042a5a0b7d7fc7df4f745ce256cb290eb",
|
||||
"moe": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0"
|
||||
}
|
||||
},
|
||||
"schema": "frontier-qwen30-prefill-surface-v1",
|
||||
"status": "partial_not_decision_bearing"
|
||||
}
|
||||
@@ -0,0 +1,647 @@
|
||||
{
|
||||
"capacity": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 4.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 4.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 4.0,
|
||||
"upper_censored": false
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"lower_censored": false,
|
||||
"maximum_tested_feasible_request_rate": 16.0,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": 4.0,
|
||||
"upper_censored": false
|
||||
}
|
||||
],
|
||||
"config_results": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 244.95408463478088,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 1.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.039607508408214,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685405,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.85124945640564,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.032004183533672,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685495,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.70260500907898,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 15.412041282697286,
|
||||
"ttft_max_ms": 314.4210260695206,
|
||||
"ttft_p50_ms": 239.37774488651664,
|
||||
"ttft_p95_ms": 309.403713596319
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.855336904525757,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.625,
|
||||
"passed": 40,
|
||||
"throughput_requests_per_second": 16.898601280793688,
|
||||
"ttft_max_ms": 1850.42236218991,
|
||||
"ttft_p50_ms": 1017.6783445024347,
|
||||
"ttft_p95_ms": 1787.92236218991
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 8,
|
||||
"name": "tp4_mns8",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.947782754898071,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 17.184897883039092,
|
||||
"ttft_max_ms": 2771.0751922610093,
|
||||
"ttft_p50_ms": 1468.937931419684,
|
||||
"ttft_p95_ms": 2661.056949107159
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.648890733718872,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 1.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.039607508408214,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685405,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.353798151016235,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.032004183533672,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685495,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.39468502998352,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 15.412041282697286,
|
||||
"ttft_max_ms": 314.4210260695206,
|
||||
"ttft_p50_ms": 239.37774488651664,
|
||||
"ttft_p95_ms": 309.403713596319
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.29756498336792,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.625,
|
||||
"passed": 40,
|
||||
"throughput_requests_per_second": 16.898601280793688,
|
||||
"ttft_max_ms": 1850.42236218991,
|
||||
"ttft_p50_ms": 1017.6783445024347,
|
||||
"ttft_p95_ms": 1787.92236218991
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 16,
|
||||
"name": "tp4_mns16",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.34343934059143,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 17.184897883039092,
|
||||
"ttft_max_ms": 2771.0751922610093,
|
||||
"ttft_p50_ms": 1468.937931419684,
|
||||
"ttft_p95_ms": 2661.056949107159
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.296925783157349,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 1.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.039607508408214,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685405,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.345365285873413,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.032004183533672,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685495,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.350512504577637,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 15.412041282697286,
|
||||
"ttft_max_ms": 314.4210260695206,
|
||||
"ttft_p50_ms": 239.37774488651664,
|
||||
"ttft_p95_ms": 309.403713596319
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.300485372543335,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.625,
|
||||
"passed": 40,
|
||||
"throughput_requests_per_second": 16.898601280793688,
|
||||
"ttft_max_ms": 1850.42236218991,
|
||||
"ttft_p50_ms": 1017.6783445024347,
|
||||
"ttft_p95_ms": 1787.92236218991
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 32,
|
||||
"name": "tp4_mns32",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.292456150054932,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 17.184897883039092,
|
||||
"ttft_max_ms": 2771.0751922610093,
|
||||
"ttft_p50_ms": 1468.937931419684,
|
||||
"ttft_p95_ms": 2661.056949107159
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"loads": [
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.294773817062378,
|
||||
"offered_request_rate": 4.0,
|
||||
"offered_request_rate_per_gpu": 1.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 4.039607508408214,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685405,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.34990382194519,
|
||||
"offered_request_rate": 8.0,
|
||||
"offered_request_rate_per_gpu": 2.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 8.032004183533672,
|
||||
"ttft_max_ms": 93.12333977685495,
|
||||
"ttft_p50_ms": 93.12333977685495,
|
||||
"ttft_p95_ms": 93.12333977685495
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.346213579177856,
|
||||
"offered_request_rate": 16.0,
|
||||
"offered_request_rate_per_gpu": 4.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112",
|
||||
"score": {
|
||||
"feasible": true,
|
||||
"pass_rate": 1.0,
|
||||
"passed": 64,
|
||||
"throughput_requests_per_second": 15.412041282697286,
|
||||
"ttft_max_ms": 314.4210260695206,
|
||||
"ttft_p50_ms": 239.37774488651664,
|
||||
"ttft_p95_ms": 309.403713596319
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.498731136322021,
|
||||
"offered_request_rate": 32.0,
|
||||
"offered_request_rate_per_gpu": 8.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.625,
|
||||
"passed": 40,
|
||||
"throughput_requests_per_second": 16.898601280793688,
|
||||
"ttft_max_ms": 1850.42236218991,
|
||||
"ttft_p50_ms": 1017.6783445024347,
|
||||
"ttft_p95_ms": 1787.92236218991
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1"
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mns": 64,
|
||||
"name": "tp4_mns64",
|
||||
"tp": 4
|
||||
},
|
||||
"elapsed_seconds": 9.300755977630615,
|
||||
"offered_request_rate": 64.0,
|
||||
"offered_request_rate_per_gpu": 16.0,
|
||||
"request_count": 64,
|
||||
"request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0",
|
||||
"score": {
|
||||
"feasible": false,
|
||||
"pass_rate": 0.390625,
|
||||
"passed": 25,
|
||||
"throughput_requests_per_second": 17.184897883039092,
|
||||
"ttft_max_ms": 2771.0751922610093,
|
||||
"ttft_p50_ms": 1468.937931419684,
|
||||
"ttft_p95_ms": 2661.056949107159
|
||||
},
|
||||
"status": "completed",
|
||||
"trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"contract": {
|
||||
"arrival": "open_loop_uniform",
|
||||
"input_tokens": 2048,
|
||||
"output_tokens": 1,
|
||||
"prefix_caching": false,
|
||||
"rates": [
|
||||
4.0,
|
||||
8.0,
|
||||
16.0,
|
||||
32.0,
|
||||
64.0
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"target_pass_rate": 0.95,
|
||||
"ttft_slo_ms": 1256.0
|
||||
},
|
||||
"frontier": {
|
||||
"git_head": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"git_status_short": " M frontier/config/config.py\n M frontier/entities/request.py\n M frontier/events/cluster_schedule_event.py\n M frontier/execution_time_predictor/sklearn_execution_time_predictor.py\n M frontier/metrics/constants.py\n M frontier/metrics/metrics_store.py\n M frontier/profiling/common/layers/rotary_embedding.py\n M frontier/profiling/moe/moe_impl.py\n M frontier/profiling/moe/moe_vllm_kernel.py\n M frontier/scheduler/cluster_scheduler/__init__.py\n M frontier/scheduler/cluster_scheduler/base_cluster_scheduler.py\n M frontier/scheduler/cluster_scheduler/cluster_scheduler_registry.py\n M frontier/scheduler/cluster_scheduler/sticky_lor_cluster_scheduler.py\n M frontier/scheduler/replica_scheduler/base_replica_scheduler.py\n M frontier/scheduler/replica_scheduler/vllm_v1_engine_replica_scheduler.py\n M frontier/scheduler/replica_stage_scheduler/replica_stage_schduler.py\n M frontier/simulator.py\n M frontier/types/cluster_scheduler_type.py\n?? data/profiling/compute/h20/\n?? frontier/scheduler/cluster_scheduler/prefix_lor_cluster_scheduler.py\n?? runs/\n?? tests/unit/test_attn_prefill_prediction_fallback.py\n",
|
||||
"source": "/tmp/replayserve-frontier-rs1b"
|
||||
},
|
||||
"profiles": {
|
||||
"coverage": {
|
||||
"attention": {
|
||||
"1": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
},
|
||||
"2": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
},
|
||||
"4": {
|
||||
"exact_prefill_2048_rows": 1,
|
||||
"profile_batch_size": 1
|
||||
}
|
||||
},
|
||||
"manifest": {
|
||||
"attention_tp_coverage": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
],
|
||||
"environment_contract": {
|
||||
"dtype": "bfloat16",
|
||||
"frontier_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"hardware": "NVIDIA H20",
|
||||
"model": "Qwen3-30B-A3B",
|
||||
"tensor_parallel_sizes": [
|
||||
1,
|
||||
2,
|
||||
4
|
||||
],
|
||||
"vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
|
||||
"vllm_version": "0.20.0"
|
||||
},
|
||||
"inputs": {
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp2-20260716-v1-dispatch-aware-20260716T140743025781Z/artifacts/artifacts/allreduce-full-tp2-v1/raw/allreduce-tp2.json": "97c3c76b5a04e95bd9192423c2b891667c668f39cc0dfecbd097d749939f2d0a",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp4-20260716-v1-dispatch-aware-20260716T141106009788Z/artifacts/artifacts/allreduce-full-tp4-v1/raw/allreduce-tp4.json": "809df9baa6f468cf12bf0c99827475acc67894dd9f3f948976590b665fac0e76",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp1-20260716-v2-20260716T135132587012Z/artifacts/artifacts/flashattn-kv-full-v2-tp1/raw/flashattn-tp1.json": "dcb4c1bf7e76b9c765f78ddd2b8a734f2d7ba2adac13ce017689a8a77fe69a27",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp2-20260716-v2-20260716T135134194295Z/artifacts/artifacts/flashattn-kv-full-v2-tp2/raw/flashattn-tp2.json": "43ce042556ba887c8860614b43ccf0f564e5cebc1a0cffbce299d0acb9fa8d07",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp4-20260716-v2-20260716T135135197200Z/artifacts/artifacts/flashattn-kv-full-v2-tp4/raw/flashattn-tp4.json": "84eef31bcad0f556907a093318a420959d14fdc94474823d11f659704bdfec73",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-frontier-linear-full-20260716-v2-max-tokens-20260716T144444676943Z/artifacts/artifacts/frontier-linear-full-v2/profiles/compute/h20/qwen3-a3b-30b-moe/linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-moe-full-20260716-v1-local-shard-20260716T141334565164Z/artifacts/artifacts/moe-full-v1/raw/moe-full.json": "588f6ad0d69c9636d1b852e3df0a12d13cfe731f050ea7ec7aea457cceefbde8",
|
||||
"/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-router-full-20260716-v3-tp-context-20260716T145446098505Z/artifacts/artifacts/router-full-v3/raw/router.json": "1962972e983bff3e06a721ef4ae4ec65728ff669681497a4a7e7f769b88b4931"
|
||||
},
|
||||
"outputs": {
|
||||
"allreduce.json": "b38d14f990578d668523d25b107aceed433da5020d8ada3b6e44d3562261a3b3",
|
||||
"attention.csv": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d",
|
||||
"attention_true_mixed_fused.csv": "43ef4be90bddc9aeac6dbbe339feec24162cd1f2129a08fbd959e6ee4eaf5f60",
|
||||
"linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"moe.csv": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0"
|
||||
},
|
||||
"profile_id": "qwen3-30b-a3b-bf16-vllm020-h20-tp1-2-4-fused-mixed-total-conserving",
|
||||
"projection_contract": {
|
||||
"allreduce": "Frozen exact runtime measurements; base profile-only comparison keeps the historical Frontier CC backend fixed to isolate compute profile fidelity",
|
||||
"attention": "Pure prefill/extend/decode FA3 core plus separately measured KV update; input/output reshape assumed zero; exported mean is used as median target; true mixed rows use a total-conserving compatibility projection",
|
||||
"attention_true_mixed": "The directly measured fused total is preserved in diagnostics. Frontier's two targets are projected by the same-TP pure prefill/decode reference ratio, with projected prefill + decode exactly equal to the fused total; the split is a schema compatibility attribution, not an observation",
|
||||
"linear": "Frontier profiler using vLLM 0.20 CUDA operators",
|
||||
"moe": "Replicated gate and fused top-k plus TP-local modular expert kernel; expert measurement already includes prepare/finalize so shuffling is zero"
|
||||
},
|
||||
"row_counts": {
|
||||
"allreduce": 24,
|
||||
"attention_frontier_compatible": 132,
|
||||
"attention_true_mixed_fused_diagnostic": 30,
|
||||
"linear": 36,
|
||||
"moe": 72
|
||||
},
|
||||
"schema_version": "frontier_qwen30_vllm020_frozen_profile.v2"
|
||||
}
|
||||
},
|
||||
"root": "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/frozen/profile-v2",
|
||||
"sha256": {
|
||||
"attention": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d",
|
||||
"linear": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c",
|
||||
"manifest": "af40545e75aff55c6333cd2d5379ccf042a5a0b7d7fc7df4f745ce256cb290eb",
|
||||
"moe": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0"
|
||||
}
|
||||
},
|
||||
"schema": "frontier-qwen30-prefill-surface-v1",
|
||||
"status": "partial_not_decision_bearing"
|
||||
}
|
||||
39
runs/frontier-phase-factorial-v0/test_phase_factorial.py
Normal file
39
runs/frontier-phase-factorial-v0/test_phase_factorial.py
Normal file
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
ROOT = Path(__file__).parent
|
||||
|
||||
|
||||
def load(name: str):
|
||||
path = ROOT / name
|
||||
spec = importlib.util.spec_from_file_location(path.stem, path)
|
||||
assert spec and spec.loader
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def test_percentile() -> None:
|
||||
client = load("qwen30_prefill_client.py")
|
||||
assert client.percentile([4.0, 1.0, 3.0, 2.0], 0.50) == 2.0
|
||||
assert client.percentile([4.0, 1.0, 3.0, 2.0], 0.95) == 4.0
|
||||
assert client.percentile([], 0.95) is None
|
||||
|
||||
|
||||
def test_grid_and_trace(tmp_path: Path) -> None:
|
||||
surface = load("run_frontier_qwen30_prefill_surface.py")
|
||||
assert len(surface.GRID) == 12
|
||||
assert {config.tp for config in surface.GRID} == {1, 2, 4}
|
||||
trace = tmp_path / "trace.csv"
|
||||
surface.write_trace(trace, requests=3, rate=4.0)
|
||||
lines = trace.read_text().splitlines()
|
||||
assert len(lines) == 4
|
||||
assert lines[1].split(",")[:3] == ["0.000000000000", "2048", "1"]
|
||||
assert lines[3].split(",")[:3] == ["0.500000000000", "2048", "1"]
|
||||
Reference in New Issue
Block a user