diff --git a/runs/frontier-phase-factorial-v0/experiment-card.md b/runs/frontier-phase-factorial-v0/experiment-card.md new file mode 100644 index 0000000..f291ab7 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/experiment-card.md @@ -0,0 +1,54 @@ +# 实验 EXP-SIMFID-PHASE-FACTORIAL:prefill-only 是否是 simulator ranking 的容易区间? + +> **状态:** 已批准,运行中(用户于 2026-07-17 明确要求先完成 235B mixed 与 30B prefill-only) + +## Claim 与决策 + +- **Parent claim:** Frontier 的 config-ranking fidelity 由 workload execution phase 决定;prefill-only 可能比 decode/mixed 更容易由 isolated operator profiles 组合。 +- **目的:** 用跨 model 的 phase factorial 区分 phase-complexity explanation 与 model/runtime/profile-specific explanation。 +- **Competing hypotheses:** H-phase:prefill-only 是低状态反馈的 compatibility envelope,因此 30B prefill-only 也能达到低-regret ranking,而 decode/mixed 更容易失真;H-stack:30B 的失败主要来自 FA3/CUDA-graph/routing/profile composition 等 stack-specific mismatch,因此即使 prefill-only 也可能失败;H-margin:mixed 的 differential 可以错误,但只要 topology margin 足够仍会选对 top set。 +- **事前预测:** 若 H-phase 成立,30B prefill-only 满足 regret ≤5%、Kendall τ-b ≥0.8,并明显优于同模型 mixed;若 H-stack 成立,30B prefill-only 仍过不了 gate;若 H-margin 成立,235B mixed 可保持 top set,但会漏掉 MNS/MBT pair directions。 +- **判定规则:** 30B prefill-only fail → “prefill-only 是充分条件”被否证,下一步优先做 same-model execution-context ablation;30B prefill-only pass 且新增 decode-heavy mixed fail → 支持 phase hypothesis;30B prefill-only 与 235B mixed 都 pass → phase 不能单独解释,转向 margin-aware compatibility envelope。 + +## Setup + +- **自变量:** model×phase:已有 Qwen3-235B-A22B-FP8 mixed;新增 Qwen3-30B-A3B BF16 prefill-only。 +- **控制变量:** dash0 H20、community vLLM 与各自 frozen Frontier profile、同一 config 内 real/sim 的 request shape、arrival lattice、SLO、prefix policy、MNS/MBT/TP 与随机种子。 +- **30B system context:** community vLLM 0.20.0+cu129,BF16 weights/activation/KV,TP∈{1,2,4},MNS∈{8,16,32,64},MBT=8192,chunked prefill on,prefix off;real 保留 runtime 默认 CUDA graph,Frontier profile-only 不做 E2E calibration。 +- **30B workload:** fixed ISL=2048、OSL=1,64 个不同 token-chain prompts,uniform open-loop QPS;fresh server per `(config, rate, round)`,target-rate warmup 与 measured requests 分离。 +- **30B SLO:** TTFT≤1256 ms,至少 61/64 requests 通过;primary score 为最大共同 tested feasible req/s / 实际 TP GPUs。 +- **235B baseline:** 已冻结 `ISL=2048, OSL=128`、8 configs、68 个 fresh-server anchors;primary sensitivity TTFT≤1256 ms、TPOT≤150 ms。 +- **Baselines:** real community vLLM;Frontier same-stack profile-only;historical Qwen30 mixed profile-only;historical frozen per-TP calibration 只作为 upper bound,不参与本 case 拟合。 +- **Metrics:** top set、worst tie-break regret、Kendall τ-b、exact/non-tied pair direction、anchor confusion、absolute capacity、TTFT p50/p95、real trial variance与GPU-hour。 + +## 预期产物与 review + +- **预期数据:** 30B frozen simulator surface;real config-rate anchors;两模型 phase comparison table;failure mechanism breakdown。 +- **Figure prototype:** `mock-phase-factorial.png`;x=model×phase,左轴=worst regret,右侧 annotation=τ-b;虚线是 5% regret gate。mock 只表达可区分趋势,不进入结论。 +- **人工 review:** 已批准。用户要求 smoke 通过后推进实验;先做这两个 case,再根据 hypothesis verdict 扩展。 +- **Review 意见:** 不把 existing 235B mixed top-set match 隐藏掉;不把“全 config 并列”算作成功 hit;30B prefill-only 必须使用相同 primary ranking objective。 + +## Benchmark design audit + +| Risk | Verdict | 处理 | +|---|---|---| +| Selective benchmarking | PASS for initial screen | 同时报已有 235B mixed success和内部 pairwise failure;后续 expansion 由预注册 verdict 触发 | +| Simplified workload | NEEDS EVIDENCE | fixed-shape 只用于 phase isolation,不外推 trace-faithful mixed | +| Calibration=evaluation | PASS | 新 case 不用 serving E2E 数据拟合 scale | +| Missing significance | NEEDS EVIDENCE until run | boundary anchors做独立 fresh-server repeat,保留 disagreement | +| Relative-only result | PASS by design | 同时报 req/s/GPU、TTFT distribution、rank/regret | + +## 复现信息 + +- **Code:** AITuner branch `codex/fidelity-prefix-pilot-20260714`;Frontier upstream `d9cfeb6d8791fbf2f295dd9744c56a666171776e` + frozen known patches。 +- **Environment:** 只使用 dash0 8×H20;Qwen30 venv `/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1`;model `/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B`。 +- **产物路径:** local/remote `runs/frontier-phase-factorial-v0/`;raw GPU artifacts 由 fleet harvest,condensed JSON/CSV 进入结果目录。 +- **已知 deviation:** 235B 为 FP8/vLLM0.10.2/FlashInfer eager,30B 为 BF16/vLLM0.20/FA3/default CUDA graph;因此跨模型只检验 hypothesis consistency,causal phase claim 最终仍需 same-model phase pair。 + +## 结果 + +- **观察事实:** 235B fixed-shape mixed 已完成:real/sim TP4 top set exact match,worst regret=0、τ-b=0.8944;但 20 个 real non-tie pairs 只保持 16 个,10/34 anchors false-infeasible,TP8 MNS×MBT interaction 被漏掉。30B prefill-only 待运行。 +- **异常:** 无。 +- **Interpretation 与剩余 alternatives:** 强版本“mixed 必然失败”已被 235B top-set result 削弱;仍可能存在 phase-dependent error magnitude,由 topology margin 掩盖。 +- **Claim update:** unchanged,等待 30B prefill-only。 +- **下一步:** freeze 30B simulator surface → guided real anchors → joint verdict;只有判定需要时扩展 decode-heavy 235B 或 trace-shaped 30B prefill。 diff --git a/runs/frontier-phase-factorial-v0/fleet.toml b/runs/frontier-phase-factorial-v0/fleet.toml new file mode 100644 index 0000000..76a9e93 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/fleet.toml @@ -0,0 +1,24 @@ +version = 1 + +[paths] +state_dir = "runs/frontier-phase-factorial-v0/fleet-state" +artifacts_dir = "runs/frontier-phase-factorial-v0/fleet-artifacts" + +[ssh] +connect_timeout_sec = 10 + +[scheduler] +gpu_free_memory_mb = 1024 +gpu_free_utilization_pct = 10 +prefer_pack = true + +[sync] +mode = "scp" +local_path = "runs/frontier-phase-factorial-v0/remote-sync-marker" + +[[hosts]] +name = "dash0" +ssh_alias = "dash0" +enabled = true +sync_remote_path = "/home/admin/cpfs/wjh/aituner/phase-factorial-sync-marker" +fleet_root = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0" diff --git a/runs/frontier-phase-factorial-v0/jobs_full.toml b/runs/frontier-phase-factorial-v0/jobs_full.toml new file mode 100644 index 0000000..c67f255 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/jobs_full.toml @@ -0,0 +1,241 @@ +version = 1 + +[[jobs]] +name = "qwen30-prefill-real-tp1-mns8-20260717-v1" +gpus = 1 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp1-mns8-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "1" +MNS = "8" +RATES = "4 8 16 32 64" +SERVER_PORT = "8720" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns8-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp1-mns16-20260717-v1" +gpus = 1 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp1-mns16-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "1" +MNS = "16" +RATES = "4 8 16 32 64" +SERVER_PORT = "8721" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns16-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp1-mns32-20260717-v1" +gpus = 1 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp1-mns32-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "1" +MNS = "32" +RATES = "4 8 16 32 64" +SERVER_PORT = "8722" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns32-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp1-mns64-20260717-v1" +gpus = 1 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp1-mns64-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "1" +MNS = "64" +RATES = "4 8 16 32 64" +SERVER_PORT = "8723" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp1-mns64-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp2-mns8-20260717-v1" +gpus = 2 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp2-mns8-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "2" +MNS = "8" +RATES = "4 8 16 32 64" +SERVER_PORT = "8724" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns8-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp2-mns16-20260717-v1" +gpus = 2 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp2-mns16-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "2" +MNS = "16" +RATES = "4 8 16 32 64" +SERVER_PORT = "8725" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns16-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp2-mns32-20260717-v1" +gpus = 2 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp2-mns32-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "2" +MNS = "32" +RATES = "4 8 16 32 64" +SERVER_PORT = "8726" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns32-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp2-mns64-20260717-v1" +gpus = 2 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp2-mns64-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "2" +MNS = "64" +RATES = "4 8 16 32 64" +SERVER_PORT = "8727" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp2-mns64-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp4-mns8-20260717-v1" +gpus = 4 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp4-mns8-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "4" +MNS = "8" +RATES = "4 8 16 32 64" +SERVER_PORT = "8728" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns8-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp4-mns16-20260717-v1" +gpus = 4 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp4-mns16-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "4" +MNS = "16" +RATES = "4 8 16 32 64" +SERVER_PORT = "8729" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns16-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp4-mns32-20260717-v1" +gpus = 4 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp4-mns32-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "4" +MNS = "32" +RATES = "4 8 16 32 64" +SERVER_PORT = "8730" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns32-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" + +[[jobs]] +name = "qwen30-prefill-real-tp4-mns64-20260717-v1" +gpus = 4 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 7200 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-tp4-mns64-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "4" +MNS = "64" +RATES = "4 8 16 32 64" +SERVER_PORT = "8731" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-tp4-mns64-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" diff --git a/runs/frontier-phase-factorial-v0/jobs_smoke.toml b/runs/frontier-phase-factorial-v0/jobs_smoke.toml new file mode 100644 index 0000000..5741e2d --- /dev/null +++ b/runs/frontier-phase-factorial-v0/jobs_smoke.toml @@ -0,0 +1,21 @@ +version = 1 + +[[jobs]] +name = "qwen30-prefill-real-smoke-tp1-mns8-20260717-v1" +gpus = 1 +gpu_model = "H20" +hosts = ["dash0"] +command = "cd /home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1/runs/frontier-phase-factorial-v0 && timeout --signal=TERM --kill-after=30s 1800 bash run_qwen30_prefill_real_config.sh" +artifacts = ["artifacts/real-smoke-tp1-mns8-v1"] + +[jobs.env] +HOME = "/tmp/wjh" +XDG_CACHE_HOME = "/tmp/wjh/.cache" +VLLM_CACHE_ROOT = "/tmp/wjh/.cache/vllm" +TP = "1" +MNS = "8" +RATES = "4" +SERVER_PORT = "8718" +OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/gpu-fleet-phase-factorial-v0/artifacts/real-smoke-tp1-mns8-v1" +VENV_ROOT = "/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1" +MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B" diff --git a/runs/frontier-phase-factorial-v0/mock-phase-factorial.png b/runs/frontier-phase-factorial-v0/mock-phase-factorial.png new file mode 100644 index 0000000..6c3c2bd Binary files /dev/null and b/runs/frontier-phase-factorial-v0/mock-phase-factorial.png differ diff --git a/runs/frontier-phase-factorial-v0/plot_mock_phase_factorial.py b/runs/frontier-phase-factorial-v0/plot_mock_phase_factorial.py new file mode 100644 index 0000000..fdfc37d --- /dev/null +++ b/runs/frontier-phase-factorial-v0/plot_mock_phase_factorial.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +"""Render the preregistered phase-factorial hypothesis schematic (mock data).""" + +from __future__ import annotations + +from pathlib import Path + +import matplotlib + +matplotlib.use("Agg") +import matplotlib.pyplot as plt + + +def main() -> None: + labels = ("235B\nprefill-only", "235B\nmixed", "30B\nprefill-only", "30B\nmixed") + x = range(len(labels)) + phase_hypothesis = (1.0, 18.0, 2.0, 28.0) + stack_hypothesis = (1.0, 4.0, 24.0, 30.0) + + fig, ax = plt.subplots(figsize=(8.6, 4.8), constrained_layout=True) + ax.plot(x, phase_hypothesis, marker="o", lw=2, label="H-phase (mock)") + ax.plot(x, stack_hypothesis, marker="s", lw=2, label="H-stack (mock)") + ax.axhline(5, color="black", ls="--", lw=1.2, label="5% regret gate") + ax.set_xticks(list(x), labels) + ax.set_ylabel("Worst selected-config regret (%) — MOCK DATA") + ax.set_title("Schematic only: predictions that distinguish phase vs stack explanations") + ax.set_ylim(0, 35) + ax.grid(axis="y", alpha=0.25) + ax.legend() + ax.text( + 0.01, + 0.98, + "MOCK DATA / NOT A RESULT", + transform=ax.transAxes, + va="top", + color="crimson", + weight="bold", + ) + output = Path(__file__).with_name("mock-phase-factorial.png") + fig.savefig(output, dpi=180) + print(output) + + +if __name__ == "__main__": + main() diff --git a/runs/frontier-phase-factorial-v0/qwen30_prefill_client.py b/runs/frontier-phase-factorial-v0/qwen30_prefill_client.py new file mode 100644 index 0000000..de40676 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/qwen30_prefill_client.py @@ -0,0 +1,216 @@ +#!/usr/bin/env python3 +"""Open-loop fixed-shape prefill-only workload for one real offered-load anchor.""" + +from __future__ import annotations + +import argparse +import concurrent.futures +import hashlib +import http.client +import json +import math +import time +from pathlib import Path +from typing import Any + + +TTFT_SLO_MS = 1256.0 +TARGET_PASS_RATE = 0.95 + + +def parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser() + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, required=True) + parser.add_argument("--served-model", required=True) + parser.add_argument("--model-path", type=Path, required=True) + parser.add_argument("--rate", type=float, required=True) + parser.add_argument("--requests", type=int, default=64) + parser.add_argument("--input-tokens", type=int, default=2048) + parser.add_argument("--timeout-seconds", type=float, default=900.0) + parser.add_argument("--output", type=Path, required=True) + return parser.parse_args() + + +def percentile(values: list[float], fraction: float) -> float | None: + if not values: + return None + ordered = sorted(values) + index = min(len(ordered) - 1, max(0, math.ceil(fraction * len(ordered)) - 1)) + return ordered[index] + + +def run_request( + *, + request_index: int, + scheduled_at: float, + benchmark_start: float, + args: argparse.Namespace, + prompt_ids: list[int], +) -> dict[str, Any]: + delay = scheduled_at - time.perf_counter() + if delay > 0: + time.sleep(delay) + admitted = time.perf_counter() + record: dict[str, Any] = { + "request_index": request_index, + "scheduled_s": scheduled_at - benchmark_start, + "admitted_s": admitted - benchmark_start, + "admission_lag_ms": (admitted - scheduled_at) * 1000.0, + "success": False, + } + connection = http.client.HTTPConnection(args.host, args.port, timeout=args.timeout_seconds) + body = { + "model": args.served_model, + "prompt": prompt_ids, + "min_tokens": 1, + "max_tokens": 1, + "ignore_eos": True, + "temperature": 0, + "stream": True, + "stream_options": {"include_usage": True}, + "return_token_ids": True, + } + try: + started = time.perf_counter() + connection.request( + "POST", + "/v1/completions", + body=json.dumps(body, separators=(",", ":")).encode(), + headers={"Content-Type": "application/json"}, + ) + response = connection.getresponse() + if response.status != 200: + raise RuntimeError( + f"HTTP {response.status}: {response.read().decode(errors='replace')}" + ) + first_token_at = None + streamed_tokens = 0 + usage = None + while True: + raw = response.readline() + if not raw: + break + line = raw.decode(errors="replace").strip() + if not line.startswith("data:"): + continue + data = line[5:].strip() + if data == "[DONE]": + break + payload = json.loads(data) + if payload.get("usage"): + usage = payload["usage"] + emitted = 0 + for choice in payload.get("choices") or []: + token_ids = choice.get("token_ids") or [] + emitted += len(token_ids) if token_ids else int(bool(choice.get("text"))) + if emitted: + first_token_at = first_token_at or time.perf_counter() + streamed_tokens += emitted + finished = time.perf_counter() + if first_token_at is None or usage is None: + raise RuntimeError("missing streaming token or usage") + prompt_tokens = int(usage["prompt_tokens"]) + completion_tokens = int(usage["completion_tokens"]) + if prompt_tokens != args.input_tokens or completion_tokens != 1: + raise RuntimeError(f"usage mismatch: {prompt_tokens}+{completion_tokens}") + ttft = (first_token_at - started) * 1000.0 + record.update( + { + "success": True, + "prompt_tokens": prompt_tokens, + "completion_tokens": completion_tokens, + "streamed_token_count": streamed_tokens, + "ttft_ms": ttft, + "e2e_ms": (finished - started) * 1000.0, + "slo_pass": ttft <= TTFT_SLO_MS, + } + ) + except Exception as error: # Failed requests remain in the SLO denominator. + record["error"] = f"{type(error).__name__}: {error}" + record["slo_pass"] = False + finally: + connection.close() + return record + + +def main() -> None: + args = parse_args() + if args.rate <= 0 or args.requests <= 0 or args.input_tokens <= 0: + raise ValueError("rate, requests, and input tokens must be positive") + from transformers import AutoTokenizer + + tokenizer = AutoTokenizer.from_pretrained(args.model_path, trust_remote_code=True) + excluded = set(tokenizer.all_special_ids) + candidates = [ + token_id for token_id in range(tokenizer.vocab_size) if token_id not in excluded + ] + if len(candidates) < args.requests + 1: + raise RuntimeError("tokenizer has too few non-special token IDs") + base_id = candidates[0] + prompts = [ + [candidates[index + 1], *([base_id] * (args.input_tokens - 1))] + for index in range(args.requests) + ] + prompt_hash = hashlib.sha256( + "\n".join(",".join(map(str, prompt)) for prompt in prompts).encode() + ).hexdigest() + + benchmark_start = time.perf_counter() + 2.0 + with concurrent.futures.ThreadPoolExecutor(max_workers=args.requests) as pool: + futures = [ + pool.submit( + run_request, + request_index=index, + scheduled_at=benchmark_start + index / args.rate, + benchmark_start=benchmark_start, + args=args, + prompt_ids=prompts[index], + ) + for index in range(args.requests) + ] + requests = [future.result() for future in futures] + requests.sort(key=lambda row: int(row["request_index"])) + completed = [row for row in requests if row["success"]] + passed = sum(bool(row["slo_pass"]) for row in requests) + ttfts = [float(row["ttft_ms"]) for row in completed] + pass_rate = passed / len(requests) + payload = { + "schema": "qwen30-prefill-rate-anchor-v1", + "workload": { + "offered_request_rate": args.rate, + "request_count": args.requests, + "input_tokens": args.input_tokens, + "output_tokens": 1, + "prefix_caching": False, + "arrival": "open_loop_uniform", + "last_scheduled_arrival_s": (args.requests - 1) / args.rate, + "prompt_vector_sha256": prompt_hash, + }, + "summary": { + "completed": len(completed), + "failed": len(requests) - len(completed), + "ttft_p50_ms": percentile(ttfts, 0.50), + "ttft_p95_ms": percentile(ttfts, 0.95), + "ttft_max_ms": max(ttfts) if ttfts else None, + "admission_lag_max_ms": max( + float(row["admission_lag_ms"]) for row in requests + ), + "slo": { + "ttft_threshold_ms": TTFT_SLO_MS, + "passed": passed, + "pass_rate": pass_rate, + "feasible": pass_rate >= TARGET_PASS_RATE, + }, + }, + "requests": requests, + } + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + print(json.dumps(payload["summary"], sort_keys=True), flush=True) + if len(completed) != args.requests: + raise SystemExit(2) + + +if __name__ == "__main__": + main() diff --git a/runs/frontier-phase-factorial-v0/remote-sync-marker/README.md b/runs/frontier-phase-factorial-v0/remote-sync-marker/README.md new file mode 100644 index 0000000..9dfd754 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/remote-sync-marker/README.md @@ -0,0 +1,6 @@ +# Phase-factorial fleet sync marker + +Experiment source is synchronized through Git into the clean dash0 checkout +`/home/admin/cpfs/wjh/aituner/aituner-qwen30-vllm020-profile-v1`. This small +directory exists only to satisfy the fleet orchestrator's explicit sync phase +without copying local simulator caches or raw metrics to the GPU host. diff --git a/runs/frontier-phase-factorial-v0/run_frontier_qwen30_prefill_surface.py b/runs/frontier-phase-factorial-v0/run_frontier_qwen30_prefill_surface.py new file mode 100644 index 0000000..10249d7 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/run_frontier_qwen30_prefill_surface.py @@ -0,0 +1,400 @@ +#!/usr/bin/env python3 +"""Freeze the Qwen30 fixed-shape prefill-only Frontier surface.""" + +from __future__ import annotations + +import argparse +import csv +import hashlib +import importlib.util +import json +import math +import os +import subprocess +import sys +import time +from dataclasses import asdict, dataclass +from pathlib import Path +from typing import Any + + +MODEL = "qwen3-a3b-30b-moe" +RATES = (4.0, 8.0, 16.0, 32.0, 64.0) +TTFT_SLO_MS = 1256.0 +TARGET_PASS_RATE = 0.95 +NUM_BLOCKS = {1: 20080, 2: 76537, 4: 191727} + + +@dataclass(frozen=True) +class Config: + tp: int + mns: int + + @property + def name(self) -> str: + return f"tp{self.tp}_mns{self.mns}" + + +GRID = tuple(Config(tp, mns) for tp in (1, 2, 4) for mns in (8, 16, 32, 64)) + + +def parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser() + parser.add_argument("--frontier-source", type=Path, required=True) + parser.add_argument("--replayserve-root", type=Path, required=True) + parser.add_argument("--profile-root", type=Path, required=True) + parser.add_argument("--python-deps", type=Path, required=True) + parser.add_argument("--output-root", type=Path, required=True) + parser.add_argument("--requests", type=int, default=64) + parser.add_argument("--rate", type=float, action="append") + parser.add_argument("--config", action="append") + parser.add_argument("--timeout-seconds", type=float, default=900.0) + parser.add_argument("--resume", action="store_true") + return parser.parse_args() + + +def sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as source: + for chunk in iter(lambda: source.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def write_json(path: Path, payload: Any) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_suffix(path.suffix + ".tmp") + temporary.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + os.replace(temporary, path) + + +def load_module(name: str, path: Path): + spec = importlib.util.spec_from_file_location(name, path) + if spec is None or spec.loader is None: + raise ImportError(path) + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +def write_trace(path: Path, *, requests: int, rate: float) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + fields = [ + "arrived_at", + "num_prefill_tokens", + "num_decode_tokens", + "session_id", + "block_hash_ids", + ] + with path.open("w", newline="") as output: + writer = csv.DictWriter(output, fieldnames=fields) + writer.writeheader() + for request_id in range(requests): + writer.writerow( + { + "arrived_at": f"{request_id / rate:.12f}", + "num_prefill_tokens": 2048, + "num_decode_tokens": 1, + "session_id": request_id, + "block_hash_ids": "|".join( + str(request_id * 128 + block + 1) for block in range(128) + ), + } + ) + + +def profile_paths(root: Path) -> dict[str, Path]: + paths = { + "linear": root / "linear_op.csv", + "attention": root / "attention.csv", + "moe": root / "moe.csv", + "manifest": root / "manifest.json", + } + missing = [str(path) for path in paths.values() if not path.is_file()] + if missing: + raise FileNotFoundError(missing) + return paths + + +def validate_profile(paths: dict[str, Path]) -> dict[str, Any]: + manifest = json.loads(paths["manifest"].read_text()) + expected = manifest["outputs"] + for filename in ("linear_op.csv", "attention.csv", "moe.csv"): + actual = sha256(paths[{"linear_op.csv": "linear", "attention.csv": "attention", "moe.csv": "moe"}[filename]]) + if actual != expected[filename]: + raise ValueError(f"profile hash mismatch for {filename}") + with paths["attention"].open(newline="") as source: + rows = list(csv.DictReader(source)) + coverage = {} + for tp in (1, 2, 4): + exact = [ + row + for row in rows + if int(row["num_tensor_parallel_workers"]) == tp + and row["is_prefill"].lower() == "true" + and row.get("is_true_mixed_batch", "").lower() != "true" + and int(float(row["total_tokens"])) == 2048 + ] + if len(exact) != 1: + raise ValueError(f"expected one exact TP{tp} 2048-token prefill row, got {len(exact)}") + coverage[str(tp)] = {"exact_prefill_2048_rows": len(exact), "profile_batch_size": int(exact[0]["batch_size"])} + return {"manifest": manifest, "attention": coverage} + + +def knobs(config: Config, paths: dict[str, Path], cache: Path) -> dict[str, Any]: + return { + "simulation_mode": "online", + "sys_arch": "co-location", + "num_replicas": 1, + "cluster_scheduler": "sticky_round_robin", + "model_name": MODEL, + "device": "h20", + "network_device": "h20_dgx", + "attn_tensor_parallel_size": config.tp, + "attn_data_parallel_size": 1, + "moe_tensor_parallel_size": config.tp, + "moe_expert_parallel_size": 1, + "num_pipeline_stages": 1, + "replica_scheduler": "vllm_v1", + "decode_cuda_graph_mode": "none", + "batch_size_cap": config.mns, + "max_tokens_in_batch": 8192, + "long_prefill_token_threshold": 0, + "block_size": 16, + "num_blocks_mode": "explicit", + "num_blocks": NUM_BLOCKS[config.tp], + "gpu_memory_utilization": 0.92, + "non_kv_cache_overhead_bytes": 0, + "trace_max_tokens": 40960, + "cache_dir": str(cache), + "enable_dummy_mode": False, + "linear_op_input_file": str(paths["linear"]), + "atten_input_file": str(paths["attention"]), + "moe_input_file": str(paths["moe"]), + "prediction_max_prefill_chunk_size": 18000, + "prediction_max_batch_size": 128, + "prediction_max_tokens_per_request": 32768, + "no_cache": False, + "skip_cpu_overhead_modeling": True, + "enable_prefix_caching": False, + "enable_chunked_prefill": True, + } + + +def find_metrics(run_dir: Path) -> tuple[Path, Path]: + systems = list((run_dir / "metrics").rglob("system_metrics.json")) + requests = list((run_dir / "metrics").rglob("request_metrics.csv")) + if len(systems) != 1 or len(requests) != 1: + raise RuntimeError(f"metric pair mismatch: {len(systems)}/{len(requests)}") + return systems[0], requests[0] + + +def score(system_path: Path, request_path: Path, expected_requests: int) -> dict[str, Any]: + system = json.loads(system_path.read_text()) + metadata = system["simulation_metadata"] + if int(metadata["completed_requests"]) != expected_requests: + raise ValueError("Frontier completion count mismatch") + with request_path.open(newline="") as source: + rows = list(csv.DictReader(source)) + if len(rows) != expected_requests: + raise ValueError("request metric count mismatch") + values = [] + passed = 0 + for row in rows: + if int(float(row["request_num_prefill_tokens"])) != 2048: + raise ValueError("prefill shape drift") + if int(float(row["request_num_decode_tokens"])) != 1: + raise ValueError("decode shape drift") + ttft = float(row["ttft"]) + if not math.isfinite(ttft) or ttft < 0: + raise ValueError("invalid TTFT") + values.append(ttft) + passed += int(ttft <= TTFT_SLO_MS) + ordered = sorted(values) + pass_rate = passed / expected_requests + return { + "ttft_p50_ms": ordered[math.ceil(0.50 * len(ordered)) - 1], + "ttft_p95_ms": ordered[math.ceil(0.95 * len(ordered)) - 1], + "ttft_max_ms": max(ordered), + "passed": passed, + "pass_rate": pass_rate, + "feasible": pass_rate >= TARGET_PASS_RATE, + "throughput_requests_per_second": float(system["throughput_metrics"]["requests_per_second"]), + } + + +def main() -> None: + args = parse_args() + args.frontier_source = args.frontier_source.resolve() + args.replayserve_root = args.replayserve_root.resolve() + args.profile_root = args.profile_root.resolve() + args.python_deps = args.python_deps.resolve() + args.output_root = args.output_root.resolve() + rates = tuple(args.rate or RATES) + selected = list(GRID) + if args.config: + wanted = set(args.config) + selected = [config for config in GRID if config.name in wanted] + if {config.name for config in selected} != wanted: + raise ValueError(f"unknown configs: {wanted - {config.name for config in selected}}") + paths = profile_paths(args.profile_root) + coverage = validate_profile(paths) + builder = load_module( + "qwen30_prefill_frontier_builder", + args.replayserve_root / "tools/run_frontier_sweep.py", + ) + frontier_head = subprocess.run( + ["git", "-C", str(args.frontier_source), "rev-parse", "HEAD"], + check=True, + text=True, + stdout=subprocess.PIPE, + ).stdout.strip() + traces = {} + for rate in rates: + trace = args.output_root / "traces" / f"r{rate:g}.csv" + write_trace(trace, requests=args.requests, rate=rate) + traces[rate] = trace + + config_results = [] + for config in selected: + loads = [] + config_knobs = knobs(config, paths, args.output_root / "cache") + for rate in rates: + run_dir = args.output_root / "runs" / config.name / f"r{rate:g}" + result_path = run_dir / "result.json" + if args.resume and result_path.is_file(): + loads.append(json.loads(result_path.read_text())) + continue + run_dir.mkdir(parents=True, exist_ok=True) + command = builder.build_frontier_command( + python_bin="/usr/bin/python3", + trace_file=traces[rate], + metrics_root=run_dir / "metrics", + run_id=f"qwen30_prefill_{config.name}_r{rate:g}", + knobs=config_knobs, + ) + write_json(run_dir / "command.json", command) + environment = os.environ.copy() + pythonpath = [str(args.python_deps), str(args.frontier_source)] + if environment.get("PYTHONPATH"): + pythonpath.append(environment["PYTHONPATH"]) + environment.update( + { + "PYTHONPATH": ":".join(pythonpath), + "CUDA_VISIBLE_DEVICES": "", + "NVIDIA_VISIBLE_DEVICES": "void", + "WANDB_DISABLED": "true", + "VIDUR_DISABLE_WANDB": "1", + "FRONTIER_LOG_LEVEL": "WARNING", + "PYTHONDONTWRITEBYTECODE": "1", + } + ) + started = time.time() + with (run_dir / "stdout.log").open("w") as stdout, ( + run_dir / "stderr.log" + ).open("w") as stderr: + completed = subprocess.run( + command, + cwd=args.frontier_source, + env=environment, + stdout=stdout, + stderr=stderr, + timeout=args.timeout_seconds, + check=False, + ) + if completed.returncode != 0: + raise RuntimeError( + f"Frontier failed for {config.name} rate={rate}: {completed.returncode}" + ) + system_path, request_path = find_metrics(run_dir) + result = { + "status": "completed", + "config": asdict(config) | {"name": config.name}, + "offered_request_rate": rate, + "offered_request_rate_per_gpu": rate / config.tp, + "request_count": args.requests, + "elapsed_seconds": time.time() - started, + "trace_sha256": sha256(traces[rate]), + "request_metrics_sha256": sha256(request_path), + "score": score(system_path, request_path, args.requests), + } + write_json(result_path, result) + loads.append(result) + print( + json.dumps( + { + "config": config.name, + "rate": rate, + "pass_rate": result["score"]["pass_rate"], + "feasible": result["score"]["feasible"], + }, + sort_keys=True, + ), + flush=True, + ) + config_results.append({"config": asdict(config) | {"name": config.name}, "loads": loads}) + + capacities = [] + for item in config_results: + feasible = [ + load["offered_request_rate"] + for load in item["loads"] + if load["score"]["feasible"] + ] + capacity = max(feasible) if feasible else None + capacities.append( + { + "config": item["config"], + "maximum_tested_feasible_request_rate": capacity, + "maximum_tested_feasible_request_rate_per_gpu": ( + capacity / item["config"]["tp"] if capacity is not None else None + ), + "lower_censored": capacity is None, + "upper_censored": capacity == max(rates) if capacity is not None else False, + } + ) + capacities.sort( + key=lambda row: ( + -(row["maximum_tested_feasible_request_rate_per_gpu"] or -1), + row["config"]["name"], + ) + ) + full = selected == list(GRID) and rates == RATES and args.requests == 64 + manifest = { + "schema": "frontier-qwen30-prefill-surface-v1", + "status": "frozen_before_real" if full else "partial_not_decision_bearing", + "contract": { + "rates": rates, + "requests_per_anchor": args.requests, + "input_tokens": 2048, + "output_tokens": 1, + "ttft_slo_ms": TTFT_SLO_MS, + "target_pass_rate": TARGET_PASS_RATE, + "prefix_caching": False, + "arrival": "open_loop_uniform", + }, + "frontier": { + "source": str(args.frontier_source), + "git_head": frontier_head, + "git_status_short": subprocess.run( + ["git", "-C", str(args.frontier_source), "status", "--short"], + check=True, + text=True, + stdout=subprocess.PIPE, + ).stdout, + }, + "profiles": { + "root": str(args.profile_root), + "coverage": coverage, + "sha256": {name: sha256(path) for name, path in paths.items()}, + }, + "config_results": config_results, + "capacity": capacities, + } + write_json(args.output_root / "frontier_surface_frozen.json", manifest) + print(args.output_root / "frontier_surface_frozen.json") + + +if __name__ == "__main__": + main() diff --git a/runs/frontier-phase-factorial-v0/run_qwen30_prefill_real_config.sh b/runs/frontier-phase-factorial-v0/run_qwen30_prefill_real_config.sh new file mode 100644 index 0000000..a406d51 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/run_qwen30_prefill_real_config.sh @@ -0,0 +1,135 @@ +#!/usr/bin/env bash + +set -euo pipefail + +OUTPUT_ROOT="${OUTPUT_ROOT:?OUTPUT_ROOT is required}" +TP="${TP:?TP is required}" +MNS="${MNS:?MNS is required}" +RATES="${RATES:-4 8 16 32 64}" +SERVER_PORT="${SERVER_PORT:?SERVER_PORT is required}" +VENV_ROOT="${VENV_ROOT:-/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1}" +MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}" +SERVED_MODEL="qwen3-30b-prefill-only" +SERVER_PID="" + +mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/provenance" +exec > >(tee -a "${OUTPUT_ROOT}/logs/controller.log") 2>&1 + +cleanup() { + if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then + kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true + for _ in $(seq 1 30); do + kill -0 "${SERVER_PID}" 2>/dev/null || break + sleep 1 + done + kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true + fi + SERVER_PID="" +} +trap cleanup EXIT INT TERM + +IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES:-}" +if [[ "${#GPU_IDS[@]}" -ne "${TP}" ]]; then + echo "ERROR: expected ${TP} allocated GPUs, got ${CUDA_VISIBLE_DEVICES:-unset}" >&2 + exit 1 +fi +read -r -a RATE_ARRAY <<< "${RATES}" + +echo "QWEN30_PREFILL_REAL_LAUNCH_ECHO host=$(hostname) gpus=${CUDA_VISIBLE_DEVICES} model=${MODEL_ROOT} runtime=vLLM-0.20.0+cu129 dtype=BF16 config=TP${TP}_MNS${MNS}_MBT8192 rates=${RATES// /,} rounds=2 requests=64 shape=ISL2048_OSL1 arrivals=uniform prefix=off cuda_graph=runtime_default isolation=fresh_server_per_anchor output=${OUTPUT_ROOT}" +date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ" +sha256sum run_qwen30_prefill_real_config.sh qwen30_prefill_client.py \ + > "${OUTPUT_ROOT}/provenance/source.sha256" +"${VENV_ROOT}/bin/python" - "${TP}" "${MNS}" "${RATES}" \ + > "${OUTPUT_ROOT}/provenance/contract.json" <<'PY' +import importlib.metadata as metadata +import json +import platform +import sys + +tp, mns, rates = sys.argv[1:] +print(json.dumps({ + "python": platform.python_version(), + "torch": metadata.version("torch"), + "transformers": metadata.version("transformers"), + "vllm": metadata.version("vllm"), + "config": {"tp": int(tp), "mns": int(mns), "mbt": 8192}, + "rates": [float(value) for value in rates.split()], + "rounds": 2, + "requests_per_anchor": 64, + "anchor_isolation": "fresh_server_per_rate_per_round", + "target_rate_warmup_requests": "min(32, max(4, ceil(rate * 2)))", + "input_tokens": 2048, + "output_tokens": 1, + "ttft_slo_ms": 1256.0, + "target_pass_rate": 0.95, +}, indent=2, sort_keys=True)) +PY +nvidia-smi --query-gpu=index,name,uuid,driver_version --format=csv,noheader \ + > "${OUTPUT_ROOT}/provenance/gpus.csv" +sha256sum "${MODEL_ROOT}/config.json" > "${OUTPUT_ROOT}/provenance/model.sha256" + +export TOKENIZERS_PARALLELISM=false +export VLLM_USE_V1=1 +export TORCH_CUDA_ARCH_LIST=9.0 +export HF_HUB_OFFLINE=1 +export TRANSFORMERS_OFFLINE=1 + +for ROUND in 1 2; do + ROUND_ROOT="${OUTPUT_ROOT}/round${ROUND}" + mkdir -p "${ROUND_ROOT}/logs" "${ROUND_ROOT}/results" + ORDERED_RATES=("${RATE_ARRAY[@]}") + if [[ "${ROUND}" -eq 2 ]]; then + ORDERED_RATES=() + for ((index=${#RATE_ARRAY[@]}-1; index>=0; index--)); do + ORDERED_RATES+=("${RATE_ARRAY[index]}") + done + fi + for RATE in "${ORDERED_RATES[@]}"; do + KEY="$(printf 'r%.2f' "${RATE}" | tr '.' 'p')" + SERVER_LOG="${ROUND_ROOT}/logs/server_${KEY}.log" + setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \ + --host 127.0.0.1 --port "${SERVER_PORT}" --served-model-name "${SERVED_MODEL}" \ + --tensor-parallel-size "${TP}" --gpu-memory-utilization 0.92 \ + --max-model-len 40960 --max-num-batched-tokens 8192 --max-num-seqs "${MNS}" \ + --no-enable-prefix-caching --enable-chunked-prefill --no-enable-log-requests \ + > "${SERVER_LOG}" 2>&1 & + SERVER_PID=$! + READY=0 + for _ in $(seq 1 120); do + if curl -fsS --max-time 2 "http://127.0.0.1:${SERVER_PORT}/v1/models" \ + > "${ROUND_ROOT}/results/models_${KEY}.json" 2>/dev/null; then + READY=1 + break + fi + if ! kill -0 "${SERVER_PID}" 2>/dev/null; then + tail -200 "${SERVER_LOG}" + exit 1 + fi + sleep 3 + done + if [[ "${READY}" -ne 1 ]]; then + tail -200 "${SERVER_LOG}" + exit 1 + fi + + WARMUP_REQUESTS="$("${VENV_ROOT}/bin/python" - "${RATE}" <<'PY' +import math +import sys +print(min(32, max(4, math.ceil(float(sys.argv[1]) * 2.0)))) +PY +)" + "${VENV_ROOT}/bin/python" qwen30_prefill_client.py --port "${SERVER_PORT}" \ + --served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" --rate "${RATE}" \ + --requests "${WARMUP_REQUESTS}" \ + --output "${ROUND_ROOT}/results/warmup_${KEY}.json" + "${VENV_ROOT}/bin/python" qwen30_prefill_client.py --port "${SERVER_PORT}" \ + --served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" --rate "${RATE}" \ + --requests 64 --output "${ROUND_ROOT}/results/${KEY}.json" + cleanup + done +done + +find "${OUTPUT_ROOT}" -type f ! -path '*/provenance/artifacts.sha256' -print0 \ + | sort -z | xargs -0 sha256sum > "${OUTPUT_ROOT}/provenance/artifacts.sha256" +date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ" +echo "QWEN30_PREFILL_REAL_CONFIG_COMPLETE" diff --git a/runs/frontier-phase-factorial-v0/simulator-tp1/frontier_surface_frozen.json b/runs/frontier-phase-factorial-v0/simulator-tp1/frontier_surface_frozen.json new file mode 100644 index 0000000..0dafcdd --- /dev/null +++ b/runs/frontier-phase-factorial-v0/simulator-tp1/frontier_surface_frozen.json @@ -0,0 +1,647 @@ +{ + "capacity": [ + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 8.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + }, + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 8.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + }, + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 8.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + }, + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 8.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + } + ], + "config_results": [ + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "loads": [ + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "elapsed_seconds": 20.451046466827393, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.0197002243768605, + "ttft_max_ms": 171.58529929215405, + "ttft_p50_ms": 171.58529929215405, + "ttft_p95_ms": 171.58529929215405 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "elapsed_seconds": 22.0544171333313, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 7.742175153637853, + "ttft_max_ms": 570.3133207139617, + "ttft_p50_ms": 391.4107605377559, + "ttft_p95_ms": 516.4107605377559 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "elapsed_seconds": 18.859471321105957, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e", + "score": { + "feasible": false, + "pass_rate": 0.3125, + "passed": 20, + "throughput_requests_per_second": 8.916224898348922, + "ttft_max_ms": 3318.8404181099845, + "ttft_p50_ms": 1814.2305264490028, + "ttft_p95_ms": 3193.8404181099845 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "elapsed_seconds": 16.640390872955322, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc", + "score": { + "feasible": false, + "pass_rate": 0.203125, + "passed": 13, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 5147.666503402199, + "ttft_p50_ms": 2739.3537469047046, + "ttft_p95_ms": 4962.713638565687 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 8, + "name": "tp1_mns8", + "tp": 1 + }, + "elapsed_seconds": 18.142091274261475, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 64.0, + "request_count": 64, + "request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03", + "score": { + "feasible": false, + "pass_rate": 0.140625, + "passed": 9, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 6100.791503402199, + "ttft_p50_ms": 3208.1037469047046, + "ttft_p95_ms": 5853.338638565687 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "loads": [ + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "elapsed_seconds": 19.96323037147522, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.0197002243768605, + "ttft_max_ms": 171.58529929215405, + "ttft_p50_ms": 171.58529929215405, + "ttft_p95_ms": 171.58529929215405 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "elapsed_seconds": 18.554351568222046, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 7.742175153637853, + "ttft_max_ms": 570.3133207139617, + "ttft_p50_ms": 391.4107605377559, + "ttft_p95_ms": 516.4107605377559 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "elapsed_seconds": 25.70284128189087, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e", + "score": { + "feasible": false, + "pass_rate": 0.3125, + "passed": 20, + "throughput_requests_per_second": 8.916224898348922, + "ttft_max_ms": 3318.8404181099845, + "ttft_p50_ms": 1814.2305264490028, + "ttft_p95_ms": 3193.8404181099845 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "elapsed_seconds": 20.056548833847046, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc", + "score": { + "feasible": false, + "pass_rate": 0.203125, + "passed": 13, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 5147.666503402199, + "ttft_p50_ms": 2739.3537469047046, + "ttft_p95_ms": 4962.713638565687 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 16, + "name": "tp1_mns16", + "tp": 1 + }, + "elapsed_seconds": 11.15293574333191, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 64.0, + "request_count": 64, + "request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03", + "score": { + "feasible": false, + "pass_rate": 0.140625, + "passed": 9, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 6100.791503402199, + "ttft_p50_ms": 3208.1037469047046, + "ttft_p95_ms": 5853.338638565687 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "loads": [ + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "elapsed_seconds": 10.411258935928345, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.0197002243768605, + "ttft_max_ms": 171.58529929215405, + "ttft_p50_ms": 171.58529929215405, + "ttft_p95_ms": 171.58529929215405 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "elapsed_seconds": 10.355234384536743, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 7.742175153637853, + "ttft_max_ms": 570.3133207139617, + "ttft_p50_ms": 391.4107605377559, + "ttft_p95_ms": 516.4107605377559 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "elapsed_seconds": 10.098335266113281, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e", + "score": { + "feasible": false, + "pass_rate": 0.3125, + "passed": 20, + "throughput_requests_per_second": 8.916224898348922, + "ttft_max_ms": 3318.8404181099845, + "ttft_p50_ms": 1814.2305264490028, + "ttft_p95_ms": 3193.8404181099845 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "elapsed_seconds": 9.8981294631958, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc", + "score": { + "feasible": false, + "pass_rate": 0.203125, + "passed": 13, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 5147.666503402199, + "ttft_p50_ms": 2739.3537469047046, + "ttft_p95_ms": 4962.713638565687 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 32, + "name": "tp1_mns32", + "tp": 1 + }, + "elapsed_seconds": 9.754717826843262, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 64.0, + "request_count": 64, + "request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03", + "score": { + "feasible": false, + "pass_rate": 0.140625, + "passed": 9, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 6100.791503402199, + "ttft_p50_ms": 3208.1037469047046, + "ttft_p95_ms": 5853.338638565687 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "loads": [ + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "elapsed_seconds": 9.84917163848877, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "f0cfb072afcc2fc82a92c59ddb4e8cfc1e3201e15848f37785eeb9fdd975e779", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.0197002243768605, + "ttft_max_ms": 171.58529929215405, + "ttft_p50_ms": 171.58529929215405, + "ttft_p95_ms": 171.58529929215405 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "elapsed_seconds": 10.197595357894897, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "cfebfbd8aa5d9603158d8ff4e786119433b4c0d705f733ba242da46a6a6fdfb0", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 7.742175153637853, + "ttft_max_ms": 570.3133207139617, + "ttft_p50_ms": 391.4107605377559, + "ttft_p95_ms": 516.4107605377559 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "elapsed_seconds": 10.00463604927063, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "03578ba406b3299e70ba7975e6b9872d381a5f4bd4fbc3c8f6a8ba49e9b53f2e", + "score": { + "feasible": false, + "pass_rate": 0.3125, + "passed": 20, + "throughput_requests_per_second": 8.916224898348922, + "ttft_max_ms": 3318.8404181099845, + "ttft_p50_ms": 1814.2305264490028, + "ttft_p95_ms": 3193.8404181099845 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "elapsed_seconds": 10.153925895690918, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "0b761b5f42e65b694eb34735433d43b473ed67415fb4e17b89894fc840b645fc", + "score": { + "feasible": false, + "pass_rate": 0.203125, + "passed": 13, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 5147.666503402199, + "ttft_p50_ms": 2739.3537469047046, + "ttft_p95_ms": 4962.713638565687 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 64, + "name": "tp1_mns64", + "tp": 1 + }, + "elapsed_seconds": 10.055675268173218, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 64.0, + "request_count": 64, + "request_metrics_sha256": "078209f2f56f21243384bdc70add105bf4942ee44ec8cc9573cad399a4a10e03", + "score": { + "feasible": false, + "pass_rate": 0.140625, + "passed": 9, + "throughput_requests_per_second": 9.07297385348011, + "ttft_max_ms": 6100.791503402199, + "ttft_p50_ms": 3208.1037469047046, + "ttft_p95_ms": 5853.338638565687 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + } + ], + "contract": { + "arrival": "open_loop_uniform", + "input_tokens": 2048, + "output_tokens": 1, + "prefix_caching": false, + "rates": [ + 4.0, + 8.0, + 16.0, + 32.0, + 64.0 + ], + "requests_per_anchor": 64, + "target_pass_rate": 0.95, + "ttft_slo_ms": 1256.0 + }, + "frontier": { + "git_head": "d9cfeb6d8791fbf2f295dd9744c56a666171776e", + "git_status_short": " M frontier/config/config.py\n M frontier/entities/request.py\n M frontier/events/cluster_schedule_event.py\n M frontier/execution_time_predictor/sklearn_execution_time_predictor.py\n M frontier/metrics/constants.py\n M frontier/metrics/metrics_store.py\n M frontier/profiling/common/layers/rotary_embedding.py\n M frontier/profiling/moe/moe_impl.py\n M frontier/profiling/moe/moe_vllm_kernel.py\n M frontier/scheduler/cluster_scheduler/__init__.py\n M frontier/scheduler/cluster_scheduler/base_cluster_scheduler.py\n M frontier/scheduler/cluster_scheduler/cluster_scheduler_registry.py\n M frontier/scheduler/cluster_scheduler/sticky_lor_cluster_scheduler.py\n M frontier/scheduler/replica_scheduler/base_replica_scheduler.py\n M frontier/scheduler/replica_scheduler/vllm_v1_engine_replica_scheduler.py\n M frontier/scheduler/replica_stage_scheduler/replica_stage_schduler.py\n M frontier/simulator.py\n M frontier/types/cluster_scheduler_type.py\n?? data/profiling/compute/h20/\n?? frontier/scheduler/cluster_scheduler/prefix_lor_cluster_scheduler.py\n?? runs/\n?? tests/unit/test_attn_prefill_prediction_fallback.py\n", + "source": "/tmp/replayserve-frontier-rs1b" + }, + "profiles": { + "coverage": { + "attention": { + "1": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + }, + "2": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + }, + "4": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + } + }, + "manifest": { + "attention_tp_coverage": [ + 1, + 2, + 4 + ], + "environment_contract": { + "dtype": "bfloat16", + "frontier_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e", + "hardware": "NVIDIA H20", + "model": "Qwen3-30B-A3B", + "tensor_parallel_sizes": [ + 1, + 2, + 4 + ], + "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1", + "vllm_version": "0.20.0" + }, + "inputs": { + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp2-20260716-v1-dispatch-aware-20260716T140743025781Z/artifacts/artifacts/allreduce-full-tp2-v1/raw/allreduce-tp2.json": "97c3c76b5a04e95bd9192423c2b891667c668f39cc0dfecbd097d749939f2d0a", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp4-20260716-v1-dispatch-aware-20260716T141106009788Z/artifacts/artifacts/allreduce-full-tp4-v1/raw/allreduce-tp4.json": "809df9baa6f468cf12bf0c99827475acc67894dd9f3f948976590b665fac0e76", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp1-20260716-v2-20260716T135132587012Z/artifacts/artifacts/flashattn-kv-full-v2-tp1/raw/flashattn-tp1.json": "dcb4c1bf7e76b9c765f78ddd2b8a734f2d7ba2adac13ce017689a8a77fe69a27", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp2-20260716-v2-20260716T135134194295Z/artifacts/artifacts/flashattn-kv-full-v2-tp2/raw/flashattn-tp2.json": "43ce042556ba887c8860614b43ccf0f564e5cebc1a0cffbce299d0acb9fa8d07", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp4-20260716-v2-20260716T135135197200Z/artifacts/artifacts/flashattn-kv-full-v2-tp4/raw/flashattn-tp4.json": "84eef31bcad0f556907a093318a420959d14fdc94474823d11f659704bdfec73", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-frontier-linear-full-20260716-v2-max-tokens-20260716T144444676943Z/artifacts/artifacts/frontier-linear-full-v2/profiles/compute/h20/qwen3-a3b-30b-moe/linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-moe-full-20260716-v1-local-shard-20260716T141334565164Z/artifacts/artifacts/moe-full-v1/raw/moe-full.json": "588f6ad0d69c9636d1b852e3df0a12d13cfe731f050ea7ec7aea457cceefbde8", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-router-full-20260716-v3-tp-context-20260716T145446098505Z/artifacts/artifacts/router-full-v3/raw/router.json": "1962972e983bff3e06a721ef4ae4ec65728ff669681497a4a7e7f769b88b4931" + }, + "outputs": { + "allreduce.json": "b38d14f990578d668523d25b107aceed433da5020d8ada3b6e44d3562261a3b3", + "attention.csv": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d", + "attention_true_mixed_fused.csv": "43ef4be90bddc9aeac6dbbe339feec24162cd1f2129a08fbd959e6ee4eaf5f60", + "linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "moe.csv": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0" + }, + "profile_id": "qwen3-30b-a3b-bf16-vllm020-h20-tp1-2-4-fused-mixed-total-conserving", + "projection_contract": { + "allreduce": "Frozen exact runtime measurements; base profile-only comparison keeps the historical Frontier CC backend fixed to isolate compute profile fidelity", + "attention": "Pure prefill/extend/decode FA3 core plus separately measured KV update; input/output reshape assumed zero; exported mean is used as median target; true mixed rows use a total-conserving compatibility projection", + "attention_true_mixed": "The directly measured fused total is preserved in diagnostics. Frontier's two targets are projected by the same-TP pure prefill/decode reference ratio, with projected prefill + decode exactly equal to the fused total; the split is a schema compatibility attribution, not an observation", + "linear": "Frontier profiler using vLLM 0.20 CUDA operators", + "moe": "Replicated gate and fused top-k plus TP-local modular expert kernel; expert measurement already includes prepare/finalize so shuffling is zero" + }, + "row_counts": { + "allreduce": 24, + "attention_frontier_compatible": 132, + "attention_true_mixed_fused_diagnostic": 30, + "linear": 36, + "moe": 72 + }, + "schema_version": "frontier_qwen30_vllm020_frozen_profile.v2" + } + }, + "root": "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/frozen/profile-v2", + "sha256": { + "attention": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d", + "linear": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "manifest": "af40545e75aff55c6333cd2d5379ccf042a5a0b7d7fc7df4f745ce256cb290eb", + "moe": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0" + } + }, + "schema": "frontier-qwen30-prefill-surface-v1", + "status": "partial_not_decision_bearing" +} diff --git a/runs/frontier-phase-factorial-v0/simulator-tp2/frontier_surface_frozen.json b/runs/frontier-phase-factorial-v0/simulator-tp2/frontier_surface_frozen.json new file mode 100644 index 0000000..34e2063 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/simulator-tp2/frontier_surface_frozen.json @@ -0,0 +1,647 @@ +{ + "capacity": [ + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + }, + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + }, + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + }, + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 8.0, + "upper_censored": false + } + ], + "config_results": [ + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "loads": [ + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "elapsed_seconds": 264.1678559780121, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.032196934357252, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "elapsed_seconds": 10.15904188156128, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.00276032449276, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "elapsed_seconds": 10.447325944900513, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 13.174042777348715, + "ttft_max_ms": 1048.2973400621365, + "ttft_p50_ms": 643.9195307403329, + "ttft_p95_ms": 962.8278877316862 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "elapsed_seconds": 10.053350687026978, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 2851.9297930156335, + "ttft_p50_ms": 1510.0519836938302, + "ttft_p95_ms": 2758.1797930156335 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 8, + "name": "tp2_mns8", + "tp": 2 + }, + "elapsed_seconds": 9.744416952133179, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a", + "score": { + "feasible": false, + "pass_rate": 0.265625, + "passed": 17, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 3773.8047930156335, + "ttft_p50_ms": 1995.3684369244447, + "ttft_p95_ms": 3622.636520081023 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "loads": [ + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "elapsed_seconds": 9.402071952819824, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.032196934357252, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "elapsed_seconds": 9.398229360580444, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.00276032449276, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "elapsed_seconds": 9.70562195777893, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 13.174042777348715, + "ttft_max_ms": 1048.2973400621365, + "ttft_p50_ms": 643.9195307403329, + "ttft_p95_ms": 962.8278877316862 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "elapsed_seconds": 9.598989009857178, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 2851.9297930156335, + "ttft_p50_ms": 1510.0519836938302, + "ttft_p95_ms": 2758.1797930156335 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 16, + "name": "tp2_mns16", + "tp": 2 + }, + "elapsed_seconds": 9.697861671447754, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a", + "score": { + "feasible": false, + "pass_rate": 0.265625, + "passed": 17, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 3773.8047930156335, + "ttft_p50_ms": 1995.3684369244447, + "ttft_p95_ms": 3622.636520081023 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "loads": [ + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "elapsed_seconds": 9.397430419921875, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.032196934357252, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "elapsed_seconds": 9.3956880569458, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.00276032449276, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "elapsed_seconds": 9.647802591323853, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 13.174042777348715, + "ttft_max_ms": 1048.2973400621365, + "ttft_p50_ms": 643.9195307403329, + "ttft_p95_ms": 962.8278877316862 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "elapsed_seconds": 9.694743156433105, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 2851.9297930156335, + "ttft_p50_ms": 1510.0519836938302, + "ttft_p95_ms": 2758.1797930156335 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 32, + "name": "tp2_mns32", + "tp": 2 + }, + "elapsed_seconds": 9.594672203063965, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a", + "score": { + "feasible": false, + "pass_rate": 0.265625, + "passed": 17, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 3773.8047930156335, + "ttft_p50_ms": 1995.3684369244447, + "ttft_p95_ms": 3622.636520081023 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "loads": [ + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "elapsed_seconds": 9.398636817932129, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "e5f6527532682ee5aca3f423367d14eba063f30e63f46d2ba0ed602db4f6eb17", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.032196934357252, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "elapsed_seconds": 9.495105981826782, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "8dd3bedcbf76d30bf4285e10b886f95bef9ae8e5d41f4bb50a7650b322b8eb83", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.00276032449276, + "ttft_max_ms": 122.24062760264154, + "ttft_p50_ms": 122.24062760264154, + "ttft_p95_ms": 122.24062760264154 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "elapsed_seconds": 9.54387378692627, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "38c1b13c1f5fbdabd7f79b82ccb1fb37fce6840bcf03614c988d7e4882814506", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 13.174042777348715, + "ttft_max_ms": 1048.2973400621365, + "ttft_p50_ms": 643.9195307403329, + "ttft_p95_ms": 962.8278877316862 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "elapsed_seconds": 9.192227840423584, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "cb3b7b5a2fcf63cd01304b3b530bb21c0a9bb1c8f5e06d489e3863afd0b4d2f3", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 2851.9297930156335, + "ttft_p50_ms": 1510.0519836938302, + "ttft_p95_ms": 2758.1797930156335 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 64, + "name": "tp2_mns64", + "tp": 2 + }, + "elapsed_seconds": 9.294084072113037, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 32.0, + "request_count": 64, + "request_metrics_sha256": "a4474c98a68171ada0fb007c36fcf69e9e2dc13f06c74889591e23f93418bd5a", + "score": { + "feasible": false, + "pass_rate": 0.265625, + "passed": 17, + "throughput_requests_per_second": 13.539443740959392, + "ttft_max_ms": 3773.8047930156335, + "ttft_p50_ms": 1995.3684369244447, + "ttft_p95_ms": 3622.636520081023 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + } + ], + "contract": { + "arrival": "open_loop_uniform", + "input_tokens": 2048, + "output_tokens": 1, + "prefix_caching": false, + "rates": [ + 4.0, + 8.0, + 16.0, + 32.0, + 64.0 + ], + "requests_per_anchor": 64, + "target_pass_rate": 0.95, + "ttft_slo_ms": 1256.0 + }, + "frontier": { + "git_head": "d9cfeb6d8791fbf2f295dd9744c56a666171776e", + "git_status_short": " M frontier/config/config.py\n M frontier/entities/request.py\n M frontier/events/cluster_schedule_event.py\n M frontier/execution_time_predictor/sklearn_execution_time_predictor.py\n M frontier/metrics/constants.py\n M frontier/metrics/metrics_store.py\n M frontier/profiling/common/layers/rotary_embedding.py\n M frontier/profiling/moe/moe_impl.py\n M frontier/profiling/moe/moe_vllm_kernel.py\n M frontier/scheduler/cluster_scheduler/__init__.py\n M frontier/scheduler/cluster_scheduler/base_cluster_scheduler.py\n M frontier/scheduler/cluster_scheduler/cluster_scheduler_registry.py\n M frontier/scheduler/cluster_scheduler/sticky_lor_cluster_scheduler.py\n M frontier/scheduler/replica_scheduler/base_replica_scheduler.py\n M frontier/scheduler/replica_scheduler/vllm_v1_engine_replica_scheduler.py\n M frontier/scheduler/replica_stage_scheduler/replica_stage_schduler.py\n M frontier/simulator.py\n M frontier/types/cluster_scheduler_type.py\n?? data/profiling/compute/h20/\n?? frontier/scheduler/cluster_scheduler/prefix_lor_cluster_scheduler.py\n?? runs/\n?? tests/unit/test_attn_prefill_prediction_fallback.py\n", + "source": "/tmp/replayserve-frontier-rs1b" + }, + "profiles": { + "coverage": { + "attention": { + "1": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + }, + "2": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + }, + "4": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + } + }, + "manifest": { + "attention_tp_coverage": [ + 1, + 2, + 4 + ], + "environment_contract": { + "dtype": "bfloat16", + "frontier_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e", + "hardware": "NVIDIA H20", + "model": "Qwen3-30B-A3B", + "tensor_parallel_sizes": [ + 1, + 2, + 4 + ], + "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1", + "vllm_version": "0.20.0" + }, + "inputs": { + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp2-20260716-v1-dispatch-aware-20260716T140743025781Z/artifacts/artifacts/allreduce-full-tp2-v1/raw/allreduce-tp2.json": "97c3c76b5a04e95bd9192423c2b891667c668f39cc0dfecbd097d749939f2d0a", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp4-20260716-v1-dispatch-aware-20260716T141106009788Z/artifacts/artifacts/allreduce-full-tp4-v1/raw/allreduce-tp4.json": "809df9baa6f468cf12bf0c99827475acc67894dd9f3f948976590b665fac0e76", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp1-20260716-v2-20260716T135132587012Z/artifacts/artifacts/flashattn-kv-full-v2-tp1/raw/flashattn-tp1.json": "dcb4c1bf7e76b9c765f78ddd2b8a734f2d7ba2adac13ce017689a8a77fe69a27", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp2-20260716-v2-20260716T135134194295Z/artifacts/artifacts/flashattn-kv-full-v2-tp2/raw/flashattn-tp2.json": "43ce042556ba887c8860614b43ccf0f564e5cebc1a0cffbce299d0acb9fa8d07", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp4-20260716-v2-20260716T135135197200Z/artifacts/artifacts/flashattn-kv-full-v2-tp4/raw/flashattn-tp4.json": "84eef31bcad0f556907a093318a420959d14fdc94474823d11f659704bdfec73", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-frontier-linear-full-20260716-v2-max-tokens-20260716T144444676943Z/artifacts/artifacts/frontier-linear-full-v2/profiles/compute/h20/qwen3-a3b-30b-moe/linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-moe-full-20260716-v1-local-shard-20260716T141334565164Z/artifacts/artifacts/moe-full-v1/raw/moe-full.json": "588f6ad0d69c9636d1b852e3df0a12d13cfe731f050ea7ec7aea457cceefbde8", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-router-full-20260716-v3-tp-context-20260716T145446098505Z/artifacts/artifacts/router-full-v3/raw/router.json": "1962972e983bff3e06a721ef4ae4ec65728ff669681497a4a7e7f769b88b4931" + }, + "outputs": { + "allreduce.json": "b38d14f990578d668523d25b107aceed433da5020d8ada3b6e44d3562261a3b3", + "attention.csv": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d", + "attention_true_mixed_fused.csv": "43ef4be90bddc9aeac6dbbe339feec24162cd1f2129a08fbd959e6ee4eaf5f60", + "linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "moe.csv": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0" + }, + "profile_id": "qwen3-30b-a3b-bf16-vllm020-h20-tp1-2-4-fused-mixed-total-conserving", + "projection_contract": { + "allreduce": "Frozen exact runtime measurements; base profile-only comparison keeps the historical Frontier CC backend fixed to isolate compute profile fidelity", + "attention": "Pure prefill/extend/decode FA3 core plus separately measured KV update; input/output reshape assumed zero; exported mean is used as median target; true mixed rows use a total-conserving compatibility projection", + "attention_true_mixed": "The directly measured fused total is preserved in diagnostics. Frontier's two targets are projected by the same-TP pure prefill/decode reference ratio, with projected prefill + decode exactly equal to the fused total; the split is a schema compatibility attribution, not an observation", + "linear": "Frontier profiler using vLLM 0.20 CUDA operators", + "moe": "Replicated gate and fused top-k plus TP-local modular expert kernel; expert measurement already includes prepare/finalize so shuffling is zero" + }, + "row_counts": { + "allreduce": 24, + "attention_frontier_compatible": 132, + "attention_true_mixed_fused_diagnostic": 30, + "linear": 36, + "moe": 72 + }, + "schema_version": "frontier_qwen30_vllm020_frozen_profile.v2" + } + }, + "root": "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/frozen/profile-v2", + "sha256": { + "attention": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d", + "linear": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "manifest": "af40545e75aff55c6333cd2d5379ccf042a5a0b7d7fc7df4f745ce256cb290eb", + "moe": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0" + } + }, + "schema": "frontier-qwen30-prefill-surface-v1", + "status": "partial_not_decision_bearing" +} diff --git a/runs/frontier-phase-factorial-v0/simulator-tp4/frontier_surface_frozen.json b/runs/frontier-phase-factorial-v0/simulator-tp4/frontier_surface_frozen.json new file mode 100644 index 0000000..82a1227 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/simulator-tp4/frontier_surface_frozen.json @@ -0,0 +1,647 @@ +{ + "capacity": [ + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 4.0, + "upper_censored": false + }, + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 4.0, + "upper_censored": false + }, + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 4.0, + "upper_censored": false + }, + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "lower_censored": false, + "maximum_tested_feasible_request_rate": 16.0, + "maximum_tested_feasible_request_rate_per_gpu": 4.0, + "upper_censored": false + } + ], + "config_results": [ + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "loads": [ + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "elapsed_seconds": 244.95408463478088, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 1.0, + "request_count": 64, + "request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.039607508408214, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685405, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "elapsed_seconds": 9.85124945640564, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.032004183533672, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685495, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "elapsed_seconds": 9.70260500907898, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 15.412041282697286, + "ttft_max_ms": 314.4210260695206, + "ttft_p50_ms": 239.37774488651664, + "ttft_p95_ms": 309.403713596319 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "elapsed_seconds": 9.855336904525757, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f", + "score": { + "feasible": false, + "pass_rate": 0.625, + "passed": 40, + "throughput_requests_per_second": 16.898601280793688, + "ttft_max_ms": 1850.42236218991, + "ttft_p50_ms": 1017.6783445024347, + "ttft_p95_ms": 1787.92236218991 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 8, + "name": "tp4_mns8", + "tp": 4 + }, + "elapsed_seconds": 9.947782754898071, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 17.184897883039092, + "ttft_max_ms": 2771.0751922610093, + "ttft_p50_ms": 1468.937931419684, + "ttft_p95_ms": 2661.056949107159 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "loads": [ + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "elapsed_seconds": 9.648890733718872, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 1.0, + "request_count": 64, + "request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.039607508408214, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685405, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "elapsed_seconds": 9.353798151016235, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.032004183533672, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685495, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "elapsed_seconds": 9.39468502998352, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 15.412041282697286, + "ttft_max_ms": 314.4210260695206, + "ttft_p50_ms": 239.37774488651664, + "ttft_p95_ms": 309.403713596319 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "elapsed_seconds": 9.29756498336792, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f", + "score": { + "feasible": false, + "pass_rate": 0.625, + "passed": 40, + "throughput_requests_per_second": 16.898601280793688, + "ttft_max_ms": 1850.42236218991, + "ttft_p50_ms": 1017.6783445024347, + "ttft_p95_ms": 1787.92236218991 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 16, + "name": "tp4_mns16", + "tp": 4 + }, + "elapsed_seconds": 9.34343934059143, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 17.184897883039092, + "ttft_max_ms": 2771.0751922610093, + "ttft_p50_ms": 1468.937931419684, + "ttft_p95_ms": 2661.056949107159 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "loads": [ + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "elapsed_seconds": 9.296925783157349, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 1.0, + "request_count": 64, + "request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.039607508408214, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685405, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "elapsed_seconds": 9.345365285873413, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.032004183533672, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685495, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "elapsed_seconds": 9.350512504577637, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 15.412041282697286, + "ttft_max_ms": 314.4210260695206, + "ttft_p50_ms": 239.37774488651664, + "ttft_p95_ms": 309.403713596319 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "elapsed_seconds": 9.300485372543335, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f", + "score": { + "feasible": false, + "pass_rate": 0.625, + "passed": 40, + "throughput_requests_per_second": 16.898601280793688, + "ttft_max_ms": 1850.42236218991, + "ttft_p50_ms": 1017.6783445024347, + "ttft_p95_ms": 1787.92236218991 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 32, + "name": "tp4_mns32", + "tp": 4 + }, + "elapsed_seconds": 9.292456150054932, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 17.184897883039092, + "ttft_max_ms": 2771.0751922610093, + "ttft_p50_ms": 1468.937931419684, + "ttft_p95_ms": 2661.056949107159 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + }, + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "loads": [ + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "elapsed_seconds": 9.294773817062378, + "offered_request_rate": 4.0, + "offered_request_rate_per_gpu": 1.0, + "request_count": 64, + "request_metrics_sha256": "c0c3c34bea43dae2477b10152b07632e5d03968f47cdfd70ad5b676e2502af4c", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 4.039607508408214, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685405, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "0cef631f351f5afd880172d0467d62d9813de5ba36fd3795f91afe3256450c05" + }, + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "elapsed_seconds": 9.34990382194519, + "offered_request_rate": 8.0, + "offered_request_rate_per_gpu": 2.0, + "request_count": 64, + "request_metrics_sha256": "b5c6ba0cd3b2a81a12fb93b2cad6f80341f9352c110385046490f59d7933a5d5", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 8.032004183533672, + "ttft_max_ms": 93.12333977685495, + "ttft_p50_ms": 93.12333977685495, + "ttft_p95_ms": 93.12333977685495 + }, + "status": "completed", + "trace_sha256": "c49314cbe4e4b07a9335b57e861ec5b6541ea4cc061ca9abf923453276c419fd" + }, + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "elapsed_seconds": 9.346213579177856, + "offered_request_rate": 16.0, + "offered_request_rate_per_gpu": 4.0, + "request_count": 64, + "request_metrics_sha256": "c40f10c895f2fc45671f7ff7c6c08f30c20854f0ff4fe3e6719934a7b08b0112", + "score": { + "feasible": true, + "pass_rate": 1.0, + "passed": 64, + "throughput_requests_per_second": 15.412041282697286, + "ttft_max_ms": 314.4210260695206, + "ttft_p50_ms": 239.37774488651664, + "ttft_p95_ms": 309.403713596319 + }, + "status": "completed", + "trace_sha256": "2504b6340f764e726110e48251f8017a2f9ba67b44b8af7d3a838bf674823d33" + }, + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "elapsed_seconds": 9.498731136322021, + "offered_request_rate": 32.0, + "offered_request_rate_per_gpu": 8.0, + "request_count": 64, + "request_metrics_sha256": "9f77ee40d102bdf14d213132ef10b4a7f4175908ffee411ef30ad3a5f1e4b16f", + "score": { + "feasible": false, + "pass_rate": 0.625, + "passed": 40, + "throughput_requests_per_second": 16.898601280793688, + "ttft_max_ms": 1850.42236218991, + "ttft_p50_ms": 1017.6783445024347, + "ttft_p95_ms": 1787.92236218991 + }, + "status": "completed", + "trace_sha256": "5508a3ce590f7d1b0b3c211a2d4f2131886a7ae2d85cafa0c436a8e3029675f1" + }, + { + "config": { + "mns": 64, + "name": "tp4_mns64", + "tp": 4 + }, + "elapsed_seconds": 9.300755977630615, + "offered_request_rate": 64.0, + "offered_request_rate_per_gpu": 16.0, + "request_count": 64, + "request_metrics_sha256": "efb43cc32f2f7129082e47d89239f08b10c6e7e6dd4a4f0b299d2364cb781af0", + "score": { + "feasible": false, + "pass_rate": 0.390625, + "passed": 25, + "throughput_requests_per_second": 17.184897883039092, + "ttft_max_ms": 2771.0751922610093, + "ttft_p50_ms": 1468.937931419684, + "ttft_p95_ms": 2661.056949107159 + }, + "status": "completed", + "trace_sha256": "8987c3ebd4a7db983dc0e99e44710cd44a9d5d21cdb8e6b58878b160918593d9" + } + ] + } + ], + "contract": { + "arrival": "open_loop_uniform", + "input_tokens": 2048, + "output_tokens": 1, + "prefix_caching": false, + "rates": [ + 4.0, + 8.0, + 16.0, + 32.0, + 64.0 + ], + "requests_per_anchor": 64, + "target_pass_rate": 0.95, + "ttft_slo_ms": 1256.0 + }, + "frontier": { + "git_head": "d9cfeb6d8791fbf2f295dd9744c56a666171776e", + "git_status_short": " M frontier/config/config.py\n M frontier/entities/request.py\n M frontier/events/cluster_schedule_event.py\n M frontier/execution_time_predictor/sklearn_execution_time_predictor.py\n M frontier/metrics/constants.py\n M frontier/metrics/metrics_store.py\n M frontier/profiling/common/layers/rotary_embedding.py\n M frontier/profiling/moe/moe_impl.py\n M frontier/profiling/moe/moe_vllm_kernel.py\n M frontier/scheduler/cluster_scheduler/__init__.py\n M frontier/scheduler/cluster_scheduler/base_cluster_scheduler.py\n M frontier/scheduler/cluster_scheduler/cluster_scheduler_registry.py\n M frontier/scheduler/cluster_scheduler/sticky_lor_cluster_scheduler.py\n M frontier/scheduler/replica_scheduler/base_replica_scheduler.py\n M frontier/scheduler/replica_scheduler/vllm_v1_engine_replica_scheduler.py\n M frontier/scheduler/replica_stage_scheduler/replica_stage_schduler.py\n M frontier/simulator.py\n M frontier/types/cluster_scheduler_type.py\n?? data/profiling/compute/h20/\n?? frontier/scheduler/cluster_scheduler/prefix_lor_cluster_scheduler.py\n?? runs/\n?? tests/unit/test_attn_prefill_prediction_fallback.py\n", + "source": "/tmp/replayserve-frontier-rs1b" + }, + "profiles": { + "coverage": { + "attention": { + "1": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + }, + "2": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + }, + "4": { + "exact_prefill_2048_rows": 1, + "profile_batch_size": 1 + } + }, + "manifest": { + "attention_tp_coverage": [ + 1, + 2, + 4 + ], + "environment_contract": { + "dtype": "bfloat16", + "frontier_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e", + "hardware": "NVIDIA H20", + "model": "Qwen3-30B-A3B", + "tensor_parallel_sizes": [ + 1, + 2, + 4 + ], + "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1", + "vllm_version": "0.20.0" + }, + "inputs": { + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp2-20260716-v1-dispatch-aware-20260716T140743025781Z/artifacts/artifacts/allreduce-full-tp2-v1/raw/allreduce-tp2.json": "97c3c76b5a04e95bd9192423c2b891667c668f39cc0dfecbd097d749939f2d0a", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-allreduce-full-tp4-20260716-v1-dispatch-aware-20260716T141106009788Z/artifacts/artifacts/allreduce-full-tp4-v1/raw/allreduce-tp4.json": "809df9baa6f468cf12bf0c99827475acc67894dd9f3f948976590b665fac0e76", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp1-20260716-v2-20260716T135132587012Z/artifacts/artifacts/flashattn-kv-full-v2-tp1/raw/flashattn-tp1.json": "dcb4c1bf7e76b9c765f78ddd2b8a734f2d7ba2adac13ce017689a8a77fe69a27", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp2-20260716-v2-20260716T135134194295Z/artifacts/artifacts/flashattn-kv-full-v2-tp2/raw/flashattn-tp2.json": "43ce042556ba887c8860614b43ccf0f564e5cebc1a0cffbce299d0acb9fa8d07", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-flashattn-kv-full-tp4-20260716-v2-20260716T135135197200Z/artifacts/artifacts/flashattn-kv-full-v2-tp4/raw/flashattn-tp4.json": "84eef31bcad0f556907a093318a420959d14fdc94474823d11f659704bdfec73", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-frontier-linear-full-20260716-v2-max-tokens-20260716T144444676943Z/artifacts/artifacts/frontier-linear-full-v2/profiles/compute/h20/qwen3-a3b-30b-moe/linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-moe-full-20260716-v1-local-shard-20260716T141334565164Z/artifacts/artifacts/moe-full-v1/raw/moe-full.json": "588f6ad0d69c9636d1b852e3df0a12d13cfe731f050ea7ec7aea457cceefbde8", + "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/qwen30-vllm020-router-full-20260716-v3-tp-context-20260716T145446098505Z/artifacts/artifacts/router-full-v3/raw/router.json": "1962972e983bff3e06a721ef4ae4ec65728ff669681497a4a7e7f769b88b4931" + }, + "outputs": { + "allreduce.json": "b38d14f990578d668523d25b107aceed433da5020d8ada3b6e44d3562261a3b3", + "attention.csv": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d", + "attention_true_mixed_fused.csv": "43ef4be90bddc9aeac6dbbe339feec24162cd1f2129a08fbd959e6ee4eaf5f60", + "linear_op.csv": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "moe.csv": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0" + }, + "profile_id": "qwen3-30b-a3b-bf16-vllm020-h20-tp1-2-4-fused-mixed-total-conserving", + "projection_contract": { + "allreduce": "Frozen exact runtime measurements; base profile-only comparison keeps the historical Frontier CC backend fixed to isolate compute profile fidelity", + "attention": "Pure prefill/extend/decode FA3 core plus separately measured KV update; input/output reshape assumed zero; exported mean is used as median target; true mixed rows use a total-conserving compatibility projection", + "attention_true_mixed": "The directly measured fused total is preserved in diagnostics. Frontier's two targets are projected by the same-TP pure prefill/decode reference ratio, with projected prefill + decode exactly equal to the fused total; the split is a schema compatibility attribution, not an observation", + "linear": "Frontier profiler using vLLM 0.20 CUDA operators", + "moe": "Replicated gate and fused top-k plus TP-local modular expert kernel; expert measurement already includes prepare/finalize so shuffling is zero" + }, + "row_counts": { + "allreduce": 24, + "attention_frontier_compatible": 132, + "attention_true_mixed_fused_diagnostic": 30, + "linear": 36, + "moe": 72 + }, + "schema_version": "frontier_qwen30_vllm020_frozen_profile.v2" + } + }, + "root": "/home/gahow/phd/aituner/runs/frontier-qwen30-vllm020-profile-v1/frozen/profile-v2", + "sha256": { + "attention": "76dcb767cebb4ec1c4e24bd04d93ddd48b5d271986ebfb51a197ab33e1b3d87d", + "linear": "67666cb0a4901b74599d468df2e31bcaa2a11a7842cc0cefba24ffce62508e0c", + "manifest": "af40545e75aff55c6333cd2d5379ccf042a5a0b7d7fc7df4f745ce256cb290eb", + "moe": "0e4dcba72918a1c4cf4e96ced31ee3829248a19ad54553cebef14417725808b0" + } + }, + "schema": "frontier-qwen30-prefill-surface-v1", + "status": "partial_not_decision_bearing" +} diff --git a/runs/frontier-phase-factorial-v0/test_phase_factorial.py b/runs/frontier-phase-factorial-v0/test_phase_factorial.py new file mode 100644 index 0000000..6e30aa5 --- /dev/null +++ b/runs/frontier-phase-factorial-v0/test_phase_factorial.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + + +ROOT = Path(__file__).parent + + +def load(name: str): + path = ROOT / name + spec = importlib.util.spec_from_file_location(path.stem, path) + assert spec and spec.loader + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +def test_percentile() -> None: + client = load("qwen30_prefill_client.py") + assert client.percentile([4.0, 1.0, 3.0, 2.0], 0.50) == 2.0 + assert client.percentile([4.0, 1.0, 3.0, 2.0], 0.95) == 4.0 + assert client.percentile([], 0.95) is None + + +def test_grid_and_trace(tmp_path: Path) -> None: + surface = load("run_frontier_qwen30_prefill_surface.py") + assert len(surface.GRID) == 12 + assert {config.tp for config in surface.GRID} == {1, 2, 4} + trace = tmp_path / "trace.csv" + surface.write_trace(trace, requests=3, rate=4.0) + lines = trace.read_text().splitlines() + assert len(lines) == 4 + assert lines[1].split(",")[:3] == ["0.000000000000", "2048", "1"] + assert lines[3].split(",")[:3] == ["0.500000000000", "2048", "1"]