Merge simulator fidelity work into feature/sim
This commit is contained in:
22
.gitignore
vendored
22
.gitignore
vendored
@@ -19,3 +19,25 @@ runs/**/*.jsonl
|
||||
.ruff_cache/
|
||||
# Recovered dash1 interaction-run stores (100 MB raw tune logs, kept on disk only)
|
||||
recovered-stores/
|
||||
|
||||
# Local reference material and accidental shell output.
|
||||
/AITuner系统优化与挑战.pdf
|
||||
/16
|
||||
/docs/assets/simulator-fidelity/*.svg
|
||||
|
||||
# Generated experiment state. Protocols, analysis code, compact result tables,
|
||||
# and frozen manifests remain tracked next to these directories.
|
||||
/runs/frontier-phase-factorial-v0/fleet-artifacts*/
|
||||
/runs/frontier-phase-factorial-v0/fleet-state*/
|
||||
/runs/frontier-phase-factorial-v0/invalid-overlap-*/
|
||||
/runs/frontier-phase-factorial-v0/simulator-smoke/
|
||||
/runs/frontier-phase-factorial-v0/simulator-*/cache
|
||||
/runs/frontier-phase-factorial-v0/simulator-*/runs/
|
||||
/runs/frontier-phase-factorial-v0/simulator-*/traces/
|
||||
/runs/frontier-phase-factorial-v0/results/final/qwen30-prefill-ranking.png
|
||||
/runs/frontier-qwen30-vllm020-profile-v1/comparison/
|
||||
/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/
|
||||
/runs/frontier-qwen30-vllm020-profile-v1/fleet-state/
|
||||
/runs/frontier-multicase-sufficiency-v1/fleet-artifacts/
|
||||
/runs/frontier-multicase-sufficiency-v1/fleet-state/
|
||||
/runs/frontier-multicase-sufficiency-v1/frontier-smoke-failure/
|
||||
|
||||
75
docs/simulator-claim-map-20260716.md
Normal file
75
docs/simulator-claim-map-20260716.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Simulator-for-config-tuning related-work claim map
|
||||
|
||||
日期：2026-07-16。目的：为「Frontier/Vidur-class simulator 能否低成本解决 config tuning」这条主线建立 related-work 边界。Vidur 与 LLMServingSim 的条目基于原文（PDF 全文核读）；SimAI 基于论文页与摘要口径。每项按 Context / Claim / Assumption / Mechanism / Evidence / Boundary / 与本 project 的关系提取。
|
||||
|
||||
## Vidur（MLSys 2024，arXiv:2405.05465）
|
||||
|
||||
| 维度 | 内容 |
|
||||
|---|---|
|
||||
| Context | MSR India。首个面向 LLM inference 的大规模模拟器。Motivation 与我们一致：config search 复杂度 O(\|M\|·\|T\|)，且 optimal config 是 (model, trace) 的函数——Fig 1b 显示跨 trace misconfiguration 代价最高 2×。 |
|
||||
| Claim | (a) request-level 预测误差 <9%：static trace 上 P95 normalized execution latency 误差 ≤3.33%（4 模型 × 3 trace），dynamic trace 在 **85% capacity** 负载下误差 <5%。(b) Vidur-Search 用约 1 小时 96-core CPU（$9.93/h）为 LLaMA2-70B 找到最优 config，对比 deployment-based exploration 估算 42K GPU-hours ≈ $218K。(c) what-if 全量探索 $125 模拟成本 vs 估算 $1.14M 真机成本。 |
|
||||
| Assumption | operator runtime 可由单 GPU profiling + 小型 ML 估计器（random forest）插值；prefill attention 可用等效单序列 sqrt(Σp_i²) 近似；decode attention runtime 只依赖总 KV 读量而非 per-request context 分布；LLM 架构同质（小算子集合跨模型共享）。 |
|
||||
| Mechanism | 声明式 model spec → 算子三分类（token-level / sequence-level / communication）→ 单 GPU CUPTI profiling → RF runtime estimator → event-driven simulator + 三层 hierarchical scheduler（支持 vLLM/Orca+/Sarathi-Serve/FasterTransformer/LightLLM 策略）→ Vidur-Search 对每个 config 二分搜索 max QPS（判据 P99 scheduling delay <5s），目标 QPS/dollar。 |
|
||||
| Evidence | LLaMA2-7B/70B、InternLM-20B、Qwen-72B（全 dense）；Azure A100/H100 4-GPU pairwise-NVLink 节点；Chat-1M / Arxiv-4K / BWB-4K trace，总长截断到 4096 tokens。 |
|
||||
| Boundary | **作者明示**：接近 capacity point 时小误差会因排队失控放大，故 fidelity 评测停在 85% capacity。**结构性**：无 MoE、无 FP8/量化、无 prefix-cache reuse（多轮对话按独立请求处理）、无 speculative decoding（列为 future work）、PP 仅同步、长上下文未覆盖（4K 截断）。metric 口径为 normalized execution latency（static 排除 scheduling delay）。**最关键**：sim 选出的 config 在真机 ground-truth surface 上的 selection regret 从未被验证；42K GPU-h/$218K 是反事实估算，分母是穷举式 exploration 而非 strong sequential tuner。 |
|
||||
| 与本 project 的关系 | Frontier 即 Vidur-class（代码直接使用 vidur backend）+ 我们的 FP8/MoE/EP/decode-profile patches。我们的所有实验恰好工作在 Vidur 声明误差爆炸并回避的 regime（capacity point + SLO gate），补的正是它缺的 selection-regret ground truth。我们的 zero-shot 失败（25–30% regret）与其 <9% 不矛盾——不同 metric、不同 load regime、不同 stack alignment，论文必须主动写明这一点。其 Fig 1b 的 workload-conditioned 结论与我们 P4 sign-flip、P6 churn 互为独立佐证 → 支持 retune 频率 / amortization 论证（C3）。 |
|
||||
|
||||
## LLMServingSim（IISWC 2024，arXiv:2408.05499）
|
||||
|
||||
| 维度 | 内容 |
|
||||
|---|---|
|
||||
| Context | KAIST。scale-out LLM serving 的 HW/SW co-simulation，面向 NPU/PIM/异构加速器设计探索，基于 ASTRA-sim。 |
|
||||
| Claim | 对真实 multi-GPU vLLM serving 平均误差 14.7% 且「趋势一致」；比 mNPUsim/GeneSys/NeuPIMs 快 34.7–491×（摘要口径 91.5×）。 |
|
||||
| Assumption | iteration-level 模拟 + decoder-block 冗余复用（编译一个 block 复制展开、attention/非 attention 分离）可在可行时间内保持足够精度；硬件行为可由可插拔 accelerator compiler+simulator 栈表达（GeneSys 原型）。 |
|
||||
| Mechanism | 逐 iteration：scheduler（iteration-level batching、KV paging、operator mapping）→ per-device 硬件模拟 → graph converter（Chakra）→ ASTRA-sim 网络级模拟 → 循环。 |
|
||||
| Evidence | 与 multi-GPU vLLM 真机对照，变量为 LLM 架构、并行方案、NPU 数量、异构度；报告平均误差与趋势一致性。 |
|
||||
| Boundary | 定位是硬件/系统设计空间探索，不是 engine-knob config tuning；validation 口径是 trend-following，无 SLO-gated capacity、无 selection-regret；14.7% 平均误差大于典型 config 间 capacity margin（我们 12-cell 面上 top-2 差距 0.76%），故该精度不足以支撑近邻 config 选择。 |
|
||||
| 与本 project 的关系 | 说明「模拟保 trend」是社区通行 validation 标准；「trend ≠ selection」这一缺口对它同样成立。不构成直接 baseline，但在 related work 中界定我们评测口径（selection regret at capacity point）的必要性。 |
|
||||
|
||||
## SimAI（NSDI 2025，Alibaba，aliyun/SimAI）
|
||||
|
||||
| 维度 | 内容 |
|
||||
|---|---|
|
||||
| Context | 大规模 LLM **training** 的架构设计与参数调优模拟，生产背景（Alibaba Cloud）。 |
|
||||
| Claim | 各测试场景平均 98.1% 与真实结果对齐；为 host 设计与参数设置提供生产可用 guidance。 |
|
||||
| Assumption | training 过程可由 framework + kernel computation + collective communication 的选择性高保真集成复现。 |
|
||||
| Mechanism | 高保真集成三层栈 + 多线程加速 + lock-free global context sharing。 |
|
||||
| Evidence | 与生产 training 场景对齐（论文口径，未逐一核读实验细节）。 |
|
||||
| Boundary | training-only。training iteration 均匀、batch 组成静态——恰是 Vidur 指出 inference 所缺的性质，因此 98.1% 不可外推到 serving capacity point。 |
|
||||
| 与本 project 的关系 | 「simulator 指导 infra 决策」的工业先例与动机背书；不与 serving config tuning claim 竞争。引用价值在 motivation，不在 evaluation 对照。 |
|
||||
|
||||
## Frontier（本 project 被测对象，非 related work）
|
||||
|
||||
内部 Vidur-class 实现（vidur backend）+ 本 project 的 FP8/MoE tuning-key、Qwen MoE serving plan、TP/EP-aware cache key、critical-lane、decode/true-mixed profile 补丁。我们全部 fidelity 结论限定于该实现与已声明的 patch 集；见 `simulator-fidelity.md`。
|
||||
|
||||
## Consensus / disagreement / uncovered regime
|
||||
|
||||
**Consensus（三方一致或与我们互证）：**
|
||||
|
||||
1. operator/iteration profile + 调度复合的模拟器，在中低负载下能达到 5–15% latency 误差；模拟成本比真机低数个数量级。
|
||||
2. optimal config 是 (model, workload) 的函数，misconfiguration 代价可达 ~2×（Vidur Fig 1b；我们 P4 pattern sign-flip 与 P6 engine-churn 独立复证）。
|
||||
|
||||
**Disagreement：** 无直接冲突数字。我们的 zero-shot 失败与 Vidur <9% 处于不同 metric/regime，论文需主动解释，防止被误读为矛盾或重复。
|
||||
|
||||
**Uncovered regime（本 project 的空间）：**
|
||||
|
||||
1. **capacity-point + SLO-gated selection regret 无人用真机 ground-truth surface 验证。** Vidur 自认该 regime 误差爆炸并把评测停在 85% load；LLMServingSim 只验 trend。而 config tuning 的决策恰好发生在 capacity point。
|
||||
2. MoE、FP8、prefix reuse、speculative decoding、EP topology、长上下文均在已发表 fidelity envelope 之外。
|
||||
3. **alignment/profiling 成本从不与真机 tuning 成本同表比较。** Vidur 的 $218K 对比用穷举做分母；正确分母是 strong sequential tuner（我们实测 0.27–0.45 H20h/task，`runs/tuning-cost/metrics.json`）。
|
||||
4. envelope 失效的低成本检测（workload/runtime/topology 变化后何时还能信 simulator）无人提出。
|
||||
|
||||
## 对本 project claim 的直接影响
|
||||
|
||||
- **C1 定位句**：不是「Vidur 错了」，而是「Vidur-class claim 停在 sub-capacity load 的 prediction fidelity；把它外推到 SLO-gated capacity selection 是社区的隐含用法，我们证明该外推在 zero-shot 下失败（25–30% regret），并给出恢复 ranking 所需的最小真机证据层级」。
|
||||
- **C2**：Vidur 没有 minimum-real-evidence 的概念（要么全模拟、要么全真机）；per-TP calibration / 同栈 profile + KV capacity + patches 的证据层级是新贡献面。
|
||||
- **C3**：省钱叙事必须从「数量级」修正为「仅在 amortization 下成立」，分母换成 strong tuner 实测值；Vidur Fig 1b + 我们 P6 churn 共同支撑 retune 频率前提。
|
||||
|
||||
## 待 triage 的相邻工作（未读原文，暂不写 claim）
|
||||
|
||||
APEX（arXiv:2411.17651，并行执行计划模拟）、LLMServingSim 2.0（arXiv:2602.23036，异构+分离式）、Charon（arXiv:2605.17164，training+inference 统一）、inference-fleet-sim（arXiv:2603.16054，排队论容量规划）、AgentServeSim（arXiv:2606.09613，多轮 agent serving）。若审稿风险评估需要，按本表格式各补一行。
|
||||
|
||||
## Sources
|
||||
|
||||
- Vidur: <https://arxiv.org/abs/2405.05465>（全文核读版本：mlsys24 PDF）
|
||||
- LLMServingSim: <https://arxiv.org/pdf/2408.05499>
|
||||
- SimAI: <https://www.usenix.org/conference/nsdi25/presentation/wang-xizheng-simai>、<https://github.com/aliyun/SimAI>
|
||||
14
docs/simulator-tuning/README.md
Normal file
14
docs/simulator-tuning/README.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Simulator tuning evaluation
|
||||
|
||||
This directory contains decision-level summaries for experiments that compare
|
||||
a serving simulator's selected configuration with the best configuration on
|
||||
real hardware.
|
||||
|
||||
Current report:
|
||||
|
||||
- [Frontier selection regret on Qwen3-30B and Qwen3-235B](frontier-selection-regret-qwen30-qwen235-20260719.md)
|
||||
|
||||
The primary quantity is **real-hardware selection regret**, not simulator
|
||||
absolute-latency error. Raw commands, profiles, traces, and experiment-specific
|
||||
audit records remain under `runs/` or in the immutable remote artifact roots
|
||||
listed by each report.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Frontier selection regret: Qwen3-30B and Qwen3-235B
|
||||
|
||||
> Date: 2026-07-19
|
||||
> Scope: H20, community vLLM 0.20, Frontier piecewise simulation, no SLO gate
|
||||
|
||||
## Question and metric
|
||||
|
||||
For each workload and latency objective, Frontier selects the configuration
|
||||
with the lowest simulated latency. We then look up that configuration on the
|
||||
complete real-hardware surface and compare it with the real-hardware optimum.
|
||||
|
||||
```text
|
||||
selection regret = real_latency(Frontier winner) / real_latency(real winner) - 1
|
||||
```
|
||||
|
||||
Lower is better. `0%` means Frontier selected the real winner. Positive values
|
||||
mean that following Frontier produces slower real serving. Each objective is
|
||||
selected independently; this table does not combine TTFT, TPOT, and E2E into a
|
||||
single score.
|
||||
|
||||
## Qwen3-30B-A3B
|
||||
|
||||
Configuration surface: `TP in {1,2,4} x MNS in {8,16,32,64}`, with
|
||||
`MBT=8192`. Each real cell uses three fresh-server trials.
|
||||
|
||||
| Workload | TTFT mean | TTFT p90 | TPOT mean | TPOT p90 | E2E mean | E2E p90 |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| Trace-PD | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |
|
||||
| Fixed-PD, 4096->256, 1.125 req/s/GPU | **58.0%** | **56.2%** | 0.0% | 0.0% | 1.7% | 5.5% |
|
||||
| Trace-PO, OSL=1 | 3.2% | 0.4% | N/A | N/A | 3.2% | 0.3% |
|
||||
| Fixed-PO, 4096->1, 1.125 req/s/GPU | 0.3% | 0.5% | N/A | N/A | 0.3% | 0.5% |
|
||||
|
||||
Interpretation: Frontier is near-optimal for Trace-PD and both prefill-only
|
||||
cases, but the high-pressure Fixed-PD TTFT choice is materially wrong: its
|
||||
selected configuration is 56--58% slower than the real TTFT optimum.
|
||||
|
||||
## Qwen3-235B-A22B-FP8
|
||||
|
||||
Configuration surface: `{TP4/EP1, TP8/EP8} x MNS in {64,128}`, with
|
||||
`MBT=8192`. Each workload has 129 requests per cell and each real cell uses
|
||||
three fresh-server trials.
|
||||
|
||||
| Workload | TTFT mean | TTFT p90 | TPOT mean | TPOT p90 | E2E mean | E2E p90 |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| Trace-PD | 0.0% | 0.0% | 0.0% | 0.0% | 0.6% | 6.2% |
|
||||
| Fixed-PD, 4096->256, 0.2 req/s/GPU | 4.2% | 0.2% | **33.0%** | **37.2%** | **30.7%** | **34.6%** |
|
||||
| Trace-PO, OSL=1 | 7.0% | **21.2%** | N/A | N/A | 7.0% | **21.2%** |
|
||||
| Fixed-PO, 4096->1, 0.2 req/s/GPU | 5.9% | 1.7% | N/A | N/A | 5.9% | 1.7% |
|
||||
|
||||
Interpretation: Trace-PD is mostly near-optimal. Fixed-PD reverses the real
|
||||
decode/E2E preference between the tested parallel configurations and incurs
|
||||
31--37% regret. Trace-PO also has a material p90 failure of 21.2%.
|
||||
|
||||
## Decision
|
||||
|
||||
The tested Frontier stack has **not** solved serving configuration tuning.
|
||||
Its selected configuration can be near-optimal for one workload and materially
|
||||
wrong for another on the same model and hardware. The strongest current
|
||||
counterexamples are Qwen3-30B Fixed-PD TTFT and Qwen3-235B Fixed-PD TPOT/E2E.
|
||||
|
||||
This statement is limited to the two tested MoE models and Frontier. It is not
|
||||
yet evidence about dense models, Vidur/APEX as separately reproduced systems,
|
||||
other hardware, or SLO-constrained tuning.
|
||||
|
||||
## Provenance
|
||||
|
||||
Primary immutable analysis artifacts on `dash0`:
|
||||
|
||||
- Qwen3-30B Trace-PD: `/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis/comparison.json`
|
||||
- Qwen3-30B Fixed-PD/PO: `/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/analysis/`
|
||||
- Qwen3-30B Trace-PO: `/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/trace-po-comparison.json`
|
||||
- Qwen3-235B four-case matrix: `/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/analysis/comparison.json`
|
||||
|
||||
The Qwen3-235B artifact root includes `provenance/artifacts.sha256`; the final
|
||||
matrix contains 48/48 valid real trials and 16/16 complete simulator cells.
|
||||
@@ -182,6 +182,59 @@ def rsync_pull(config: FleetConfig, host: HostSpec, remote_path: str, local_path
|
||||
run_local(argv, cwd=config.project_root, capture_output=True, check=True)
|
||||
|
||||
|
||||
def scp_push(config: FleetConfig, host: HostSpec) -> None:
|
||||
ensure_remote_dir(config, host, host.sync_remote_path)
|
||||
local_src = str(config.sync.local_path.resolve()) + "/."
|
||||
remote_dst = f"{host.ssh_alias}:{host.sync_remote_path.rstrip('/')}/"
|
||||
argv = [
|
||||
"scp",
|
||||
"-o",
|
||||
"BatchMode=yes",
|
||||
"-o",
|
||||
f"ConnectTimeout={config.ssh_timeout_sec}",
|
||||
"-r",
|
||||
"-p",
|
||||
local_src,
|
||||
remote_dst,
|
||||
]
|
||||
run_local(argv, cwd=config.project_root, capture_output=True, check=True)
|
||||
|
||||
|
||||
def scp_pull(config: FleetConfig, host: HostSpec, remote_path: str, local_path: Path) -> None:
|
||||
remote_src = remote_path
|
||||
if remote_path.endswith("/"):
|
||||
ensure_dir(local_path)
|
||||
remote_src = remote_path.rstrip("/") + "/."
|
||||
else:
|
||||
ensure_dir(local_path.parent)
|
||||
argv = [
|
||||
"scp",
|
||||
"-o",
|
||||
"BatchMode=yes",
|
||||
"-o",
|
||||
f"ConnectTimeout={config.ssh_timeout_sec}",
|
||||
"-r",
|
||||
"-p",
|
||||
f"{host.ssh_alias}:{remote_src}",
|
||||
str(local_path),
|
||||
]
|
||||
run_local(argv, cwd=config.project_root, capture_output=True, check=True)
|
||||
|
||||
|
||||
def sync_push(config: FleetConfig, host: HostSpec) -> None:
|
||||
if config.sync.mode == "rsync":
|
||||
rsync_push(config, host)
|
||||
else:
|
||||
scp_push(config, host)
|
||||
|
||||
|
||||
def sync_pull(config: FleetConfig, host: HostSpec, remote_path: str, local_path: Path) -> None:
|
||||
if config.sync.mode == "rsync":
|
||||
rsync_pull(config, host, remote_path, local_path)
|
||||
else:
|
||||
scp_pull(config, host, remote_path, local_path)
|
||||
|
||||
|
||||
def ensure_remote_dir(config: FleetConfig, host: HostSpec, remote_path: str) -> None:
|
||||
run_ssh(config, host, f"mkdir -p {shlex.quote(remote_path)}", capture_output=True, check=True)
|
||||
|
||||
@@ -206,8 +259,10 @@ def load_config(path: Path) -> FleetConfig:
|
||||
local_path=relative_to_root(project_root, sync_raw.get("local_path"), project_root),
|
||||
exclude=[str(item) for item in sync_raw.get("exclude", [])],
|
||||
)
|
||||
if sync.mode != "rsync":
|
||||
if sync.mode not in {"rsync", "scp"}:
|
||||
raise FleetError(f"unsupported sync.mode: {sync.mode}")
|
||||
if sync.mode == "scp" and sync.exclude:
|
||||
raise FleetError("sync.exclude is not supported for sync.mode=scp")
|
||||
|
||||
scheduler_raw = raw.get("scheduler", {})
|
||||
scheduler = SchedulerSpec(
|
||||
@@ -639,7 +694,7 @@ def harvest_run(config: FleetConfig, manifest: dict[str, Any]) -> dict[str, Any]
|
||||
|
||||
local_base = ensure_dir(config.artifacts_dir / refreshed["run_id"])
|
||||
remote_run_dir = refreshed["remote_run_dir"].rstrip("/")
|
||||
rsync_pull(config, host, f"{remote_run_dir}/", local_base / "remote_run")
|
||||
sync_pull(config, host, f"{remote_run_dir}/", local_base / "remote_run")
|
||||
|
||||
for artifact in refreshed.get("artifacts", []):
|
||||
artifact_remote = f"{refreshed['remote_sync_path'].rstrip('/')}/{artifact}"
|
||||
@@ -652,7 +707,7 @@ def harvest_run(config: FleetConfig, manifest: dict[str, Any]) -> dict[str, Any]
|
||||
check=False,
|
||||
)
|
||||
if check.returncode == 0:
|
||||
rsync_pull(config, host, artifact_remote, target)
|
||||
sync_pull(config, host, artifact_remote, target)
|
||||
|
||||
refreshed["harvested_at"] = utc_now()
|
||||
write_run_manifest(config, refreshed)
|
||||
@@ -866,7 +921,7 @@ def dispatch_jobs(
|
||||
)
|
||||
continue
|
||||
if host.name not in synced_hosts:
|
||||
rsync_push(config, host)
|
||||
sync_push(config, host)
|
||||
synced_hosts.add(host.name)
|
||||
manifest = launch_job(config, host, job, gpu_ids)
|
||||
manifests.append(manifest)
|
||||
|
||||
@@ -8,6 +8,7 @@ simulator-f1/
|
||||
simulator-f1-steady/
|
||||
simulator-f2/
|
||||
simulator-t1/
|
||||
simulator-t1-v4/
|
||||
fleet-state/
|
||||
fleet-artifacts/
|
||||
profiles/profile-v3-batch/
|
||||
|
||||
100
runs/frontier-fidelity-envelope-v1/augment_qwen235_v020_true_mixed_profiles.py
Executable file
100
runs/frontier-fidelity-envelope-v1/augment_qwen235_v020_true_mixed_profiles.py
Executable file
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Append measured true-mixed attention rows to an immutable Qwen235 profile."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
MODEL = "Qwen3-235B-A22B"
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--base-frozen", type=Path, required=True)
|
||||
parser.add_argument("--cuda-tp4", type=Path, required=True)
|
||||
parser.add_argument("--cuda-tp8", type=Path, required=True)
|
||||
parser.add_argument("--kernel-tp4", type=Path, required=True)
|
||||
parser.add_argument("--kernel-tp8", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def digest(path: Path) -> str:
|
||||
value = hashlib.sha256()
|
||||
value.update(path.read_bytes())
|
||||
return value.hexdigest()
|
||||
|
||||
|
||||
def read_rows(path: Path) -> tuple[list[str], list[dict[str, str]]]:
|
||||
with path.open(newline="") as source:
|
||||
reader = csv.DictReader(source)
|
||||
return list(reader.fieldnames or []), list(reader)
|
||||
|
||||
|
||||
def merge_csv(inputs: list[Path], output: Path, measurement_type: str) -> int:
|
||||
fields: list[str] = []
|
||||
rows: list[dict[str, str]] = []
|
||||
true_mixed_count = 0
|
||||
for index, path in enumerate(inputs):
|
||||
input_fields, input_rows = read_rows(path)
|
||||
for field in input_fields:
|
||||
if field not in fields:
|
||||
fields.append(field)
|
||||
if index:
|
||||
for row in input_rows:
|
||||
if row.get("is_true_mixed_batch", "").lower() != "true":
|
||||
raise ValueError(f"non-true-mixed row in {path}")
|
||||
if row.get("measurement_type") != measurement_type:
|
||||
raise ValueError(f"measurement type mismatch in {path}")
|
||||
true_mixed_count += len(input_rows)
|
||||
rows.extend(input_rows)
|
||||
if not fields or not rows or not true_mixed_count:
|
||||
raise ValueError("cannot assemble an empty true-mixed augmentation")
|
||||
with output.open("w", newline="") as target:
|
||||
writer = csv.DictWriter(target, fieldnames=fields, lineterminator="\n")
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
return true_mixed_count
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
output = args.output_root.resolve()
|
||||
if output.exists():
|
||||
raise FileExistsError(output)
|
||||
output.mkdir(parents=True)
|
||||
for name in ("linear_op.csv", "linear_op_kernel_only.csv", "moe.csv", "moe_kernel_only.csv"):
|
||||
shutil.copy2(args.base_frozen / name, output / name)
|
||||
cuda_count = merge_csv(
|
||||
[args.base_frozen / "attention.csv", args.cuda_tp4, args.cuda_tp8],
|
||||
output / "attention.csv",
|
||||
"CUDA_EVENT",
|
||||
)
|
||||
kernel_count = merge_csv(
|
||||
[args.base_frozen / "attention_kernel_only.csv", args.kernel_tp4, args.kernel_tp8],
|
||||
output / "attention_kernel_only.csv",
|
||||
"KERNEL_ONLY",
|
||||
)
|
||||
base_manifest = json.loads((args.base_frozen / "manifest.json").read_text())
|
||||
manifest = {
|
||||
**base_manifest,
|
||||
"schema": "qwen235-v020-frontier-profile-v2-true-mixed",
|
||||
"true_mixed_inputs": {
|
||||
name: str(getattr(args, name).resolve())
|
||||
for name in ("cuda_tp4", "cuda_tp8", "kernel_tp4", "kernel_tp8")
|
||||
},
|
||||
"true_mixed_rows": {"CUDA_EVENT": cuda_count, "KERNEL_ONLY": kernel_count},
|
||||
"outputs": {path.name: digest(path) for path in sorted(output.glob("*.csv"))},
|
||||
}
|
||||
(output / "manifest.json").write_text(json.dumps(manifest, indent=2, sort_keys=True) + "\n")
|
||||
print(json.dumps(manifest, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,8 +1,8 @@
|
||||
# EXP-SIMFID-Q235-V020-PORTABILITY: large FP8 MoE boundary case
|
||||
|
||||
> Status: compatibility gate prepared. No Qwen235 vLLM 0.20/deadc4a profile or
|
||||
> real/simulator latency result exists yet; historical vLLM 0.10.2 SLO data is
|
||||
> explicitly excluded.
|
||||
> Status: initial compatibility failure diagnosed on 2026-07-18; the vLLM
|
||||
> 0.20 profile path was subsequently repaired and the four-case matrix
|
||||
> completed on 2026-07-19. Historical vLLM 0.10.2 SLO data remains excluded.
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -39,3 +39,42 @@ the two plans above have existing historical memory/topology evidence.
|
||||
|
||||
This ordering prevents a profile-runtime mismatch from being misreported as a
|
||||
simulator tuning failure.
|
||||
|
||||
## Initial result: the real plans ran before Frontier had an admissible counterpart
|
||||
|
||||
Both real vLLM 0.20 portability gates completed one exact Fixed-P request
|
||||
(`ISL=2048`, `OSL=1`, prefix off) on H20. These are startup/semantic gates,
|
||||
not estimates of mean or p90 latency.
|
||||
|
||||
| Real plan | Runtime-resolved MoE state | Completed request | TTFT / E2E |
|
||||
|---|---|---:|---:|
|
||||
| TP4/EP1, MNS64 | FP8 MoE `TRITON`, 128 experts per rank | 1/1 | 1655.99 / 1656.04 ms |
|
||||
| TP8/EP8, MNS64 | FP8 MoE `FLASHINFER_CUTLASS`, 16/128 experts per EP rank | 1/1 | 720.54 / 720.58 ms |
|
||||
|
||||
The matching Frontier `deadc4a` profile gate failed before any GPU measurement:
|
||||
|
||||
1. `Qwen3-235B-A22B.json` exposes `torch_dtype=BF16` but no FP8
|
||||
`quantization_config` / `[128,128]` block shape, while the checkpoint uses
|
||||
dynamic block-FP8 weights. Frontier rejects the valid request as a profile
|
||||
configuration mismatch.
|
||||
2. Its MoE profiler then imports the removed vLLM symbol
|
||||
`invoke_fused_moe_kernel`, so it is not ABI-compatible with the installed
|
||||
community vLLM 0.20 runtime.
|
||||
|
||||
This was an admissible intermediate coverage failure. Filling it with
|
||||
historical vLLM 0.10.2 profiles would have changed both the runtime and the
|
||||
selected kernels and would not have answered the stated comparison question.
|
||||
|
||||
## Final result after profile repair
|
||||
|
||||
The later campaign added graph-compatible kernel-only decode profiles and true
|
||||
mixed attention coverage, then completed all four workloads on the frozen
|
||||
`{TP4/EP1, TP8/EP8} x MNS{64,128}` surface. All 48 real fresh-server trials
|
||||
and all 16 Frontier cells completed. The decision-level selection regrets are
|
||||
maintained in the consolidated
|
||||
[Qwen30/Qwen235 report](../../docs/simulator-tuning/frontier-selection-regret-qwen30-qwen235-20260719.md).
|
||||
|
||||
The decisive counterexample is Fixed-PD: Frontier's selected configuration is
|
||||
33.0--37.2% slower for real TPOT and 30.7--34.6% slower for real E2E than the
|
||||
real-hardware optimum. Thus repairing the coverage gate enabled a valid
|
||||
comparison but did not establish tuning fidelity.
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# EXP-SIMFID-Q30-LATENCY-EXPANSION: remaining Fixed/Trace × PD/P cases
|
||||
|
||||
> Status: prepared for runtime-alignment preflight (2026-07-18). The already
|
||||
> completed Qwen3-30B-A3B Trace-PD surface is excluded from this card.
|
||||
> Status: completed (2026-07-18). The low-load Fixed cases below are retained
|
||||
> as sensitivity evidence; the later 4096-token, 1.125 req/s/GPU Fixed matrix
|
||||
> is the primary comparison. Trace-PD is excluded from this card.
|
||||
|
||||
## Question and fixed boundary
|
||||
|
||||
@@ -47,3 +48,45 @@ metric is a coverage failure, not a high-latency cell.
|
||||
The no-request Fixed runtime preflight is capped at 2 H20-GPUh. Each 36-run
|
||||
real surface is estimated at 13 nominal / 41 worst-case H20-GPUh, plus CPU-only
|
||||
Frontier replay. Launch logs record the resolved inputs, paths, and duration.
|
||||
|
||||
## Result: exact-selection fidelity is not stable across workload phase/state
|
||||
|
||||
All three real surfaces completed 36 valid fresh-server trials (12
|
||||
TP×MNS cells × 3 trials), each with 129/129 exact-usage requests. The
|
||||
corresponding 12-cell Frontier replays completed all 129 requests per cell.
|
||||
The following table compares the simulator argmin with the pooled real argmin;
|
||||
`regret` is the selected simulator configuration's real relative latency
|
||||
regret. It deliberately reports exact winner equality rather than treating a
|
||||
high pairwise-order agreement as success.
|
||||
|
||||
| Case | Objective | Frontier winner | Real winner | Match | Real regret |
|
||||
|---|---|---|---|---:|---:|
|
||||
| Trace-P (OSL=1) | TTFT mean | TP4/MNS16 | TP4/MNS64 | no | 3.2% |
|
||||
| Trace-P (OSL=1) | TTFT p90 | TP4/MNS16 | TP4/MNS8 | no | 0.4% |
|
||||
| Trace-P (OSL=1) | E2E mean | TP4/MNS16 | TP4/MNS64 | no | 3.2% |
|
||||
| Trace-P (OSL=1) | E2E p90 | TP4/MNS16 | TP4/MNS8 | no | 0.3% |
|
||||
| Fixed-PD | TTFT mean/p90 | TP4/MNS16 | TP4/MNS16 | yes | 0.0% |
|
||||
| Fixed-PD | TPOT mean | TP1/MNS16 | TP4/MNS32 | no | 14.0% |
|
||||
| Fixed-PD | TPOT p90 | TP1/MNS16 | TP4/MNS8 | no | 13.4% |
|
||||
| Fixed-PD | E2E mean | TP1/MNS16 | TP4/MNS32 | no | 26.9% |
|
||||
| Fixed-PD | E2E p90 | TP1/MNS16 | TP4/MNS8 | no | 25.7% |
|
||||
| Fixed-P (OSL=1) | TTFT/E2E mean/p90 | TP4/MNS16 | TP4/MNS8 | no | 1.5--3.6% |
|
||||
|
||||
The decisive failure is Fixed-PD: its TPOT and E2E pairwise orders are exactly
|
||||
reversed on all 48 informative pairs. Thus the graph-aligned profile can
|
||||
preserve coarse TP ordering yet still choose the wrong configuration for the
|
||||
serving objective. The prior Trace-PD success is therefore a workload-specific
|
||||
success, not evidence that Frontier has solved config tuning.
|
||||
|
||||
## Superseding primary Fixed workload
|
||||
|
||||
The subsequent pressure sweep changed the Fixed shapes to 4096->256 and
|
||||
4096->1, raised the offered load to 1.125 req/s/GPU, and used 257 requests per
|
||||
cell. Those Fixed-PD/PO results, together with Trace-PD and Trace-PO, are
|
||||
reported in the consolidated
|
||||
[Qwen30/Qwen235 selection-regret report](../../docs/simulator-tuning/frontier-selection-regret-qwen30-qwen235-20260719.md).
|
||||
|
||||
Under that primary Fixed-PD workload, Frontier's selected config is 58.0%
|
||||
slower for real mean TTFT and 56.2% slower for real p90 TTFT than the
|
||||
real-hardware optimum. The low-load table above must therefore not be quoted
|
||||
as the final Fixed-PD result.
|
||||
|
||||
@@ -171,7 +171,7 @@ async def replay(args: argparse.Namespace, rows: list[dict[str, Any]]) -> list[d
|
||||
import aiohttp
|
||||
|
||||
timeout = aiohttp.ClientTimeout(total=args.timeout_seconds)
|
||||
connector = aiohttp.TCPConnector(limit=0, ttl_dns_cache=300)
|
||||
connector = aiohttp.TCPConnector(limit=0, ttl_dns_cache=300, force_close=True)
|
||||
benchmark_start = asyncio.get_running_loop().time() + 2.0
|
||||
async with aiohttp.ClientSession(
|
||||
base_url=f"http://{args.host}:{args.port}",
|
||||
@@ -229,6 +229,7 @@ def main() -> None:
|
||||
"arrival": "original_trace_timestamp_and_order",
|
||||
"input_output_prompt": "exact_source_values",
|
||||
"served_model_alias": args.served_model,
|
||||
"http_connection_reuse": False,
|
||||
"ttft_slo": "1000ms + 1000ms * input_tokens / 8000",
|
||||
"tpot_slo_ms": args.tpot_slo_ms,
|
||||
"target_pass_rate": TARGET_PASS_RATE,
|
||||
|
||||
@@ -112,6 +112,7 @@ def knobs(config: Config, paths: dict[str, Path], contract: dict, cache: Path, p
|
||||
"batch_size_cap": config.mns,
|
||||
"max_tokens_in_batch": 8192,
|
||||
"long_prefill_token_threshold": 0,
|
||||
"enable_chunked_prefill": True,
|
||||
"block_size": 16,
|
||||
"num_blocks_mode": "explicit",
|
||||
"num_blocks": int(resolved["num_gpu_blocks"]),
|
||||
@@ -180,7 +181,9 @@ def main() -> None:
|
||||
run_dir = args.output_root / "runs" / config.name / trace["label"]
|
||||
result_path = run_dir / "result.json"
|
||||
if args.resume and result_path.is_file():
|
||||
results.append(json.loads(result_path.read_text()))
|
||||
previous = json.loads(result_path.read_text())
|
||||
if previous.get("status") == "completed":
|
||||
results.append(previous)
|
||||
continue
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
command = builder.build_frontier_command(
|
||||
|
||||
@@ -13,6 +13,8 @@ VENV_ROOT="${VENV_ROOT:-/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1}"
|
||||
# FlashInfer's TensorRT-LLM MoE runtime invokes `nvcc` by name when it
|
||||
# autotunes a previously unseen FP8 grouped-GEMM shape.
|
||||
export PATH="/usr/local/cuda/bin:${PATH}"
|
||||
export FRONTIER_VLLM_MODEL_TYPE=qwen3_moe
|
||||
export VLLM_KV_CACHE_LAYOUT=NHD
|
||||
command -v nvcc >/dev/null
|
||||
|
||||
mkdir -p "${OUTPUT_ROOT}/supervisor" "${PROFILE_ROOT}"
|
||||
|
||||
@@ -53,8 +53,8 @@ run_real() {
|
||||
}
|
||||
run_real fixed-pd false 9300
|
||||
run_real fixed-po false 9400
|
||||
run_real trace-pd true 9500
|
||||
run_real trace-po true 9600
|
||||
run_real trace-pd true 9800
|
||||
run_real trace-po true 9900
|
||||
|
||||
"${VENV_ROOT}/bin/python" "${RUNNER_DIR}/extract_qwen235_v020_runtime_contract.py" \
|
||||
--case-root "${CAMPAIGN_ROOT}/real/fixed-pd" \
|
||||
|
||||
@@ -35,8 +35,9 @@ common_profile() {
|
||||
env CUDA_VISIBLE_DEVICES="${gpu}" PYTHONPATH="${FRONTIER_SOURCE}" \
|
||||
"${VENV_ROOT}/bin/python" -m frontier.profiling.attention.main \
|
||||
--disable_ray --num_gpus 1 --device h20 --output_dir "${root}" --models "${MODEL}" \
|
||||
--num_tensor_parallel_workers 4 8 --max_model_len 40960 --max_seq_len 8192 \
|
||||
--batch_size_list "${BATCHES[@]}" --decode_kv_cache_size_list "${KV[@]}" \
|
||||
--num_tensor_parallel_workers 4 8 --max_model_len 40960 --max_seq_len 40960 \
|
||||
--max_batch_size 256 --batch_size_list "${BATCHES[@]}" \
|
||||
--decode_kv_cache_size_list "${KV[@]}" \
|
||||
--fixed_chunked_prefill_size 8192 --attention_backend FLASHINFER \
|
||||
--profile_method "${method}" --yes
|
||||
}
|
||||
|
||||
@@ -15,10 +15,14 @@ VENV_ROOT="${VENV_ROOT:-/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1}"
|
||||
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8}"
|
||||
SERVER_READY_ATTEMPTS="${SERVER_READY_ATTEMPTS:-600}"
|
||||
IDLE_GPU_MEMORY_TOLERANCE_MIB="${IDLE_GPU_MEMORY_TOLERANCE_MIB:-16}"
|
||||
IDLE_GPU_SETTLE_ATTEMPTS="${IDLE_GPU_SETTLE_ATTEMPTS:-120}"
|
||||
RESUME_VALID_CELLS="${RESUME_VALID_CELLS:-true}"
|
||||
PORT="${BASE_PORT:-9300}"
|
||||
REQUEST_COUNT="$(wc -l < "${TRACE_ROOT}/tp4/private/real_requests.jsonl")"
|
||||
|
||||
export PATH="/usr/local/cuda/bin:${PATH}"
|
||||
command -v nvcc >/dev/null
|
||||
|
||||
[[ "$(wc -l < "${TRACE_ROOT}/tp8/private/real_requests.jsonl")" == "${REQUEST_COUNT}" ]] || {
|
||||
echo 'ERROR: TP-specific request counts differ' >&2
|
||||
exit 1
|
||||
@@ -40,10 +44,18 @@ PY
|
||||
}
|
||||
|
||||
preflight_gpus() {
|
||||
local attempt
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader \
|
||||
| tee -a "${OUT}/controller.log"
|
||||
nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits \
|
||||
| awk -v tolerance="${IDLE_GPU_MEMORY_TOLERANCE_MIB}" '$1 > tolerance {exit 1}'
|
||||
for ((attempt = 1; attempt <= IDLE_GPU_SETTLE_ATTEMPTS; attempt++)); do
|
||||
if nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits \
|
||||
| awk -v tolerance="${IDLE_GPU_MEMORY_TOLERANCE_MIB}" '$1 > tolerance {exit 1}'; then
|
||||
return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
echo "ERROR: GPU memory did not settle below ${IDLE_GPU_MEMORY_TOLERANCE_MIB} MiB after ${IDLE_GPU_SETTLE_ATTEMPTS}s" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
assert_no_server() {
|
||||
@@ -60,7 +72,8 @@ wait_for_wave() {
|
||||
launch_config() {
|
||||
local trial="$1" tp="$2" mns="$3" gpus="$4" ep=false ep_size=1
|
||||
if [[ "${tp}" == 8 ]]; then ep=true; ep_size=8; fi
|
||||
local config="tp${tp}_ep${ep_size}_mns${mns}" run_out="${OUT}/real/${config}/trial${trial}"
|
||||
local config="tp${tp}_ep${ep_size}_mns${mns}"
|
||||
local run_out="${OUT}/real/${config}/trial${trial}"
|
||||
local requests="${TRACE_ROOT}/tp${tp}/private/real_requests.jsonl" port="${PORT}"
|
||||
PORT=$((PORT + 1))
|
||||
if [[ "${RESUME_VALID_CELLS}" == true ]] && has_valid_result "${run_out}/results/result.json"; then
|
||||
@@ -97,12 +110,16 @@ run_trial() {
|
||||
"${CASE_NAME}" "${trial}" "${mnss[0]}" "${mnss[1]}" | tee -a "${OUT}/controller.log"
|
||||
launch_config "${trial}" 4 "${mnss[0]}" '0,1,2,3'
|
||||
launch_config "${trial}" 4 "${mnss[1]}" '4,5,6,7'
|
||||
wait_for_wave && preflight_gpus && assert_no_server
|
||||
wait_for_wave
|
||||
preflight_gpus
|
||||
assert_no_server
|
||||
for mns in "${mnss[@]}"; do
|
||||
printf 'WAVE_START case=%s trial=%s tp=8 mns=%s\n' \
|
||||
"${CASE_NAME}" "${trial}" "${mns}" | tee -a "${OUT}/controller.log"
|
||||
launch_config "${trial}" 8 "${mns}" '0,1,2,3,4,5,6,7'
|
||||
wait_for_wave && preflight_gpus && assert_no_server
|
||||
wait_for_wave
|
||||
preflight_gpus
|
||||
assert_no_server
|
||||
done
|
||||
}
|
||||
|
||||
|
||||
@@ -54,6 +54,11 @@ if [[ ! -x "${TRTLLM_DG_NVCC_COMPILER}" ]]; then
|
||||
echo "ERROR: TRTLLM_DG_NVCC_COMPILER is not executable: ${TRTLLM_DG_NVCC_COMPILER}" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! "${VENV_ROOT}/bin/python" "${CLIENT}" --help 2>&1 \
|
||||
| grep -F --quiet -- '--model-path'; then
|
||||
echo "ERROR: CLIENT must implement the fixed-shape client contract (--model-path, --rate, --input-tokens, --output-tokens): ${CLIENT}" >&2
|
||||
exit 1
|
||||
fi
|
||||
case "${DEEP_GEMM_JIT_DEBUG}" in
|
||||
true|false) ;;
|
||||
*) echo "ERROR: DEEP_GEMM_JIT_DEBUG must be true or false" >&2; exit 1 ;;
|
||||
|
||||
51
runs/frontier-fidelity-envelope-v1/run_qwen235_v020_true_mixed_profiles.sh
Executable file
51
runs/frontier-fidelity-envelope-v1/run_qwen235_v020_true_mixed_profiles.sh
Executable file
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:?OUTPUT_ROOT is required}"
|
||||
FRONTIER_SOURCE="${FRONTIER_SOURCE:-/home/admin/cpfs/wjh/aituner/frontier-q235-v020-5b953f5}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1}"
|
||||
MODEL=Qwen3-235B-A22B
|
||||
PREFILL_BATCHES=(1 2 4 8)
|
||||
PREFILL_CHUNKS=(128 512 2048 8192)
|
||||
DECODE_BATCHES=(1 8 32 64 120)
|
||||
DECODE_KV=(128 1024 4096 16384 32768)
|
||||
|
||||
mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/provenance"
|
||||
exec > >(tee -a "${OUTPUT_ROOT}/controller.log") 2>&1
|
||||
|
||||
echo "Q235_TRUE_MIXED_PROFILE_LAUNCH_ECHO host=dash0 model=${MODEL} vllm=0.20.0 frontier=$(git -C "${FRONTIER_SOURCE}" rev-parse HEAD) device=H20 methods={CUDA_EVENT,KERNEL_ONLY} tp={4,8} grid=4x4x5x5 parallel_gpus=4 expected_wall=10-30m expected_cost=0.7-2_H20-GPUh output=${OUTPUT_ROOT}"
|
||||
date -u +START_UTC=%Y-%m-%dT%H:%M:%SZ
|
||||
nvidia-smi --query-gpu=index,memory.used --format=csv,noheader,nounits \
|
||||
| awk '$2 > 16 {exit 1}'
|
||||
git -C "${FRONTIER_SOURCE}" rev-parse HEAD > "${OUTPUT_ROOT}/provenance/frontier.commit"
|
||||
"${VENV_ROOT}/bin/vllm" --version > "${OUTPUT_ROOT}/provenance/vllm.version"
|
||||
cd "${FRONTIER_SOURCE}"
|
||||
|
||||
profile() {
|
||||
local gpu="$1" method="$2" tp="$3" root="$4"
|
||||
env CUDA_VISIBLE_DEVICES="${gpu}" VLLM_KV_CACHE_LAYOUT=NHD PYTHONPATH="${FRONTIER_SOURCE}" \
|
||||
"${VENV_ROOT}/bin/python" -m frontier.profiling.attention.main \
|
||||
--disable_ray --num_gpus 1 --device h20 --output_dir "${root}" --models "${MODEL}" \
|
||||
--num_tensor_parallel_workers "${tp}" --max_model_len 40960 --max_seq_len 40960 \
|
||||
--max_batch_size 128 --batch_size_list 1 --decode_kv_cache_size_list 128 \
|
||||
--fixed_chunked_prefill_size 8192 --attention_backend FLASHINFER \
|
||||
--enable_true_mixed \
|
||||
--true_mixed_prefill_batch_sizes "${PREFILL_BATCHES[@]}" \
|
||||
--true_mixed_prefill_chunk_sizes "${PREFILL_CHUNKS[@]}" \
|
||||
--true_mixed_decode_batch_sizes "${DECODE_BATCHES[@]}" \
|
||||
--true_mixed_decode_kv_cache_sizes "${DECODE_KV[@]}" \
|
||||
--true_mixed_prefill_kv_cache_size 0 \
|
||||
--profile_method "${method}" --yes
|
||||
}
|
||||
|
||||
declare -a pids=()
|
||||
profile 0 cuda_event 4 "${OUTPUT_ROOT}/cuda-tp4" > "${OUTPUT_ROOT}/logs/cuda-tp4.log" 2>&1 & pids+=("$!")
|
||||
profile 1 record_function 4 "${OUTPUT_ROOT}/kernel-tp4" > "${OUTPUT_ROOT}/logs/kernel-tp4.log" 2>&1 & pids+=("$!")
|
||||
profile 2 cuda_event 8 "${OUTPUT_ROOT}/cuda-tp8" > "${OUTPUT_ROOT}/logs/cuda-tp8.log" 2>&1 & pids+=("$!")
|
||||
profile 3 record_function 8 "${OUTPUT_ROOT}/kernel-tp8" > "${OUTPUT_ROOT}/logs/kernel-tp8.log" 2>&1 & pids+=("$!")
|
||||
failed=0
|
||||
for pid in "${pids[@]}"; do wait "${pid}" || failed=1; done
|
||||
[[ "${failed}" -eq 0 ]] || { tail -n 80 "${OUTPUT_ROOT}"/logs/*.log; exit 1; }
|
||||
date -u +END_UTC=%Y-%m-%dT%H:%M:%SZ
|
||||
echo Q235_V020_TRUE_MIXED_PROFILES_COMPLETE
|
||||
280
runs/frontier-multicase-sufficiency-v0/audit_ground_truth.py
Normal file
280
runs/frontier-multicase-sufficiency-v0/audit_ground_truth.py
Normal file
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audit the Qwen235B real-machine surfaces before comparing Frontier.
|
||||
|
||||
This script intentionally does not consume simulator output. It establishes
|
||||
whether each real response surface is complete and discriminative enough to
|
||||
support a later claim about simulator config selection.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
SCHEMA = "frontier-multicase-ground-truth-v0"
|
||||
EXPECTED_PROBES = 6
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict[str, Any]:
|
||||
with path.open(encoding="utf-8") as source:
|
||||
value = json.load(source)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError(f"expected JSON object: {path}")
|
||||
return value
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def config_from_result(result: dict[str, Any]) -> dict[str, int]:
|
||||
flags = result["config_patch"]["flag_patch"]
|
||||
tp = int(flags["tensor-parallel-size"])
|
||||
dp = int(flags.get("data-parallel-size", 1))
|
||||
return {
|
||||
"tp": tp,
|
||||
"dp": dp,
|
||||
"ep": int(flags.get("expert-parallel-size", 1)),
|
||||
"mns": int(flags["max-num-seqs"]),
|
||||
"mbt": int(flags["max-num-batched-tokens"]),
|
||||
"gpu_count": tp * dp,
|
||||
}
|
||||
|
||||
|
||||
def cell_id(config: dict[str, int]) -> str:
|
||||
topology = f"tp{config['tp']}"
|
||||
if config["dp"] != 1 or config["ep"] != 1:
|
||||
topology += f"_dp{config['dp']}_ep{config['ep']}"
|
||||
return f"{topology}_mns{config['mns']}_mbt{config['mbt']}"
|
||||
|
||||
|
||||
def trial_record(case: str, path: Path) -> dict[str, Any]:
|
||||
result = load_json(path)
|
||||
config = config_from_result(result)
|
||||
score = float(result["best_request_rate"]) / config["gpu_count"]
|
||||
probes = result.get("probes", [])
|
||||
infeasible_above = [
|
||||
float(probe.get("payload", probe)["request_rate"]) / config["gpu_count"]
|
||||
for probe in probes
|
||||
if not probe["feasible"]
|
||||
and float(probe.get("payload", probe)["request_rate"])
|
||||
/ config["gpu_count"]
|
||||
> score
|
||||
]
|
||||
upper_bound = min(infeasible_above) if infeasible_above else None
|
||||
probe_count = len(result.get("probes", []))
|
||||
primary_result = result.get("best_source") == "primary_search"
|
||||
no_probe_failure = not bool(result.get("completed_with_probe_failure", False))
|
||||
fully_valid = (
|
||||
result.get("status") == "completed"
|
||||
and probe_count == EXPECTED_PROBES
|
||||
and primary_result
|
||||
and no_probe_failure
|
||||
)
|
||||
return {
|
||||
"case": case,
|
||||
"cell_id": cell_id(config),
|
||||
**config,
|
||||
"score_req_s_per_gpu": score,
|
||||
"capacity_lower_bound_req_s_per_gpu": score,
|
||||
"capacity_upper_bound_req_s_per_gpu": upper_bound,
|
||||
"capacity_bracket_width_req_s_per_gpu": (
|
||||
upper_bound - score if upper_bound is not None else None
|
||||
),
|
||||
"best_request_rate_req_s": float(result["best_request_rate"]),
|
||||
"best_sampling_u": float(result["best_sampling_u"]),
|
||||
"best_pass_rate": float(result["best_pass_rate"]),
|
||||
"probe_count": probe_count,
|
||||
"best_source": result.get("best_source"),
|
||||
"completed_with_probe_failure": bool(
|
||||
result.get("completed_with_probe_failure", False)
|
||||
),
|
||||
"fully_valid": fully_valid,
|
||||
"result_path": str(path),
|
||||
"result_sha256": sha256_file(path),
|
||||
}
|
||||
|
||||
|
||||
def summarize_case(case: str, rows: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
if not rows:
|
||||
raise ValueError(f"no rows for case: {case}")
|
||||
ids = [row["cell_id"] for row in rows]
|
||||
if len(ids) != len(set(ids)):
|
||||
duplicates = sorted(cell for cell in set(ids) if ids.count(cell) > 1)
|
||||
raise ValueError(f"duplicate cells for {case}: {duplicates}")
|
||||
|
||||
best = max(row["score_req_s_per_gpu"] for row in rows)
|
||||
tolerance = max(1e-12, best * 1e-9)
|
||||
top = [
|
||||
row["cell_id"]
|
||||
for row in rows
|
||||
if math.isclose(row["score_req_s_per_gpu"], best, abs_tol=tolerance)
|
||||
]
|
||||
distinct_scores = []
|
||||
for score in sorted({row["score_req_s_per_gpu"] for row in rows}, reverse=True):
|
||||
if not any(math.isclose(score, seen, abs_tol=tolerance) for seen in distinct_scores):
|
||||
distinct_scores.append(score)
|
||||
|
||||
max_lower_bound = max(row["capacity_lower_bound_req_s_per_gpu"] for row in rows)
|
||||
possibly_optimal = [
|
||||
row["cell_id"]
|
||||
for row in rows
|
||||
if row["capacity_upper_bound_req_s_per_gpu"] is None
|
||||
or row["capacity_upper_bound_req_s_per_gpu"] + tolerance >= max_lower_bound
|
||||
]
|
||||
|
||||
total_pairs = len(rows) * (len(rows) - 1) // 2
|
||||
tied_pairs = sum(
|
||||
1
|
||||
for left_index, left in enumerate(rows)
|
||||
for right in rows[left_index + 1 :]
|
||||
if math.isclose(
|
||||
left["score_req_s_per_gpu"],
|
||||
right["score_req_s_per_gpu"],
|
||||
abs_tol=tolerance,
|
||||
)
|
||||
)
|
||||
return {
|
||||
"case": case,
|
||||
"cell_count": len(rows),
|
||||
"fully_valid_cell_count": sum(bool(row["fully_valid"]) for row in rows),
|
||||
"invalid_cells": [row["cell_id"] for row in rows if not row["fully_valid"]],
|
||||
"best_score_req_s_per_gpu": best,
|
||||
"top_set": sorted(top),
|
||||
"top_set_size": len(top),
|
||||
"random_top_set_hit_rate": len(top) / len(rows),
|
||||
"distinct_score_count": len(distinct_scores),
|
||||
"distinct_scores_req_s_per_gpu": distinct_scores,
|
||||
"possibly_optimal_set_from_search_brackets": sorted(possibly_optimal),
|
||||
"possibly_optimal_set_size": len(possibly_optimal),
|
||||
"pair_count": total_pairs,
|
||||
"tied_pair_count": tied_pairs,
|
||||
"informative_pair_count": total_pairs - tied_pairs,
|
||||
"informative_pair_fraction": (
|
||||
(total_pairs - tied_pairs) / total_pairs if total_pairs else 0.0
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def markdown_report(metrics: dict[str, Any], rows: list[dict[str, Any]]) -> str:
|
||||
lines = [
|
||||
"# Qwen235B ground-truth audit",
|
||||
"",
|
||||
"Objective: maximum SLO-feasible offered request throughput per GPU.",
|
||||
"This report contains real-machine data only; it makes no Frontier match claim.",
|
||||
"",
|
||||
"| case | valid cells | score levels | top-set size | random top-set hit | informative pairs |",
|
||||
"|---|---:|---:|---:|---:|---:|",
|
||||
]
|
||||
for case in metrics["cases"]:
|
||||
lines.append(
|
||||
f"| {case['case']} | {case['fully_valid_cell_count']}/{case['cell_count']} "
|
||||
f"| {case['distinct_score_count']} | {case['top_set_size']}/{case['cell_count']} "
|
||||
f"| {case['random_top_set_hit_rate']:.1%} "
|
||||
f"| {case['informative_pair_count']}/{case['pair_count']} "
|
||||
f"({case['informative_pair_fraction']:.1%}) |"
|
||||
)
|
||||
|
||||
lines.extend(["", "## Cells", ""])
|
||||
for case in metrics["cases"]:
|
||||
lines.extend(
|
||||
[
|
||||
f"### {case['case']}",
|
||||
"",
|
||||
"| cell | capacity bracket (req/s/GPU) | valid | probes | source |",
|
||||
"|---|---:|---:|---:|---|",
|
||||
]
|
||||
)
|
||||
for row in sorted(
|
||||
(row for row in rows if row["case"] == case["case"]),
|
||||
key=lambda row: row["cell_id"],
|
||||
):
|
||||
upper = row["capacity_upper_bound_req_s_per_gpu"]
|
||||
bracket = (
|
||||
f"[{row['capacity_lower_bound_req_s_per_gpu']:.9f}, "
|
||||
f"{upper:.9f})"
|
||||
if upper is not None
|
||||
else f"[{row['capacity_lower_bound_req_s_per_gpu']:.9f}, +inf)"
|
||||
)
|
||||
lines.append(
|
||||
f"| {row['cell_id']} | {bracket} "
|
||||
f"| {'yes' if row['fully_valid'] else 'no'} | {row['probe_count']} "
|
||||
f"| {row['best_source']} |"
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
f"Top set: `{', '.join(case['top_set'])}`.",
|
||||
f"Possibly optimal under binary-search brackets: "
|
||||
f"`{', '.join(case['possibly_optimal_set_from_search_brackets'])}`.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
|
||||
lines.extend(
|
||||
[
|
||||
"## Interpretation guardrail",
|
||||
"",
|
||||
"A Frontier top-set hit is insufficient by itself because the surfaces contain "
|
||||
"large ties. The later comparison must report selected-config regret and "
|
||||
"tie-aware pairwise ranking, and must keep invalid real cells visible.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def write_csv(path: Path, rows: list[dict[str, Any]]) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8", newline="") as target:
|
||||
writer = csv.DictWriter(target, fieldnames=list(rows[0]))
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--prefill-root", action="append", type=Path, required=True)
|
||||
parser.add_argument("--decode-root", action="append", type=Path, required=True)
|
||||
parser.add_argument("--output-dir", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
cases = {"qwen235b_prefill_only": args.prefill_root, "qwen235b_decode_only": args.decode_root}
|
||||
rows = []
|
||||
for case, roots in cases.items():
|
||||
for root in roots:
|
||||
paths = sorted(root.glob("store/*/trials/trial-*/result.json"))
|
||||
if not paths:
|
||||
raise ValueError(f"no result files below {root}")
|
||||
rows.extend(trial_record(case, path) for path in paths)
|
||||
|
||||
summaries = [
|
||||
summarize_case(case, [row for row in rows if row["case"] == case])
|
||||
for case in cases
|
||||
]
|
||||
metrics = {"schema": SCHEMA, "cases": summaries}
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
write_csv(args.output_dir / "cells.csv", rows)
|
||||
(args.output_dir / "metrics.json").write_text(
|
||||
json.dumps(metrics, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
(args.output_dir / "report.md").write_text(
|
||||
markdown_report(metrics, rows), encoding="utf-8"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
110
runs/frontier-multicase-sufficiency-v0/audit_qwen30_baseline.py
Normal file
110
runs/frontier-multicase-sufficiency-v0/audit_qwen30_baseline.py
Normal file
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract the predictive-versus-calibrated Frontier Qwen30B baseline."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODES = ("uncalibrated/SLO-gated", "frozen-calibrated/SLO-gated")
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict[str, Any]:
|
||||
with path.open(encoding="utf-8") as source:
|
||||
value = json.load(source)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError(f"expected JSON object: {path}")
|
||||
return value
|
||||
|
||||
|
||||
def extract(metrics: dict[str, Any], protocol: dict[str, Any]) -> dict[str, Any]:
|
||||
analyses = metrics["analyses"]
|
||||
rows = []
|
||||
for mode in MODES:
|
||||
analysis = analyses[mode]
|
||||
values = analysis["metrics"]
|
||||
top1 = values["top1"]
|
||||
confusion = analysis["false_feasibility"]["overall"]
|
||||
rows.append(
|
||||
{
|
||||
"mode": mode,
|
||||
"selected_cells": top1["candidate_cells"],
|
||||
"optimistic_real_regret": top1["optimistic_regret"],
|
||||
"worst_case_real_regret": top1["worst_case_regret"],
|
||||
"kendall_tau_b": values["kendall_tau_b"]["tau_b"],
|
||||
"pairwise_exact_sign_accuracy": values["pairwise_direction"][
|
||||
"exact_sign_accuracy"
|
||||
],
|
||||
"false_feasible": confusion["false_feasible"],
|
||||
"false_infeasible": confusion["false_infeasible"],
|
||||
"agreement": confusion["agreement"],
|
||||
}
|
||||
)
|
||||
return {
|
||||
"schema": "frontier-qwen30-calibration-audit-v0",
|
||||
"rows": rows,
|
||||
"calibration": {
|
||||
"fitted_a_tp": protocol["fitted_a_tp"],
|
||||
"fit_fixture": protocol["fit_fixture"],
|
||||
"holdout_fixture": protocol["holdout_fixture"],
|
||||
"loss": protocol["loss"],
|
||||
"refit_on_holdout": protocol["refit_on_holdout"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def report(result: dict[str, Any]) -> str:
|
||||
lines = [
|
||||
"# Qwen30B Frontier baseline audit",
|
||||
"",
|
||||
"| mode | selected cells | worst real regret | Kendall tau-b | pair sign accuracy | feasibility (agree/FP/FN) |",
|
||||
"|---|---|---:|---:|---:|---:|",
|
||||
]
|
||||
for row in result["rows"]:
|
||||
lines.append(
|
||||
f"| {row['mode']} | {', '.join(row['selected_cells'])} "
|
||||
f"| {row['worst_case_real_regret']:.2%} | {row['kendall_tau_b']:.4f} "
|
||||
f"| {row['pairwise_exact_sign_accuracy']:.2%} "
|
||||
f"| {row['agreement']}/{row['false_feasible']}/{row['false_infeasible']} |"
|
||||
)
|
||||
calibration = result["calibration"]
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"The calibrated mode applies a distinct end-to-end execution-time scale per TP: "
|
||||
+ ", ".join(
|
||||
f"TP{tp}={value:.6f}"
|
||||
for tp, value in sorted(calibration["fitted_a_tp"].items())
|
||||
)
|
||||
+ ".",
|
||||
"",
|
||||
f"Those scales were fitted against real total throughput on "
|
||||
f"`{calibration['fit_fixture']}` and checked without refitting on "
|
||||
f"`{calibration['holdout_fixture']}`. This validates within-workload transfer of "
|
||||
"the calibration, not zero-shot Frontier prediction across TP.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--metrics", type=Path, required=True)
|
||||
parser.add_argument("--calibration-protocol", type=Path, required=True)
|
||||
parser.add_argument("--output-dir", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
result = extract(load_json(args.metrics), load_json(args.calibration_protocol))
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
(args.output_dir / "metrics.json").write_text(
|
||||
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
(args.output_dir / "report.md").write_text(report(result), encoding="utf-8")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,166 @@
|
||||
# Community vLLM Qwen235B versus Frontier protocol v0
|
||||
|
||||
## Hypothesis
|
||||
|
||||
I believe that collecting compute and communication profiles from the same
|
||||
community vLLM execution stack used for serving removes the current
|
||||
internal-runtime confounder. I will verify this by freezing Frontier's
|
||||
profile-only predictions before collecting the corresponding serving response
|
||||
surface, then measuring config-selection regret and rank agreement.
|
||||
|
||||
This experiment tests Frontier inside a declared compatibility envelope. It
|
||||
does not claim fidelity for the previous internal vLLM, EAGLE3, DeepEP, or
|
||||
external-KV setup.
|
||||
|
||||
## Frozen system boundary
|
||||
|
||||
- Host: `dash0`, 8 NVIDIA H20 GPUs, driver `580.95.05`.
|
||||
- Model: `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8`.
|
||||
- Model config SHA256:
|
||||
`702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e`.
|
||||
- Model index SHA256:
|
||||
`e03d4abec9611fca05844d2c8b5a08318ddc3c86acfb9481a0ef5692bf76e6d6`.
|
||||
- Frontier: NetX-lab/Frontier commit
|
||||
`d9cfeb6d8791fbf2f295dd9744c56a666171776e`.
|
||||
- vLLM target: exact community release `0.10.2`, installed in a new isolated
|
||||
environment with wheel/source hash recorded. Frontier declares
|
||||
`vllm>=0.10,<0.11`; the model README declares `vllm>=0.8.5`.
|
||||
- Transformers is pinned to `4.55.2` and tokenizers resolves to `0.21.4`.
|
||||
vLLM 0.10.2 only declares a lower bound on Transformers, while current
|
||||
Transformers 5.x removes a tokenizer API used by this vLLM release.
|
||||
- Do not use the shared `/usr/local` vLLM. Its imported version and package
|
||||
metadata disagree, so it is not a reproducible community baseline.
|
||||
|
||||
The profiler and serving process must import the same vLLM installation. The
|
||||
following execution choices are fixed for the first validation pass:
|
||||
|
||||
- FlashInfer attention in both profiler and serving;
|
||||
- eager execution, with CUDA graphs disabled;
|
||||
- no speculative decoding;
|
||||
- no external KV connector;
|
||||
- no prefix reuse;
|
||||
- chunked prefill enabled;
|
||||
- BF16 KV cache (`auto` for this model), while the checkpoint's block-wise FP8
|
||||
weight quantization remains enabled;
|
||||
- community vLLM default MoE implementation unless a backend is explicitly
|
||||
frozen and supported on both sides.
|
||||
|
||||
The SM90 path cannot use `CutlassBlockScaledGroupedGemm` (the vLLM 0.10.2
|
||||
implementation gates that path on SM100), so this system uses Triton block-FP8
|
||||
MoE. However, the two sides do not yet select the same Triton tuning config.
|
||||
Serving resolves the checked-in H20 config named with
|
||||
`dtype=fp8_w8a8`, while Frontier's standalone MoE profiler requests a filename
|
||||
without that FP8 dtype component and falls back to a default config. Formal
|
||||
comparison is gated on repairing this profiler/runtime tuning-config mismatch
|
||||
and re-profiling; recording the same vLLM package version is not sufficient.
|
||||
|
||||
These controls isolate operator composition and scheduling. CUDA graphs,
|
||||
FlashAttention, speculative decoding, external KV, and optimized expert
|
||||
communication become separate stress cases after the controlled pass.
|
||||
|
||||
## Case P: prefill-only
|
||||
|
||||
Reuse the original trace window and SLO because the community model's 40,960
|
||||
token limit covers the filtered input plus the one-token completion.
|
||||
|
||||
- Window: `thinking_w20260327_1000`.
|
||||
- Input filter: 0--32,768 tokens.
|
||||
- Output override: 1 token.
|
||||
- Replay scale: 1.0.
|
||||
- SLO pass rate: at least 0.95.
|
||||
- TTFT SLO: 1 s for input <=8,191; 2 s otherwise.
|
||||
- Objective: maximum SLO-feasible offered requests/s/GPU.
|
||||
- Candidate grid: TP `{4,8}` x MNS `{64,128}` x MBT `{8192,16384}`.
|
||||
- DP=1 and expert parallel disabled.
|
||||
|
||||
This case can test the TP4-versus-TP8 decision and batching effects. It must not
|
||||
reuse performance values from the internal 256k model as ground truth.
|
||||
|
||||
## Case D: decode-dominant
|
||||
|
||||
Do not initially reproduce the previous strict decode-only case. It depends on
|
||||
`DecodeBenchConnector`, EAGLE3, FP8 KV, DeepEP/NVSHMEM, and decode CUDA graphs,
|
||||
which are outside the controlled Frontier profile contract.
|
||||
|
||||
Construct a community-only decode-dominant case that both systems can express:
|
||||
|
||||
- same trace window and timestamp/sampling fields;
|
||||
- input filter: 1--512 tokens;
|
||||
- output override: 512 tokens with EOS ignored;
|
||||
- SLO pass rate: at least 0.95;
|
||||
- TPOT SLO: 40 ms;
|
||||
- objective: maximum SLO-feasible offered requests/s/GPU;
|
||||
- topology grid: `(TP=4, DP=2, EP=8)` and `(TP=2, DP=4, EP=8)`;
|
||||
- batching grid: MNS `{64,128}` x MBT `{256,384}`.
|
||||
|
||||
This is deliberately named decode-dominant, not decode-only. A strict
|
||||
decode-only claim requires an initial-KV state contract in Frontier.
|
||||
|
||||
## Required profile closure
|
||||
|
||||
Profile data are measurement inputs, not end-to-end calibration. No serving
|
||||
throughput or latency from either case may scale the profiles.
|
||||
|
||||
1. Linear/operator profiles for the TP degrees consumed by the two grids.
|
||||
2. FlashInfer attention profiles covering the observed prefill/decode batch,
|
||||
context-length, and chunk-size ranges.
|
||||
3. FP8 MoE profiles for the actually consumed parallel pairs:
|
||||
`(MoE TP=4, EP=1)`, `(MoE TP=8, EP=1)`, and `(MoE TP=1, EP=8)`.
|
||||
4. H20 intra-node collective profiles for TP all-reduce at world sizes 2, 4,
|
||||
and 8 and the EP8 all-to-all path.
|
||||
|
||||
Frontier already provides an H20 device description but no checked-in H20
|
||||
network profiles. Its public collective profiler covers all-reduce and
|
||||
send/recv, not all-to-all. Therefore Case P may proceed after H20 all-reduce
|
||||
closure; Case D remains blocked until EP8 all-to-all is either measured and
|
||||
consumed or the selected communication model is independently validated
|
||||
against those measurements.
|
||||
|
||||
## Blind run order
|
||||
|
||||
1. Build the isolated environment and record package/binary hashes.
|
||||
2. Run one-row compute and collective smokes, then one TP4 server-load/request
|
||||
smoke. A smoke failure stops the campaign.
|
||||
3. Collect profiles and validate CSV metadata/coverage.
|
||||
4. Run Frontier for every candidate and offered-load anchor.
|
||||
5. Freeze simulator outputs and their SHA256 checksums.
|
||||
6. Only then collect community-vLLM serving ground truth. Randomize the first
|
||||
trial order and reverse it for the second trial.
|
||||
7. Refine only decision-relevant capacity intervals that still overlap.
|
||||
|
||||
## Metrics and decision rule
|
||||
|
||||
Report per case:
|
||||
|
||||
- absolute simulated and real TTFT/TPOT/throughput values;
|
||||
- anchor-level SLO feasibility confusion;
|
||||
- selected-config real regret, including interval-robust regret;
|
||||
- Kendall tau-b with ties preserved;
|
||||
- informative-pair direction accuracy;
|
||||
- top-set hit and random top-set hit probability;
|
||||
- profile and real-GPU measurement cost.
|
||||
|
||||
Frontier is sufficient as a config ranker for this controlled family only if
|
||||
every completed case has:
|
||||
|
||||
- worst selected-config real regret <=5%;
|
||||
- Kendall tau-b >=0.8 on enough informative pairs;
|
||||
- no unresolved ground-truth interval capable of reversing the decision;
|
||||
- no per-case or per-action end-to-end calibration.
|
||||
|
||||
If the controlled pass succeeds, enable one omitted mechanism at a time. If it
|
||||
fails, use operator/communication/stage residuals to localize which composition
|
||||
assumption reverses the ranking before proposing a new tuner mechanism.
|
||||
|
||||
## Launch gates and initial cost cap
|
||||
|
||||
The first authorized GPU action should be smoke-only:
|
||||
|
||||
- isolated vLLM import and Qwen235B TP4 load;
|
||||
- one request through the community server;
|
||||
- one representative FP8 linear, attention, MoE, and all-reduce profile point;
|
||||
- expected wall time: 20--40 minutes;
|
||||
- hard GPU budget: 2 H20-GPU-hours.
|
||||
|
||||
No full response-surface sweep is authorized by this protocol. Its cost and
|
||||
anchor count must be resolved from the smoke timings and echoed separately.
|
||||
114
runs/frontier-multicase-sufficiency-v0/findings.md
Normal file
114
runs/frontier-multicase-sufficiency-v0/findings.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Frontier multi-case sufficiency: current finding
|
||||
|
||||
## Bottom line
|
||||
|
||||
The existing evidence does **not** show that Frontier already solves config
|
||||
selection. The strongest Qwen30B match is an action-conditioned calibrated
|
||||
result, while the zero-shot/profile-only result selects the wrong TP family.
|
||||
The Qwen235B surfaces are useful follow-up cases, but decode ground truth and
|
||||
Frontier's execution semantics must be repaired before a match claim is valid.
|
||||
|
||||
## Qwen30B: calibration is decision-bearing
|
||||
|
||||
Under the aligned maximum-SLO-feasible-throughput objective:
|
||||
|
||||
| reading | Frontier selection | worst real regret | Kendall tau-b |
|
||||
|---|---|---:|---:|
|
||||
| profile-only | TP4/MNS32 or TP4/MNS64 | 25.63% | 0.0000 |
|
||||
| per-TP calibrated | TP2/MNS32 or TP2/MNS64 | 0.76% | 0.9668 |
|
||||
|
||||
The calibrated run multiplies all simulated execution times by a separately
|
||||
fitted factor for each TP: TP1=0.723481, TP2=0.468089, TP4=0.352137. These
|
||||
factors were fitted to real end-to-end throughput on the same model/workload
|
||||
family. The large, TP-dependent residual is therefore evidence that the
|
||||
profile-only simulator misses action-dependent execution behavior; it is not
|
||||
evidence that the unmodified simulator predicts the action correctly.
|
||||
|
||||
## Qwen235B real response surfaces
|
||||
|
||||
### Prefill-only
|
||||
|
||||
- 8/8 cells are complete primary searches.
|
||||
- The point-estimate top set contains all four TP8 cells, so a random cell has a
|
||||
50% top-set hit probability.
|
||||
- Capacity brackets still separate all TP8 cells from all TP4 cells. Thus this
|
||||
case can test the **TP4 versus TP8** decision, but the current six probes do
|
||||
not distinguish MNS/MBT within TP8.
|
||||
|
||||
### Decode-only
|
||||
|
||||
- Only 7/8 cells are fully valid; `TP2/DP4/EP8, MNS128, MBT384` is marked
|
||||
`partial_probe_before_failure` after an engine restart/port failure.
|
||||
- There are only two point-estimate score levels, and the top set contains 5/8
|
||||
cells (62.5% random hit probability).
|
||||
- More importantly, all eight binary-search capacity brackets overlap the best
|
||||
observed lower bound. The current data cannot rule out any cell as optimal.
|
||||
|
||||
Consequently, a decode top-set hit on these results is not a match. After the
|
||||
simulator selects a cell, the cheapest rigorous next step is to refine only
|
||||
that cell and one competing topology/batching cell until their capacity
|
||||
intervals separate or remain statistically indistinguishable.
|
||||
|
||||
## Frontier semantic coverage for Qwen235B
|
||||
|
||||
The topology itself is expressible: Frontier exposes attention TP/DP and MoE
|
||||
TP/EP separately, so TP4/DP2/EP8 and TP2/DP4/EP8 need not be decomposed into
|
||||
independent scheduling and execution problems.
|
||||
|
||||
The current execution model is not yet aligned, however:
|
||||
|
||||
- no checked-in H20 Qwen235B `linear_op.csv`, `attention.csv`, or `moe.csv`;
|
||||
- checked-in Qwen235B config is BF16 with max position 40960, whereas the real
|
||||
run uses FP8 weights, FP8 KV, and max model length 262144;
|
||||
- real prefill uses FlashAttention and internal BLADNN kernels, while Frontier's
|
||||
attention profiler exposes only FlashInfer and NO-OP backends;
|
||||
- real decode starts with a dummy-filled external KV cache through
|
||||
`DecodeBenchConnector`; Frontier's trace request generator requires positive
|
||||
prefill tokens and has no equivalent initial-KV trace contract;
|
||||
- real decode combines DeepEP/NVSHMEM, EAGLE3, and FULL_DECODE_ONLY CUDA graphs;
|
||||
Frontier explicitly treats speculative decode plus decode CUDA graphs as a
|
||||
conflicting/diagnostic combination.
|
||||
|
||||
Running dummy profiles or silently substituting FlashInfer/eager decode would
|
||||
produce a number, but it would not test whether Frontier matches this system.
|
||||
|
||||
## Community-vLLM Qwen235B smoke
|
||||
|
||||
The controlled community stack is feasible: vLLM 0.10.2 loaded the
|
||||
Qwen3-235B-A22B-FP8 checkpoint on TP4, allocated a BF16 KV cache, and completed
|
||||
a real request. Representative FP8 linear/MoE, FlashInfer attention, and TP4
|
||||
NCCL paths also executed successfully.
|
||||
|
||||
The smoke nevertheless found a profiler/runtime mismatch before any scheduler
|
||||
model was involved. Frontier's MoE wrapper calls `get_config_dtype_str` without
|
||||
`use_fp8_w8a8=True`, so it misses vLLM's tuned H20 block-FP8 Triton config and
|
||||
uses a default. At TP4/EP1 with 16 tokens, a paired five-routing-seed factorial
|
||||
measured:
|
||||
|
||||
| variant | grouped-GEMM mean | paired delta vs original |
|
||||
|---|---:|---:|
|
||||
| original default config + FP16 compute type | 0.3100 ms | 0.00% |
|
||||
| FP8 config key only | 0.2508 ms | -19.12% |
|
||||
| BF16 compute type only | 0.3088 ms | -0.40% |
|
||||
| both aligned | 0.2512 ms | -18.99% |
|
||||
|
||||
The config-key-only routing-seed 95% interval is [-21.26%, -16.98%]; the
|
||||
compute-type-only interval crosses zero. This localizes the dominant error at
|
||||
this point to kernel tuning-config selection. The original MoE CSV must not be
|
||||
used for a formal Frontier ranking until this path and the related shuffling
|
||||
block-size lookup are aligned and re-profiled. Full evidence and hashes are in
|
||||
`results/community-qwen235b-smoke/`.
|
||||
|
||||
## Research implication
|
||||
|
||||
The most interesting observation is already visible: Frontier's error is not a
|
||||
single global time bias. It changes strongly with the execution action (TP in
|
||||
Qwen30B), and a per-action residual can reverse the selected config. Qwen235B
|
||||
prefill and decode offer held-out tests of whether those residuals are explained
|
||||
by measurable execution-state features (kernel family, communication mode,
|
||||
graph mode, speculative width, and KV initial state) or require case-specific
|
||||
end-to-end fitting.
|
||||
|
||||
That is a systems question: **which execution-state transitions make operator
|
||||
profile composition non-invariant across configurations, and what is the
|
||||
minimum real evidence needed to recover the counterfactual ordering?**
|
||||
79
runs/frontier-multicase-sufficiency-v0/protocol.md
Normal file
79
runs/frontier-multicase-sufficiency-v0/protocol.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Frontier multi-case sufficiency protocol v0
|
||||
|
||||
## Claim under test
|
||||
|
||||
Can Frontier select a low-regret configuration for the frozen candidate grid
|
||||
under the objective **maximum SLO-feasible offered request throughput per GPU**?
|
||||
|
||||
The allowed conclusion is scoped to the tested model/runtime/hardware/workload
|
||||
family. Three matches cannot establish universal simulator fidelity.
|
||||
|
||||
## Cases
|
||||
|
||||
1. Qwen3-30B-A3B mixed chat, H20, TP x MNS grid (existing data).
|
||||
2. Qwen3-235B-A22B prefill-only, H20, TP x MNS x MBT grid.
|
||||
3. Qwen3-235B-A22B decode-only, H20, TP/DP/EP x MNS x MBT grid.
|
||||
|
||||
For Qwen235B, keep the real trace window, request filtering, SLO, model build,
|
||||
runtime flags, and candidate cells recorded by the original trials. Frontier is
|
||||
evaluated at the same offered-load anchors; no capacity extrapolation beyond
|
||||
the common anchors is allowed.
|
||||
|
||||
## Two separate simulator readings
|
||||
|
||||
- **Zero-shot/profile-only:** operator profiles and documented hardware/model
|
||||
inputs are allowed; no end-to-end measurement from the evaluated workload is
|
||||
used to scale Frontier.
|
||||
- **Calibrated:** every real measurement used to fit a scalar or residual is
|
||||
charged and reported. Calibration is fitted on a declared train fixture and
|
||||
evaluated on a disjoint workload/config holdout.
|
||||
|
||||
These readings must never be merged. A per-TP scale fitted from the same
|
||||
workload does not count as zero-shot simulator accuracy.
|
||||
|
||||
## Ground-truth gate
|
||||
|
||||
Before a match decision:
|
||||
|
||||
- every result must be a completed primary search without probe failure;
|
||||
- capacity is an interval from the largest feasible anchor to the next
|
||||
infeasible anchor, not merely the feasible lower bound;
|
||||
- refine probes when the possibly-optimal set induced by those intervals is too
|
||||
broad to distinguish the simulator's selected cell;
|
||||
- record the random top-set hit rate and the number of informative (non-tied)
|
||||
pairs.
|
||||
|
||||
## Primary metrics and predeclared decision rule
|
||||
|
||||
Per case, report:
|
||||
|
||||
1. selected-config real regret (point estimate and interval-robust bound);
|
||||
2. Kendall tau-b and exact pair-direction accuracy with real ties preserved;
|
||||
3. top-set hit and its random-hit baseline;
|
||||
4. anchor-level SLO feasibility confusion;
|
||||
5. profile/calibration cost and all right-censored cells.
|
||||
|
||||
The profile-only simulator is considered sufficient as a **config ranker for
|
||||
the tested family** only if every case has:
|
||||
|
||||
- worst selected-config real regret <= 5%;
|
||||
- Kendall tau-b >= 0.8 on a response surface with enough informative pairs;
|
||||
- no unresolved ground-truth interval that can reverse the selected decision;
|
||||
- no per-case or per-action end-to-end calibration.
|
||||
|
||||
SLO-oracle sufficiency is a stronger claim and additionally requires low false
|
||||
feasible/false infeasible rates; ranker success alone does not establish it.
|
||||
|
||||
## Current representational gaps to disclose
|
||||
|
||||
- The checked-in Frontier Qwen235B model config is BF16 and has no H20
|
||||
Qwen235B operator profiles, while the real runtime uses FP8 weights and FP8 KV.
|
||||
- Prefill uses internal vLLM/BLADNN paths and PIECEWISE CUDA graphs.
|
||||
- Decode uses TP/DP with EP8, DeepEP/NVSHMEM, EAGLE3, DecodeBenchConnector, and
|
||||
FULL_DECODE_ONLY CUDA graphs.
|
||||
- Frontier currently rejects the faithful combination of speculative decoding
|
||||
and decode CUDA-graph modeling unless using a diagnostic opt-in; its public
|
||||
examples disable decode CUDA graphs for speculative decoding.
|
||||
|
||||
Therefore an as-is Frontier run and an upgraded/semantically aligned run, if
|
||||
implemented, must be reported separately.
|
||||
@@ -0,0 +1,333 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3MoeForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"decoder_sparse_step": 1,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 12288,
|
||||
"max_position_embeddings": 40960,
|
||||
"max_window_layers": 94,
|
||||
"mlp_only_layers": [],
|
||||
"model_type": "qwen3_moe",
|
||||
"moe_intermediate_size": 1536,
|
||||
"norm_topk_prob": true,
|
||||
"num_attention_heads": 64,
|
||||
"num_experts": 128,
|
||||
"num_experts_per_tok": 8,
|
||||
"num_hidden_layers": 94,
|
||||
"num_key_value_heads": 4,
|
||||
"output_router_logits": false,
|
||||
"rms_norm_eps": 0.000001,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000.0,
|
||||
"router_aux_loss_coef": 0.001,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.0",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936,
|
||||
"quantization_config": {
|
||||
"activation_scheme": "dynamic",
|
||||
"modules_to_not_convert": [
|
||||
"lm_head",
|
||||
"model.layers.0.input_layernorm",
|
||||
"model.layers.0.mlp.gate",
|
||||
"model.layers.0.post_attention_layernorm",
|
||||
"model.layers.1.input_layernorm",
|
||||
"model.layers.1.mlp.gate",
|
||||
"model.layers.1.post_attention_layernorm",
|
||||
"model.layers.2.input_layernorm",
|
||||
"model.layers.2.mlp.gate",
|
||||
"model.layers.2.post_attention_layernorm",
|
||||
"model.layers.3.input_layernorm",
|
||||
"model.layers.3.mlp.gate",
|
||||
"model.layers.3.post_attention_layernorm",
|
||||
"model.layers.4.input_layernorm",
|
||||
"model.layers.4.mlp.gate",
|
||||
"model.layers.4.post_attention_layernorm",
|
||||
"model.layers.5.input_layernorm",
|
||||
"model.layers.5.mlp.gate",
|
||||
"model.layers.5.post_attention_layernorm",
|
||||
"model.layers.6.input_layernorm",
|
||||
"model.layers.6.mlp.gate",
|
||||
"model.layers.6.post_attention_layernorm",
|
||||
"model.layers.7.input_layernorm",
|
||||
"model.layers.7.mlp.gate",
|
||||
"model.layers.7.post_attention_layernorm",
|
||||
"model.layers.8.input_layernorm",
|
||||
"model.layers.8.mlp.gate",
|
||||
"model.layers.8.post_attention_layernorm",
|
||||
"model.layers.9.input_layernorm",
|
||||
"model.layers.9.mlp.gate",
|
||||
"model.layers.9.post_attention_layernorm",
|
||||
"model.layers.10.input_layernorm",
|
||||
"model.layers.10.mlp.gate",
|
||||
"model.layers.10.post_attention_layernorm",
|
||||
"model.layers.11.input_layernorm",
|
||||
"model.layers.11.mlp.gate",
|
||||
"model.layers.11.post_attention_layernorm",
|
||||
"model.layers.12.input_layernorm",
|
||||
"model.layers.12.mlp.gate",
|
||||
"model.layers.12.post_attention_layernorm",
|
||||
"model.layers.13.input_layernorm",
|
||||
"model.layers.13.mlp.gate",
|
||||
"model.layers.13.post_attention_layernorm",
|
||||
"model.layers.14.input_layernorm",
|
||||
"model.layers.14.mlp.gate",
|
||||
"model.layers.14.post_attention_layernorm",
|
||||
"model.layers.15.input_layernorm",
|
||||
"model.layers.15.mlp.gate",
|
||||
"model.layers.15.post_attention_layernorm",
|
||||
"model.layers.16.input_layernorm",
|
||||
"model.layers.16.mlp.gate",
|
||||
"model.layers.16.post_attention_layernorm",
|
||||
"model.layers.17.input_layernorm",
|
||||
"model.layers.17.mlp.gate",
|
||||
"model.layers.17.post_attention_layernorm",
|
||||
"model.layers.18.input_layernorm",
|
||||
"model.layers.18.mlp.gate",
|
||||
"model.layers.18.post_attention_layernorm",
|
||||
"model.layers.19.input_layernorm",
|
||||
"model.layers.19.mlp.gate",
|
||||
"model.layers.19.post_attention_layernorm",
|
||||
"model.layers.20.input_layernorm",
|
||||
"model.layers.20.mlp.gate",
|
||||
"model.layers.20.post_attention_layernorm",
|
||||
"model.layers.21.input_layernorm",
|
||||
"model.layers.21.mlp.gate",
|
||||
"model.layers.21.post_attention_layernorm",
|
||||
"model.layers.22.input_layernorm",
|
||||
"model.layers.22.mlp.gate",
|
||||
"model.layers.22.post_attention_layernorm",
|
||||
"model.layers.23.input_layernorm",
|
||||
"model.layers.23.mlp.gate",
|
||||
"model.layers.23.post_attention_layernorm",
|
||||
"model.layers.24.input_layernorm",
|
||||
"model.layers.24.mlp.gate",
|
||||
"model.layers.24.post_attention_layernorm",
|
||||
"model.layers.25.input_layernorm",
|
||||
"model.layers.25.mlp.gate",
|
||||
"model.layers.25.post_attention_layernorm",
|
||||
"model.layers.26.input_layernorm",
|
||||
"model.layers.26.mlp.gate",
|
||||
"model.layers.26.post_attention_layernorm",
|
||||
"model.layers.27.input_layernorm",
|
||||
"model.layers.27.mlp.gate",
|
||||
"model.layers.27.post_attention_layernorm",
|
||||
"model.layers.28.input_layernorm",
|
||||
"model.layers.28.mlp.gate",
|
||||
"model.layers.28.post_attention_layernorm",
|
||||
"model.layers.29.input_layernorm",
|
||||
"model.layers.29.mlp.gate",
|
||||
"model.layers.29.post_attention_layernorm",
|
||||
"model.layers.30.input_layernorm",
|
||||
"model.layers.30.mlp.gate",
|
||||
"model.layers.30.post_attention_layernorm",
|
||||
"model.layers.31.input_layernorm",
|
||||
"model.layers.31.mlp.gate",
|
||||
"model.layers.31.post_attention_layernorm",
|
||||
"model.layers.32.input_layernorm",
|
||||
"model.layers.32.mlp.gate",
|
||||
"model.layers.32.post_attention_layernorm",
|
||||
"model.layers.33.input_layernorm",
|
||||
"model.layers.33.mlp.gate",
|
||||
"model.layers.33.post_attention_layernorm",
|
||||
"model.layers.34.input_layernorm",
|
||||
"model.layers.34.mlp.gate",
|
||||
"model.layers.34.post_attention_layernorm",
|
||||
"model.layers.35.input_layernorm",
|
||||
"model.layers.35.mlp.gate",
|
||||
"model.layers.35.post_attention_layernorm",
|
||||
"model.layers.36.input_layernorm",
|
||||
"model.layers.36.mlp.gate",
|
||||
"model.layers.36.post_attention_layernorm",
|
||||
"model.layers.37.input_layernorm",
|
||||
"model.layers.37.mlp.gate",
|
||||
"model.layers.37.post_attention_layernorm",
|
||||
"model.layers.38.input_layernorm",
|
||||
"model.layers.38.mlp.gate",
|
||||
"model.layers.38.post_attention_layernorm",
|
||||
"model.layers.39.input_layernorm",
|
||||
"model.layers.39.mlp.gate",
|
||||
"model.layers.39.post_attention_layernorm",
|
||||
"model.layers.40.input_layernorm",
|
||||
"model.layers.40.mlp.gate",
|
||||
"model.layers.40.post_attention_layernorm",
|
||||
"model.layers.41.input_layernorm",
|
||||
"model.layers.41.mlp.gate",
|
||||
"model.layers.41.post_attention_layernorm",
|
||||
"model.layers.42.input_layernorm",
|
||||
"model.layers.42.mlp.gate",
|
||||
"model.layers.42.post_attention_layernorm",
|
||||
"model.layers.43.input_layernorm",
|
||||
"model.layers.43.mlp.gate",
|
||||
"model.layers.43.post_attention_layernorm",
|
||||
"model.layers.44.input_layernorm",
|
||||
"model.layers.44.mlp.gate",
|
||||
"model.layers.44.post_attention_layernorm",
|
||||
"model.layers.45.input_layernorm",
|
||||
"model.layers.45.mlp.gate",
|
||||
"model.layers.45.post_attention_layernorm",
|
||||
"model.layers.46.input_layernorm",
|
||||
"model.layers.46.mlp.gate",
|
||||
"model.layers.46.post_attention_layernorm",
|
||||
"model.layers.47.input_layernorm",
|
||||
"model.layers.47.mlp.gate",
|
||||
"model.layers.47.post_attention_layernorm",
|
||||
"model.layers.48.input_layernorm",
|
||||
"model.layers.48.mlp.gate",
|
||||
"model.layers.48.post_attention_layernorm",
|
||||
"model.layers.49.input_layernorm",
|
||||
"model.layers.49.mlp.gate",
|
||||
"model.layers.49.post_attention_layernorm",
|
||||
"model.layers.50.input_layernorm",
|
||||
"model.layers.50.mlp.gate",
|
||||
"model.layers.50.post_attention_layernorm",
|
||||
"model.layers.51.input_layernorm",
|
||||
"model.layers.51.mlp.gate",
|
||||
"model.layers.51.post_attention_layernorm",
|
||||
"model.layers.52.input_layernorm",
|
||||
"model.layers.52.mlp.gate",
|
||||
"model.layers.52.post_attention_layernorm",
|
||||
"model.layers.53.input_layernorm",
|
||||
"model.layers.53.mlp.gate",
|
||||
"model.layers.53.post_attention_layernorm",
|
||||
"model.layers.54.input_layernorm",
|
||||
"model.layers.54.mlp.gate",
|
||||
"model.layers.54.post_attention_layernorm",
|
||||
"model.layers.55.input_layernorm",
|
||||
"model.layers.55.mlp.gate",
|
||||
"model.layers.55.post_attention_layernorm",
|
||||
"model.layers.56.input_layernorm",
|
||||
"model.layers.56.mlp.gate",
|
||||
"model.layers.56.post_attention_layernorm",
|
||||
"model.layers.57.input_layernorm",
|
||||
"model.layers.57.mlp.gate",
|
||||
"model.layers.57.post_attention_layernorm",
|
||||
"model.layers.58.input_layernorm",
|
||||
"model.layers.58.mlp.gate",
|
||||
"model.layers.58.post_attention_layernorm",
|
||||
"model.layers.59.input_layernorm",
|
||||
"model.layers.59.mlp.gate",
|
||||
"model.layers.59.post_attention_layernorm",
|
||||
"model.layers.60.input_layernorm",
|
||||
"model.layers.60.mlp.gate",
|
||||
"model.layers.60.post_attention_layernorm",
|
||||
"model.layers.61.input_layernorm",
|
||||
"model.layers.61.mlp.gate",
|
||||
"model.layers.61.post_attention_layernorm",
|
||||
"model.layers.62.input_layernorm",
|
||||
"model.layers.62.mlp.gate",
|
||||
"model.layers.62.post_attention_layernorm",
|
||||
"model.layers.63.input_layernorm",
|
||||
"model.layers.63.mlp.gate",
|
||||
"model.layers.63.post_attention_layernorm",
|
||||
"model.layers.64.input_layernorm",
|
||||
"model.layers.64.mlp.gate",
|
||||
"model.layers.64.post_attention_layernorm",
|
||||
"model.layers.65.input_layernorm",
|
||||
"model.layers.65.mlp.gate",
|
||||
"model.layers.65.post_attention_layernorm",
|
||||
"model.layers.66.input_layernorm",
|
||||
"model.layers.66.mlp.gate",
|
||||
"model.layers.66.post_attention_layernorm",
|
||||
"model.layers.67.input_layernorm",
|
||||
"model.layers.67.mlp.gate",
|
||||
"model.layers.67.post_attention_layernorm",
|
||||
"model.layers.68.input_layernorm",
|
||||
"model.layers.68.mlp.gate",
|
||||
"model.layers.68.post_attention_layernorm",
|
||||
"model.layers.69.input_layernorm",
|
||||
"model.layers.69.mlp.gate",
|
||||
"model.layers.69.post_attention_layernorm",
|
||||
"model.layers.70.input_layernorm",
|
||||
"model.layers.70.mlp.gate",
|
||||
"model.layers.70.post_attention_layernorm",
|
||||
"model.layers.71.input_layernorm",
|
||||
"model.layers.71.mlp.gate",
|
||||
"model.layers.71.post_attention_layernorm",
|
||||
"model.layers.72.input_layernorm",
|
||||
"model.layers.72.mlp.gate",
|
||||
"model.layers.72.post_attention_layernorm",
|
||||
"model.layers.73.input_layernorm",
|
||||
"model.layers.73.mlp.gate",
|
||||
"model.layers.73.post_attention_layernorm",
|
||||
"model.layers.74.input_layernorm",
|
||||
"model.layers.74.mlp.gate",
|
||||
"model.layers.74.post_attention_layernorm",
|
||||
"model.layers.75.input_layernorm",
|
||||
"model.layers.75.mlp.gate",
|
||||
"model.layers.75.post_attention_layernorm",
|
||||
"model.layers.76.input_layernorm",
|
||||
"model.layers.76.mlp.gate",
|
||||
"model.layers.76.post_attention_layernorm",
|
||||
"model.layers.77.input_layernorm",
|
||||
"model.layers.77.mlp.gate",
|
||||
"model.layers.77.post_attention_layernorm",
|
||||
"model.layers.78.input_layernorm",
|
||||
"model.layers.78.mlp.gate",
|
||||
"model.layers.78.post_attention_layernorm",
|
||||
"model.layers.79.input_layernorm",
|
||||
"model.layers.79.mlp.gate",
|
||||
"model.layers.79.post_attention_layernorm",
|
||||
"model.layers.80.input_layernorm",
|
||||
"model.layers.80.mlp.gate",
|
||||
"model.layers.80.post_attention_layernorm",
|
||||
"model.layers.81.input_layernorm",
|
||||
"model.layers.81.mlp.gate",
|
||||
"model.layers.81.post_attention_layernorm",
|
||||
"model.layers.82.input_layernorm",
|
||||
"model.layers.82.mlp.gate",
|
||||
"model.layers.82.post_attention_layernorm",
|
||||
"model.layers.83.input_layernorm",
|
||||
"model.layers.83.mlp.gate",
|
||||
"model.layers.83.post_attention_layernorm",
|
||||
"model.layers.84.input_layernorm",
|
||||
"model.layers.84.mlp.gate",
|
||||
"model.layers.84.post_attention_layernorm",
|
||||
"model.layers.85.input_layernorm",
|
||||
"model.layers.85.mlp.gate",
|
||||
"model.layers.85.post_attention_layernorm",
|
||||
"model.layers.86.input_layernorm",
|
||||
"model.layers.86.mlp.gate",
|
||||
"model.layers.86.post_attention_layernorm",
|
||||
"model.layers.87.input_layernorm",
|
||||
"model.layers.87.mlp.gate",
|
||||
"model.layers.87.post_attention_layernorm",
|
||||
"model.layers.88.input_layernorm",
|
||||
"model.layers.88.mlp.gate",
|
||||
"model.layers.88.post_attention_layernorm",
|
||||
"model.layers.89.input_layernorm",
|
||||
"model.layers.89.mlp.gate",
|
||||
"model.layers.89.post_attention_layernorm",
|
||||
"model.layers.90.input_layernorm",
|
||||
"model.layers.90.mlp.gate",
|
||||
"model.layers.90.post_attention_layernorm",
|
||||
"model.layers.91.input_layernorm",
|
||||
"model.layers.91.mlp.gate",
|
||||
"model.layers.91.post_attention_layernorm",
|
||||
"model.layers.92.input_layernorm",
|
||||
"model.layers.92.mlp.gate",
|
||||
"model.layers.92.post_attention_layernorm",
|
||||
"model.layers.93.input_layernorm",
|
||||
"model.layers.93.mlp.gate",
|
||||
"model.layers.93.post_attention_layernorm"
|
||||
],
|
||||
"fmt": "e4m3",
|
||||
"quant_method": "fp8",
|
||||
"weight_block_size": [
|
||||
128,
|
||||
128
|
||||
],
|
||||
"is_checkpoint_fp8_serialized": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03094400092959404,0.052000001072883606,0.03423200035467744,0.032816000282764435,0.004586225105504425,0.05225599929690361,0.08137600123882294,0.06316960025578737,0.0586559996008873,0.009046516570964667,0.025087999179959297,0.05766399949789047,0.030939200054854156,0.028768000192940235,0.006844666670168347,0.23388800024986267,0.31091201305389404,0.24544477462768555,0.23836800456047058,0.018666831776499748,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030368000268936157,0.05008000135421753,0.03488799966871738,0.03270399942994118,0.004964435488742052,0.05142400041222572,0.07401599735021591,0.05652640014886856,0.054847998544573784,0.005118024227402908,0.02534399926662445,0.0424639992415905,0.02885119989514351,0.028447999618947506,0.004027125677475938,0.24774399399757385,0.26895999908447266,0.25432640314102173,0.25200000405311584,0.00582256680354476,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.0307839997112751,0.04447999969124794,0.03474240032956004,0.03252799995243549,0.00435773849976965,0.051231998950242996,0.07468800246715546,0.057651200145483014,0.0561280008405447,0.006050027441177822,0.02579200081527233,0.033504001796245575,0.02867360021919012,0.02796800062060356,0.00238178652420847,0.2533760070800781,0.2958720028400421,0.2585648000240326,0.25679999589920044,0.009045161306858063,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030592000111937523,0.04108799993991852,0.03223840007558465,0.031279999762773514,0.002451635813407811,0.05177599936723709,0.077504001557827,0.05718399975448847,0.0533440001308918,0.00711033940595454,0.02457600086927414,0.051552001386880875,0.03192960014566779,0.028431999497115612,0.007824328074264588,0.2531839907169342,0.28329598903656006,0.26060959696769714,0.2584640085697174,0.006803486030548811,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030527999624609947,0.03500799834728241,0.03184479987248778,0.03139200061559677,0.001123070738289876,0.05104000121355057,0.07103999704122543,0.05895199999213219,0.05676800012588501,0.0068221931474717916,0.024383999407291412,0.04211200028657913,0.02812959998846054,0.026959999464452267,0.003887262202895282,0.2314240038394928,0.2559039890766144,0.23696160316467285,0.23375999927520752,0.006377317477017641,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/aligned
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03232000023126602,0.07411199808120728,0.040055999718606475,0.03792000003159046,0.010027335937042725,0.05955199897289276,0.09151999652385712,0.06805919948965311,0.06542399898171425,0.009156033646696494,0.02735999971628189,0.04198399931192398,0.031628800183534624,0.03081599995493889,0.0038650835976484594,0.29440000653266907,0.37036800384521484,0.3045775890350342,0.3012160062789917,0.01622786745429039,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.03190400078892708,0.0639680027961731,0.036531200259923936,0.03387199901044369,0.007307958553439168,0.05407999828457832,0.08684799820184708,0.0631104001775384,0.059248000383377075,0.008437822166776614,0.02537599951028824,0.045343998819589615,0.029841599892824887,0.028095999732613564,0.004578510902843549,0.29868799448013306,0.3216319978237152,0.30579519271850586,0.30294400453567505,0.006667278707027435,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.031199999153614044,0.06745599955320358,0.03718719966709614,0.03391999937593937,0.008084069620200455,0.05353600159287453,0.06828799843788147,0.059427200257778166,0.05902400054037571,0.0036973349098869714,0.026688000187277794,0.03788800165057182,0.030459199845790864,0.02908799983561039,0.0032231049972104124,0.30588799715042114,0.3216319978237152,0.3109855651855469,0.31036800146102905,0.004134960938245058,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.031488001346588135,0.04790399968624115,0.03488959986716509,0.033615998923778534,0.003995903359891917,0.05503999814391136,0.08563199639320374,0.06150399968028068,0.057312000542879105,0.007950341155323642,0.026335999369621277,0.04598399996757507,0.030313600040972232,0.02792000025510788,0.005356499011224523,0.31516799330711365,0.32972800731658936,0.3209056258201599,0.3196159899234772,0.004338567610830069,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030751999467611313,0.044895999133586884,0.03544640000909567,0.03302400000393391,0.00457756723742277,0.05331199988722801,0.07606399804353714,0.05961279980838299,0.05753600038588047,0.006503363955077464,0.02630399912595749,0.04368000105023384,0.03112160013988614,0.028815999627113342,0.005258638278682459,0.2905920147895813,0.3538239896297455,0.3015664219856262,0.29631999135017395,0.014866933226585388,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/compute_type
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.029888000339269638,0.05558399856090546,0.036182400118559596,0.03598400019109249,0.00590375348379892,0.05135999992489815,0.07407999783754349,0.05939359981566668,0.05599999986588955,0.007274243962538696,0.024831999093294144,0.060575999319553375,0.03204159988090396,0.029280000366270542,0.008986441399241034,0.23401600122451782,0.2914240062236786,0.24253761768341064,0.23852799832820892,0.012656064704060555,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.0306560005992651,0.04927999898791313,0.0345616003498435,0.032096000388264656,0.005272357401818311,0.0514880008995533,0.08454400300979614,0.05950720049440861,0.055904000997543335,0.008599599958802731,0.024639999493956566,0.04726399853825569,0.029726399946957825,0.028032000176608562,0.005891950556036845,0.24633599817752838,0.276095986366272,0.2544096112251282,0.25091201066970825,0.006771203130483627,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030208000913262367,0.062144000083208084,0.03510080017149449,0.03203200176358223,0.007286512102806094,0.050592001527547836,0.07017599791288376,0.05696159955114126,0.054735999554395676,0.006055532934843585,0.0244159996509552,0.04364800080657005,0.02875520009547472,0.02700799982994795,0.00429941663275523,0.252703994512558,0.27452799677848816,0.25873440504074097,0.2567040026187897,0.006180537864565849,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.028863999992609024,0.04185599833726883,0.03244960019364953,0.03129600081592798,0.00309197601199972,0.05049600079655647,0.07241600006818771,0.05591519977897406,0.05273599922657013,0.006825017255220366,0.024000000208616257,0.04150399938225746,0.027609600126743315,0.02619200013577938,0.0041488046270812895,0.25491198897361755,0.27008000016212463,0.2593088150024414,0.25811201333999634,0.003932233899831772,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030400000512599945,0.039135999977588654,0.03235360030084848,0.031888000667095184,0.002007792126883597,0.050944000482559204,0.06815999746322632,0.054641599953174594,0.053279999643564224,0.003969876178544029,0.024064000695943832,0.03145600110292435,0.027134399861097336,0.026559999212622643,0.0021875101737656014,0.23164799809455872,0.2622720003128052,0.2388928234577179,0.23686400055885315,0.007063580676913261,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/config_key
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03094400092959404,0.05427199974656105,0.03751839986070991,0.03566399961709976,0.0063355673062837495,0.052960000932216644,0.10150399804115295,0.06679840013384819,0.06265599839389324,0.012010412593353396,0.025087999179959297,0.0607680007815361,0.03164320001378655,0.028655999340116978,0.008879480144565048,0.29548799991607666,0.3161599934101105,0.30511200428009033,0.30371201038360596,0.006655826233327389,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030368000268936157,0.045184001326560974,0.03435200024396181,0.03299199976027012,0.003866832846889978,0.053247999399900436,0.10054399818181992,0.061161600053310394,0.05702400021255016,0.010951609505981603,0.025248000398278236,0.039872001856565475,0.029195200372487306,0.027520000003278255,0.003716525371772022,0.29631999135017395,0.3163839876651764,0.30516156554222107,0.30246400833129883,0.0062532913871109486,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030688000842928886,0.04044799879193306,0.03336800048127771,0.032368000596761703,0.002819613112867307,0.053568001836538315,0.06889600306749344,0.05800320040434599,0.055616000667214394,0.00500911399891226,0.025631999596953392,0.040991999208927155,0.028961599990725517,0.027951999567449093,0.003321893930324548,0.30828800797462463,0.3516480028629303,0.3159376084804535,0.3127039968967438,0.009800842963159084,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.031007999554276466,0.044319998472929,0.03418559962883592,0.03271999955177307,0.0038270996330861703,0.0541439987719059,0.07507199794054031,0.06345439981669188,0.06393599882721901,0.006346258200489106,0.025407999753952026,0.03494400158524513,0.028180800192058087,0.027375999838113785,0.0024874374625372497,0.3179520070552826,0.331712007522583,0.3228943943977356,0.32150399684906006,0.004389750771224499,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030912000685930252,0.04745600000023842,0.03517600009217858,0.03243200108408928,0.005035846835878437,0.053279999643564224,0.09200000017881393,0.06018720027059317,0.05721599981188774,0.008845081624923161,0.025151999667286873,0.04851200059056282,0.0323488000780344,0.02820800058543682,0.0077821073314592575,0.2917119860649109,0.3189440071582794,0.301118403673172,0.29817599058151245,0.008174914866685867,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/original
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,5 @@
|
||||
time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type
|
||||
0.010463999584317207,0.04681599885225296,0.018502399697899817,0.01104000024497509,0.014205381674858167,0.021247999742627144,0.033055998384952545,0.025439999625086786,0.024639999493956566,0.004202660707027564,0.05206400156021118,0.09040000289678574,0.0665344014763832,0.06451199948787689,0.013110543934264527,0.007327999919652939,0.011680000461637974,0.008480000123381615,0.0077760000713169575,0.0016149067494504018,0.009472000412642956,0.011103999800980091,0.010118400119245053,0.009600000455975533,0.0007061491557578713,4096,64,4,16,4,40960,1,128,0,True,FLASHINFER,False,even,[128],128,128,128,128.0,128,0.0,0.0,0.0,False,0,128,128,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
0.010751999914646149,0.012896000407636166,0.011795200034976005,0.011359999887645245,0.0009191526000700493,0.020191999152302742,0.02755199931561947,0.02319999970495701,0.021247999742627144,0.002944486774412461,0.049215998500585556,0.06700800359249115,0.05813760012388229,0.058240000158548355,0.006895964777123649,0.007519999984651804,0.009727999567985535,0.008140799775719642,0.007807999849319458,0.0008127242879711131,0.009472000412642956,0.025248000398278236,0.01303040012717247,0.010015999898314476,0.0061233119538994085,4096,64,4,16,4,40960,1,32,0,True,FLASHINFER,False,even,[32],32,32,32,32.0,32,0.0,0.0,0.0,False,0,32,32,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
0.010239999741315842,0.011008000001311302,0.010515199974179268,0.010432000271975994,0.0002635681936991548,0.02112000063061714,0.03843199834227562,0.025894399732351303,0.023231999948620796,0.006378760808481684,0.05417599901556969,0.06438399851322174,0.05709439888596535,0.055615998804569244,0.003697870830170538,0.007135999854654074,0.00774399982765317,0.007500800024718046,0.007552000228315592,0.0002009640969704077,0.009279999881982803,0.012736000120639801,0.01031040009111166,0.009568000212311745,0.0012940667684018582,4096,64,4,16,4,40960,1,64,0,True,FLASHINFER,False,even,[64],64,64,64,64.0,64,0.0,0.0,0.0,False,0,64,64,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
0.010367999784648418,0.02332800067961216,0.013459200039505959,0.010847999714314938,0.004984552697426283,0.02054399996995926,0.0261439997702837,0.02255360037088394,0.022112000733613968,0.001928344367854761,0.04851200059056282,0.07968000322580338,0.058873600512743,0.056352000683546066,0.010813603870069636,0.00723200011998415,0.008352000266313553,0.007686400134116411,0.007648000027984381,0.00039524862182609345,0.008960000239312649,0.011103999800980091,0.009881599992513656,0.00940799992531538,0.0009090251198168286,4096,64,4,16,4,40960,1,96,0,True,FLASHINFER,False,even,[96],96,96,96,96.0,96,0.0,0.0,0.0,False,0,96,96,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
|
@@ -0,0 +1,5 @@
|
||||
time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type,is_true_mixed_batch
|
||||
0.010463999584317207,0.04681599885225296,0.018502399697899817,0.01104000024497509,0.014205381674858167,0.021247999742627144,0.033055998384952545,0.025439999625086786,0.024639999493956566,0.004202660707027564,0.05206400156021118,0.09040000289678574,0.0665344014763832,0.06451199948787689,0.013110543934264527,0.007327999919652939,0.011680000461637974,0.008480000123381615,0.0077760000713169575,0.0016149067494504018,0.009472000412642956,0.011103999800980091,0.010118400119245053,0.009600000455975533,0.0007061491557578713,4096,64,4,16,4,40960,1,128,0,True,FLASHINFER,False,even,[128],128,128,128,128.0,128,0.0,0.0,0.0,False,0,128,128,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
0.010751999914646149,0.012896000407636166,0.011795200034976005,0.011359999887645245,0.0009191526000700493,0.020191999152302742,0.02755199931561947,0.02319999970495701,0.021247999742627144,0.002944486774412461,0.049215998500585556,0.06700800359249115,0.05813760012388229,0.058240000158548355,0.006895964777123649,0.007519999984651804,0.009727999567985535,0.008140799775719642,0.007807999849319458,0.0008127242879711131,0.009472000412642956,0.025248000398278236,0.01303040012717247,0.010015999898314476,0.0061233119538994085,4096,64,4,16,4,40960,1,32,0,True,FLASHINFER,False,even,[32],32,32,32,32.0,32,0.0,0.0,0.0,False,0,32,32,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
0.010239999741315842,0.011008000001311302,0.010515199974179268,0.010432000271975994,0.0002635681936991548,0.02112000063061714,0.03843199834227562,0.025894399732351303,0.023231999948620796,0.006378760808481684,0.05417599901556969,0.06438399851322174,0.05709439888596535,0.055615998804569244,0.003697870830170538,0.007135999854654074,0.00774399982765317,0.007500800024718046,0.007552000228315592,0.0002009640969704077,0.009279999881982803,0.012736000120639801,0.01031040009111166,0.009568000212311745,0.0012940667684018582,4096,64,4,16,4,40960,1,64,0,True,FLASHINFER,False,even,[64],64,64,64,64.0,64,0.0,0.0,0.0,False,0,64,64,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
0.010367999784648418,0.02332800067961216,0.013459200039505959,0.010847999714314938,0.004984552697426283,0.02054399996995926,0.0261439997702837,0.02255360037088394,0.022112000733613968,0.001928344367854761,0.04851200059056282,0.07968000322580338,0.058873600512743,0.056352000683546066,0.010813603870069636,0.00723200011998415,0.008352000266313553,0.007686400134116411,0.007648000027984381,0.00039524862182609345,0.008960000239312649,0.011103999800980091,0.009881599992513656,0.00940799992531538,0.0009090251198168286,4096,64,4,16,4,40960,1,96,0,True,FLASHINFER,False,even,[96],96,96,96,96.0,96,0.0,0.0,0.0,False,0,96,96,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
|
@@ -0,0 +1,3 @@
|
||||
time_stats.attn_pre_proj.min,time_stats.attn_pre_proj.max,time_stats.attn_pre_proj.mean,time_stats.attn_pre_proj.median,time_stats.attn_pre_proj.std,time_stats.attn_rope.min,time_stats.attn_rope.max,time_stats.attn_rope.mean,time_stats.attn_rope.median,time_stats.attn_rope.std,time_stats.attn_post_proj.min,time_stats.attn_post_proj.max,time_stats.attn_post_proj.mean,time_stats.attn_post_proj.median,time_stats.attn_post_proj.std,time_stats.emb.min,time_stats.emb.max,time_stats.emb.mean,time_stats.emb.median,time_stats.emb.std,time_stats.input_layernorm.min,time_stats.input_layernorm.max,time_stats.input_layernorm.mean,time_stats.input_layernorm.median,time_stats.input_layernorm.std,time_stats.post_attention_layernorm.min,time_stats.post_attention_layernorm.max,time_stats.post_attention_layernorm.mean,time_stats.post_attention_layernorm.median,time_stats.post_attention_layernorm.std,n_head,n_kv_head,n_embd,n_expanded_embd,vocab_size,use_gated_mlp,use_qk_norm,attn_output_gate,num_tokens,num_tensor_parallel_workers,padded_n_embd,padded_n_expanded_embd,model_arch,is_step2_mini,share_expert_dim,share_q_dim,measurement_type,profiling_precision,quant_signature
|
||||
0.19225600361824036,0.26678401231765747,0.22318400144577027,0.22147200256586075,0.020290217906394733,0.024639999493956566,0.04320000112056732,0.028563199937343596,0.026335999369621277,0.006067654243700026,0.09071999788284302,0.12992000579833984,0.11053120009601117,0.11033599823713303,0.011489030217015594,,,,,,,,,,,,,,,,64,4,4096,1536,151936,True,True,False,16,4,4096,1536,generic,False,,,CUDA_EVENT,BF16,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
,,,,,,,,,,,,,,,0.05990400165319443,0.1345279961824417,0.08581680012866855,0.0899839997291565,0.020920650895404905,0.020160000771284103,0.0360959991812706,0.02369120018556714,0.021824000403285027,0.00439515404502141,0.01833599992096424,0.02425600029528141,0.020108799915760756,0.019407999701797962,0.001409577392533308,64,4,4096,1536,151936,True,True,False,16,1,4096,1536,generic,False,,,CUDA_EVENT,BF16,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,2 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03097599931061268,0.049056001007556915,0.03467839974910021,0.03254400007426739,0.005093522706269737,0.05193600058555603,0.08419200032949448,0.06054240055382252,0.05641600117087364,0.009051489911083033,0.025919999927282333,0.04064000025391579,0.030527999717742206,0.030608000233769417,0.0040231266205605675,0.29603201150894165,0.3494400084018707,0.3075023889541626,0.30137598514556885,0.014090820215642452,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,65 @@
|
||||
attention_backend: FLASHINFER
|
||||
batch_size_list:
|
||||
- 1
|
||||
block_shape: null
|
||||
block_size: 16
|
||||
decode_kv_cache_size_list: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_chunked_prefill_grid_search: false
|
||||
enable_mixed_prefill: false
|
||||
enable_true_mixed: false
|
||||
fixed_chunked_prefill_size: 128
|
||||
max_batch_size: 1
|
||||
max_mixed_batch_size: 8
|
||||
max_model_len: 40960
|
||||
max_pipeline_parallel_size: 1
|
||||
max_seq_len: 128
|
||||
min_batch_size: 1
|
||||
mixed_batch_size_list: null
|
||||
mixed_batch_size_max: 32
|
||||
mixed_batch_size_min: 2
|
||||
mixed_kv_cache_size_list:
|
||||
- 0
|
||||
mixed_mode: both
|
||||
mixed_num_samples: 3
|
||||
mixed_profile_strategy: default
|
||||
mixed_shapes_per_point: 2
|
||||
mixed_total_tokens_list: null
|
||||
mixed_total_tokens_max: 1055
|
||||
mixed_total_tokens_min: 1025
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
profile_only_decode: false
|
||||
profile_only_prefill: true
|
||||
skip_confirmation: true
|
||||
true_mixed_decode_batch_sizes:
|
||||
- 1
|
||||
- 2
|
||||
- 4
|
||||
- 8
|
||||
true_mixed_decode_kv_cache_sizes:
|
||||
- 128
|
||||
- 256
|
||||
- 512
|
||||
- 1024
|
||||
- 2048
|
||||
true_mixed_prefill_batch_sizes:
|
||||
- 1
|
||||
- 2
|
||||
- 4
|
||||
true_mixed_prefill_chunk_sizes:
|
||||
- 64
|
||||
- 128
|
||||
- 256
|
||||
- 512
|
||||
- 1024
|
||||
true_mixed_prefill_kv_cache_size: 0
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,23 @@
|
||||
attn_tp: null
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
extra_num_tokens: null
|
||||
ffn_tp: null
|
||||
include_target_embedded_mtp: false
|
||||
is_moe: true
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
ray_enable_datasets_serializers: false
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 1
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"flashinfer_metadata_version": "0.3.1.post1",
|
||||
"flashinfer_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/flashinfer/__init__.py",
|
||||
"frontier_metadata_version": "0.1.0",
|
||||
"frontier_path": "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6/frontier",
|
||||
"python": "3.12.3",
|
||||
"torch": "2.8.0+cu128",
|
||||
"torch_cuda": "12.8",
|
||||
"vllm_import_version": "0.10.2",
|
||||
"vllm_metadata_version": "0.10.2",
|
||||
"vllm_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/vllm/__init__.py"
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"flashinfer-python": "0.3.1.post1",
|
||||
"frontier-simulator": "0.1.0",
|
||||
"python": "3.12.3",
|
||||
"tokenizers": "0.21.4",
|
||||
"torch": "2.8.0",
|
||||
"transformers": "4.55.2",
|
||||
"vllm": "0.10.2"
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
5184cb3f458685616364a09db28eee1603af26d58894073428a67ab677d655f5 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/environment.post_transformers_pin.json
|
||||
2d17f91ca22ed56dbffc4549972814be911901f321963f8f07a2b940df191340 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/requirements.freeze.txt
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"flashinfer_metadata_version": "0.3.1.post1",
|
||||
"flashinfer_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/flashinfer/__init__.py",
|
||||
"frontier_metadata_version": "0.1.0",
|
||||
"frontier_path": "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6/frontier",
|
||||
"python": "3.12.3",
|
||||
"torch": "2.8.0+cu128",
|
||||
"torch_cuda": "12.8",
|
||||
"vllm_import_version": "0.10.2",
|
||||
"vllm_metadata_version": "0.10.2",
|
||||
"vllm_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/vllm/__init__.py"
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
8564395327d10aff62c48e0e4fb74bdd050d17faed3d8d3948ff2e96e767d03a /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/environment.json
|
||||
a931219d3038697371775d2e5d797321adefc4ca48da2ab5999e1206e17a3881 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/requirements.freeze.txt
|
||||
@@ -0,0 +1,332 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3MoeForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"decoder_sparse_step": 1,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 12288,
|
||||
"max_position_embeddings": 40960,
|
||||
"max_window_layers": 94,
|
||||
"mlp_only_layers": [],
|
||||
"model_type": "qwen3_moe",
|
||||
"moe_intermediate_size": 1536,
|
||||
"norm_topk_prob": true,
|
||||
"num_attention_heads": 64,
|
||||
"num_experts": 128,
|
||||
"num_experts_per_tok": 8,
|
||||
"num_hidden_layers": 94,
|
||||
"num_key_value_heads": 4,
|
||||
"output_router_logits": false,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000.0,
|
||||
"router_aux_loss_coef": 0.001,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.0",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936,
|
||||
"quantization_config": {
|
||||
"activation_scheme": "dynamic",
|
||||
"modules_to_not_convert": [
|
||||
"lm_head",
|
||||
"model.layers.0.input_layernorm",
|
||||
"model.layers.0.mlp.gate",
|
||||
"model.layers.0.post_attention_layernorm",
|
||||
"model.layers.1.input_layernorm",
|
||||
"model.layers.1.mlp.gate",
|
||||
"model.layers.1.post_attention_layernorm",
|
||||
"model.layers.2.input_layernorm",
|
||||
"model.layers.2.mlp.gate",
|
||||
"model.layers.2.post_attention_layernorm",
|
||||
"model.layers.3.input_layernorm",
|
||||
"model.layers.3.mlp.gate",
|
||||
"model.layers.3.post_attention_layernorm",
|
||||
"model.layers.4.input_layernorm",
|
||||
"model.layers.4.mlp.gate",
|
||||
"model.layers.4.post_attention_layernorm",
|
||||
"model.layers.5.input_layernorm",
|
||||
"model.layers.5.mlp.gate",
|
||||
"model.layers.5.post_attention_layernorm",
|
||||
"model.layers.6.input_layernorm",
|
||||
"model.layers.6.mlp.gate",
|
||||
"model.layers.6.post_attention_layernorm",
|
||||
"model.layers.7.input_layernorm",
|
||||
"model.layers.7.mlp.gate",
|
||||
"model.layers.7.post_attention_layernorm",
|
||||
"model.layers.8.input_layernorm",
|
||||
"model.layers.8.mlp.gate",
|
||||
"model.layers.8.post_attention_layernorm",
|
||||
"model.layers.9.input_layernorm",
|
||||
"model.layers.9.mlp.gate",
|
||||
"model.layers.9.post_attention_layernorm",
|
||||
"model.layers.10.input_layernorm",
|
||||
"model.layers.10.mlp.gate",
|
||||
"model.layers.10.post_attention_layernorm",
|
||||
"model.layers.11.input_layernorm",
|
||||
"model.layers.11.mlp.gate",
|
||||
"model.layers.11.post_attention_layernorm",
|
||||
"model.layers.12.input_layernorm",
|
||||
"model.layers.12.mlp.gate",
|
||||
"model.layers.12.post_attention_layernorm",
|
||||
"model.layers.13.input_layernorm",
|
||||
"model.layers.13.mlp.gate",
|
||||
"model.layers.13.post_attention_layernorm",
|
||||
"model.layers.14.input_layernorm",
|
||||
"model.layers.14.mlp.gate",
|
||||
"model.layers.14.post_attention_layernorm",
|
||||
"model.layers.15.input_layernorm",
|
||||
"model.layers.15.mlp.gate",
|
||||
"model.layers.15.post_attention_layernorm",
|
||||
"model.layers.16.input_layernorm",
|
||||
"model.layers.16.mlp.gate",
|
||||
"model.layers.16.post_attention_layernorm",
|
||||
"model.layers.17.input_layernorm",
|
||||
"model.layers.17.mlp.gate",
|
||||
"model.layers.17.post_attention_layernorm",
|
||||
"model.layers.18.input_layernorm",
|
||||
"model.layers.18.mlp.gate",
|
||||
"model.layers.18.post_attention_layernorm",
|
||||
"model.layers.19.input_layernorm",
|
||||
"model.layers.19.mlp.gate",
|
||||
"model.layers.19.post_attention_layernorm",
|
||||
"model.layers.20.input_layernorm",
|
||||
"model.layers.20.mlp.gate",
|
||||
"model.layers.20.post_attention_layernorm",
|
||||
"model.layers.21.input_layernorm",
|
||||
"model.layers.21.mlp.gate",
|
||||
"model.layers.21.post_attention_layernorm",
|
||||
"model.layers.22.input_layernorm",
|
||||
"model.layers.22.mlp.gate",
|
||||
"model.layers.22.post_attention_layernorm",
|
||||
"model.layers.23.input_layernorm",
|
||||
"model.layers.23.mlp.gate",
|
||||
"model.layers.23.post_attention_layernorm",
|
||||
"model.layers.24.input_layernorm",
|
||||
"model.layers.24.mlp.gate",
|
||||
"model.layers.24.post_attention_layernorm",
|
||||
"model.layers.25.input_layernorm",
|
||||
"model.layers.25.mlp.gate",
|
||||
"model.layers.25.post_attention_layernorm",
|
||||
"model.layers.26.input_layernorm",
|
||||
"model.layers.26.mlp.gate",
|
||||
"model.layers.26.post_attention_layernorm",
|
||||
"model.layers.27.input_layernorm",
|
||||
"model.layers.27.mlp.gate",
|
||||
"model.layers.27.post_attention_layernorm",
|
||||
"model.layers.28.input_layernorm",
|
||||
"model.layers.28.mlp.gate",
|
||||
"model.layers.28.post_attention_layernorm",
|
||||
"model.layers.29.input_layernorm",
|
||||
"model.layers.29.mlp.gate",
|
||||
"model.layers.29.post_attention_layernorm",
|
||||
"model.layers.30.input_layernorm",
|
||||
"model.layers.30.mlp.gate",
|
||||
"model.layers.30.post_attention_layernorm",
|
||||
"model.layers.31.input_layernorm",
|
||||
"model.layers.31.mlp.gate",
|
||||
"model.layers.31.post_attention_layernorm",
|
||||
"model.layers.32.input_layernorm",
|
||||
"model.layers.32.mlp.gate",
|
||||
"model.layers.32.post_attention_layernorm",
|
||||
"model.layers.33.input_layernorm",
|
||||
"model.layers.33.mlp.gate",
|
||||
"model.layers.33.post_attention_layernorm",
|
||||
"model.layers.34.input_layernorm",
|
||||
"model.layers.34.mlp.gate",
|
||||
"model.layers.34.post_attention_layernorm",
|
||||
"model.layers.35.input_layernorm",
|
||||
"model.layers.35.mlp.gate",
|
||||
"model.layers.35.post_attention_layernorm",
|
||||
"model.layers.36.input_layernorm",
|
||||
"model.layers.36.mlp.gate",
|
||||
"model.layers.36.post_attention_layernorm",
|
||||
"model.layers.37.input_layernorm",
|
||||
"model.layers.37.mlp.gate",
|
||||
"model.layers.37.post_attention_layernorm",
|
||||
"model.layers.38.input_layernorm",
|
||||
"model.layers.38.mlp.gate",
|
||||
"model.layers.38.post_attention_layernorm",
|
||||
"model.layers.39.input_layernorm",
|
||||
"model.layers.39.mlp.gate",
|
||||
"model.layers.39.post_attention_layernorm",
|
||||
"model.layers.40.input_layernorm",
|
||||
"model.layers.40.mlp.gate",
|
||||
"model.layers.40.post_attention_layernorm",
|
||||
"model.layers.41.input_layernorm",
|
||||
"model.layers.41.mlp.gate",
|
||||
"model.layers.41.post_attention_layernorm",
|
||||
"model.layers.42.input_layernorm",
|
||||
"model.layers.42.mlp.gate",
|
||||
"model.layers.42.post_attention_layernorm",
|
||||
"model.layers.43.input_layernorm",
|
||||
"model.layers.43.mlp.gate",
|
||||
"model.layers.43.post_attention_layernorm",
|
||||
"model.layers.44.input_layernorm",
|
||||
"model.layers.44.mlp.gate",
|
||||
"model.layers.44.post_attention_layernorm",
|
||||
"model.layers.45.input_layernorm",
|
||||
"model.layers.45.mlp.gate",
|
||||
"model.layers.45.post_attention_layernorm",
|
||||
"model.layers.46.input_layernorm",
|
||||
"model.layers.46.mlp.gate",
|
||||
"model.layers.46.post_attention_layernorm",
|
||||
"model.layers.47.input_layernorm",
|
||||
"model.layers.47.mlp.gate",
|
||||
"model.layers.47.post_attention_layernorm",
|
||||
"model.layers.48.input_layernorm",
|
||||
"model.layers.48.mlp.gate",
|
||||
"model.layers.48.post_attention_layernorm",
|
||||
"model.layers.49.input_layernorm",
|
||||
"model.layers.49.mlp.gate",
|
||||
"model.layers.49.post_attention_layernorm",
|
||||
"model.layers.50.input_layernorm",
|
||||
"model.layers.50.mlp.gate",
|
||||
"model.layers.50.post_attention_layernorm",
|
||||
"model.layers.51.input_layernorm",
|
||||
"model.layers.51.mlp.gate",
|
||||
"model.layers.51.post_attention_layernorm",
|
||||
"model.layers.52.input_layernorm",
|
||||
"model.layers.52.mlp.gate",
|
||||
"model.layers.52.post_attention_layernorm",
|
||||
"model.layers.53.input_layernorm",
|
||||
"model.layers.53.mlp.gate",
|
||||
"model.layers.53.post_attention_layernorm",
|
||||
"model.layers.54.input_layernorm",
|
||||
"model.layers.54.mlp.gate",
|
||||
"model.layers.54.post_attention_layernorm",
|
||||
"model.layers.55.input_layernorm",
|
||||
"model.layers.55.mlp.gate",
|
||||
"model.layers.55.post_attention_layernorm",
|
||||
"model.layers.56.input_layernorm",
|
||||
"model.layers.56.mlp.gate",
|
||||
"model.layers.56.post_attention_layernorm",
|
||||
"model.layers.57.input_layernorm",
|
||||
"model.layers.57.mlp.gate",
|
||||
"model.layers.57.post_attention_layernorm",
|
||||
"model.layers.58.input_layernorm",
|
||||
"model.layers.58.mlp.gate",
|
||||
"model.layers.58.post_attention_layernorm",
|
||||
"model.layers.59.input_layernorm",
|
||||
"model.layers.59.mlp.gate",
|
||||
"model.layers.59.post_attention_layernorm",
|
||||
"model.layers.60.input_layernorm",
|
||||
"model.layers.60.mlp.gate",
|
||||
"model.layers.60.post_attention_layernorm",
|
||||
"model.layers.61.input_layernorm",
|
||||
"model.layers.61.mlp.gate",
|
||||
"model.layers.61.post_attention_layernorm",
|
||||
"model.layers.62.input_layernorm",
|
||||
"model.layers.62.mlp.gate",
|
||||
"model.layers.62.post_attention_layernorm",
|
||||
"model.layers.63.input_layernorm",
|
||||
"model.layers.63.mlp.gate",
|
||||
"model.layers.63.post_attention_layernorm",
|
||||
"model.layers.64.input_layernorm",
|
||||
"model.layers.64.mlp.gate",
|
||||
"model.layers.64.post_attention_layernorm",
|
||||
"model.layers.65.input_layernorm",
|
||||
"model.layers.65.mlp.gate",
|
||||
"model.layers.65.post_attention_layernorm",
|
||||
"model.layers.66.input_layernorm",
|
||||
"model.layers.66.mlp.gate",
|
||||
"model.layers.66.post_attention_layernorm",
|
||||
"model.layers.67.input_layernorm",
|
||||
"model.layers.67.mlp.gate",
|
||||
"model.layers.67.post_attention_layernorm",
|
||||
"model.layers.68.input_layernorm",
|
||||
"model.layers.68.mlp.gate",
|
||||
"model.layers.68.post_attention_layernorm",
|
||||
"model.layers.69.input_layernorm",
|
||||
"model.layers.69.mlp.gate",
|
||||
"model.layers.69.post_attention_layernorm",
|
||||
"model.layers.70.input_layernorm",
|
||||
"model.layers.70.mlp.gate",
|
||||
"model.layers.70.post_attention_layernorm",
|
||||
"model.layers.71.input_layernorm",
|
||||
"model.layers.71.mlp.gate",
|
||||
"model.layers.71.post_attention_layernorm",
|
||||
"model.layers.72.input_layernorm",
|
||||
"model.layers.72.mlp.gate",
|
||||
"model.layers.72.post_attention_layernorm",
|
||||
"model.layers.73.input_layernorm",
|
||||
"model.layers.73.mlp.gate",
|
||||
"model.layers.73.post_attention_layernorm",
|
||||
"model.layers.74.input_layernorm",
|
||||
"model.layers.74.mlp.gate",
|
||||
"model.layers.74.post_attention_layernorm",
|
||||
"model.layers.75.input_layernorm",
|
||||
"model.layers.75.mlp.gate",
|
||||
"model.layers.75.post_attention_layernorm",
|
||||
"model.layers.76.input_layernorm",
|
||||
"model.layers.76.mlp.gate",
|
||||
"model.layers.76.post_attention_layernorm",
|
||||
"model.layers.77.input_layernorm",
|
||||
"model.layers.77.mlp.gate",
|
||||
"model.layers.77.post_attention_layernorm",
|
||||
"model.layers.78.input_layernorm",
|
||||
"model.layers.78.mlp.gate",
|
||||
"model.layers.78.post_attention_layernorm",
|
||||
"model.layers.79.input_layernorm",
|
||||
"model.layers.79.mlp.gate",
|
||||
"model.layers.79.post_attention_layernorm",
|
||||
"model.layers.80.input_layernorm",
|
||||
"model.layers.80.mlp.gate",
|
||||
"model.layers.80.post_attention_layernorm",
|
||||
"model.layers.81.input_layernorm",
|
||||
"model.layers.81.mlp.gate",
|
||||
"model.layers.81.post_attention_layernorm",
|
||||
"model.layers.82.input_layernorm",
|
||||
"model.layers.82.mlp.gate",
|
||||
"model.layers.82.post_attention_layernorm",
|
||||
"model.layers.83.input_layernorm",
|
||||
"model.layers.83.mlp.gate",
|
||||
"model.layers.83.post_attention_layernorm",
|
||||
"model.layers.84.input_layernorm",
|
||||
"model.layers.84.mlp.gate",
|
||||
"model.layers.84.post_attention_layernorm",
|
||||
"model.layers.85.input_layernorm",
|
||||
"model.layers.85.mlp.gate",
|
||||
"model.layers.85.post_attention_layernorm",
|
||||
"model.layers.86.input_layernorm",
|
||||
"model.layers.86.mlp.gate",
|
||||
"model.layers.86.post_attention_layernorm",
|
||||
"model.layers.87.input_layernorm",
|
||||
"model.layers.87.mlp.gate",
|
||||
"model.layers.87.post_attention_layernorm",
|
||||
"model.layers.88.input_layernorm",
|
||||
"model.layers.88.mlp.gate",
|
||||
"model.layers.88.post_attention_layernorm",
|
||||
"model.layers.89.input_layernorm",
|
||||
"model.layers.89.mlp.gate",
|
||||
"model.layers.89.post_attention_layernorm",
|
||||
"model.layers.90.input_layernorm",
|
||||
"model.layers.90.mlp.gate",
|
||||
"model.layers.90.post_attention_layernorm",
|
||||
"model.layers.91.input_layernorm",
|
||||
"model.layers.91.mlp.gate",
|
||||
"model.layers.91.post_attention_layernorm",
|
||||
"model.layers.92.input_layernorm",
|
||||
"model.layers.92.mlp.gate",
|
||||
"model.layers.92.post_attention_layernorm",
|
||||
"model.layers.93.input_layernorm",
|
||||
"model.layers.93.mlp.gate",
|
||||
"model.layers.93.post_attention_layernorm"
|
||||
],
|
||||
"fmt": "e4m3",
|
||||
"quant_method": "fp8",
|
||||
"weight_block_size": [
|
||||
128,
|
||||
128
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/model_config.original.json
|
||||
1d7389f77563bb26bdb8cad077f9ae81f94dbe0b7f2e51cc72f7f4ff0f550acb /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6/data/config/models/Qwen3-235B-A22B-FP8.json
|
||||
@@ -0,0 +1,2 @@
|
||||
{'name': 'Qwen3-235B-A22B-FP8', 'num_layers': 94, 'num_q_heads': 64, 'num_kv_heads': 4, 'embedding_dim': 4096, 'mlp_hidden_dim': 1536, 'max_position_embeddings': 40960, 'use_gated_mlp': True, 'use_bias': False, 'use_qkv_bias': False, 'activation': 'silu', 'norm': 'rms_norm', 'post_attn_norm': True, 'vocab_size': 151936, 'is_neox_style': True, 'rope_theta': 1000000.0, 'rope_scaling': None, 'partial_rotary_factor': 1.0, 'no_tensor_parallel': False, 'is_moe': True, 'num_experts': 128, 'num_experts_per_tok': 8, 'moe_layers_enum': None, 'use_qk_norm': True, 'attn_output_gate': False, 'rms_norm_eps': 1e-06, 'dtype': 'BF16', 'model_type': 'qwen3_moe', 'fused_add_norm_capability': True, 'model_arch': 'generic', 'share_expert_dim': None, 'share_q_dim': None, 'head_dim': 128, 'quantization_config': {'quant_method': 'fp8', 'activation_scheme': 'dynamic', 'is_checkpoint_fp8_serialized': True, 'weight_block_size': (128, 128), 'ignored_layers': []}, 'tie_word_embeddings': False}
|
||||
{'tp_size': 4, 'attn_enabled': True, 'ffn_enabled': True, 'attn_sharded_enabled': True, 'ffn_sharded_enabled': True, 'replicated_enabled': True, 'disable_replicated': False, 'enabled_ops': ['input_layernorm', 'post_attention_layernorm', 'add', 'emb', 'attn_pre_proj', 'attn_rope', 'attn_post_proj'], 'disabled_ops': [], 'replicated_ops': ['input_layernorm', 'post_attention_layernorm', 'add', 'emb'], 'padded_n_embd': 4096, 'padded_n_expanded_embd': 1536, 'skip_reasons': []}
|
||||
@@ -0,0 +1 @@
|
||||
4459cca6fe7f01c44477460ccca85ec1f2a0eacdc764170c68fa02a102f4104e /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/platform.txt
|
||||
@@ -0,0 +1,99 @@
|
||||
captured_utc=2026-07-15T09:34:36Z
|
||||
ds-07429c65-1-6c5fd97778-9vhkr
|
||||
Linux ds-07429c65-1-6c5fd97778-9vhkr 5.10.134-013.8.2.kangaroo.al8.x86_64 #1 SMP Thu Mar 12 10:20:37 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
|
||||
PRETTY_NAME="Ubuntu 24.04.2 LTS"
|
||||
NAME="Ubuntu"
|
||||
VERSION_ID="24.04"
|
||||
VERSION="24.04.2 LTS (Noble Numbat)"
|
||||
VERSION_CODENAME=noble
|
||||
ID=ubuntu
|
||||
ID_LIKE=debian
|
||||
HOME_URL="https://www.ubuntu.com/"
|
||||
SUPPORT_URL="https://help.ubuntu.com/"
|
||||
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
|
||||
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
|
||||
UBUNTU_CODENAME=noble
|
||||
LOGO=ubuntu-logo
|
||||
Architecture: x86_64
|
||||
CPU op-mode(s): 32-bit, 64-bit
|
||||
Address sizes: 52 bits physical, 57 bits virtual
|
||||
Byte Order: Little Endian
|
||||
CPU(s): 160
|
||||
On-line CPU(s) list: 0-159
|
||||
Vendor ID: GenuineIntel
|
||||
Model name: Intel(R) Xeon(R) Processor
|
||||
CPU family: 6
|
||||
Model: 143
|
||||
Thread(s) per core: 1
|
||||
Core(s) per socket: 80
|
||||
Socket(s): 2
|
||||
Stepping: 8
|
||||
BogoMIPS: 5200.00
|
||||
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd avx512vbmi umip pku waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities
|
||||
Hypervisor vendor: KVM
|
||||
Virtualization type: full
|
||||
L1d cache: 3.8 MiB (80 instances)
|
||||
L1i cache: 2.5 MiB (80 instances)
|
||||
L2 cache: 160 MiB (80 instances)
|
||||
L3 cache: 195 MiB (2 instances)
|
||||
NUMA node(s): 2
|
||||
NUMA node0 CPU(s): 0-79
|
||||
NUMA node1 CPU(s): 80-159
|
||||
Vulnerability Itlb multihit: Not affected
|
||||
Vulnerability L1tf: Not affected
|
||||
Vulnerability Mds: Not affected
|
||||
Vulnerability Meltdown: Not affected
|
||||
Vulnerability Mmio stale data: Not affected
|
||||
Vulnerability Retbleed: Not affected
|
||||
Vulnerability Spec rstack overflow: Not affected
|
||||
Vulnerability Spec store bypass: Vulnerable
|
||||
Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
|
||||
Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Vulnerable
|
||||
Vulnerability Srbds: Not affected
|
||||
Vulnerability Tsx async abort: Not affected
|
||||
0, NVIDIA H20, GPU-ad3e049a-5bf0-44b7-e7f1-9af297b172af, 580.95.05, 97871 MiB, 9.0
|
||||
1, NVIDIA H20, GPU-8c088079-d0f5-ba23-8650-5e6b1436691f, 580.95.05, 97871 MiB, 9.0
|
||||
2, NVIDIA H20, GPU-a9f6fe67-324b-8bb2-19b3-c7f1b9cce96a, 580.95.05, 97871 MiB, 9.0
|
||||
3, NVIDIA H20, GPU-6bcd68b7-ffa7-26b5-df6a-b7eb3f65c901, 580.95.05, 97871 MiB, 9.0
|
||||
4, NVIDIA H20, GPU-b409f9c9-05b6-55ef-3f3b-12eaa7c6ebfe, 580.95.05, 97871 MiB, 9.0
|
||||
5, NVIDIA H20, GPU-56932433-efce-8215-6418-98166d8ab798, 580.95.05, 97871 MiB, 9.0
|
||||
6, NVIDIA H20, GPU-ddcd1b77-e38d-75f5-ac24-788c76e56c89, 580.95.05, 97871 MiB, 9.0
|
||||
7, NVIDIA H20, GPU-9b4a31bd-5e78-b5a7-55aa-d786ae5a3f21, 580.95.05, 97871 MiB, 9.0
|
||||
[4mGPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 NIC0 NIC1 NIC2 NIC3 CPU Affinity NUMA Affinity GPU NUMA ID[0m
|
||||
GPU0 X NV18 NV18 NV18 NV18 NV18 NV18 NV18 PIX PHB SYS SYS 0-79 0 N/A
|
||||
GPU1 NV18 X NV18 NV18 NV18 NV18 NV18 NV18 PXB PHB SYS SYS 0-79 0 N/A
|
||||
GPU2 NV18 NV18 X NV18 NV18 NV18 NV18 NV18 PHB PIX SYS SYS 0-79 0 N/A
|
||||
GPU3 NV18 NV18 NV18 X NV18 NV18 NV18 NV18 PHB PXB SYS SYS 0-79 0 N/A
|
||||
GPU4 NV18 NV18 NV18 NV18 X NV18 NV18 NV18 SYS SYS PIX PHB 80-159 1 N/A
|
||||
GPU5 NV18 NV18 NV18 NV18 NV18 X NV18 NV18 SYS SYS PXB PHB 80-159 1 N/A
|
||||
GPU6 NV18 NV18 NV18 NV18 NV18 NV18 X NV18 SYS SYS PHB PIX 80-159 1 N/A
|
||||
GPU7 NV18 NV18 NV18 NV18 NV18 NV18 NV18 X SYS SYS PHB PXB 80-159 1 N/A
|
||||
NIC0 PIX PXB PHB PHB SYS SYS SYS SYS X PHB SYS SYS
|
||||
NIC1 PHB PHB PIX PXB SYS SYS SYS SYS PHB X SYS SYS
|
||||
NIC2 SYS SYS SYS SYS PIX PXB PHB PHB SYS SYS X PHB
|
||||
NIC3 SYS SYS SYS SYS PHB PHB PIX PXB SYS SYS PHB X
|
||||
|
||||
Legend:
|
||||
|
||||
X = Self
|
||||
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
|
||||
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
|
||||
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
|
||||
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
|
||||
PIX = Connection traversing at most a single PCIe bridge
|
||||
NV# = Connection traversing a bonded set of # NVLinks
|
||||
|
||||
NIC Legend:
|
||||
|
||||
NIC0: mlx5_0
|
||||
NIC1: mlx5_1
|
||||
NIC2: mlx5_2
|
||||
NIC3: mlx5_3
|
||||
|
||||
nvcc: NVIDIA (R) Cuda compiler driver
|
||||
Copyright (c) 2005-2025 NVIDIA Corporation
|
||||
Built on Tue_May_27_02:21:03_PDT_2025
|
||||
Cuda compilation tools, release 12.9, V12.9.86
|
||||
Build cuda_12.9.r12.9/compiler.36037853_0
|
||||
gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
|
||||
ldd (Ubuntu GLIBC 2.39-0ubuntu8.4) 2.39
|
||||
@@ -0,0 +1 @@
|
||||
2d17f91ca22ed56dbffc4549972814be911901f321963f8f07a2b940df191340 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/requirements.freeze.txt
|
||||
@@ -0,0 +1,163 @@
|
||||
aiohappyeyeballs==2.7.1
|
||||
aiohttp==3.14.1
|
||||
aiosignal==1.4.0
|
||||
annotated-doc==0.0.4
|
||||
annotated-types==0.7.0
|
||||
anyio==4.14.2
|
||||
astor==0.8.1
|
||||
attrs==26.1.0
|
||||
blake3==1.0.9
|
||||
cachetools==7.1.4
|
||||
cbor2==6.1.3
|
||||
certifi==2026.6.17
|
||||
cffi==2.1.0
|
||||
charset-normalizer==3.4.9
|
||||
click==8.4.2
|
||||
cloudpickle==3.1.2
|
||||
compressed-tensors==0.11.0
|
||||
cuda-pathfinder==1.5.6
|
||||
cupy-cuda12x==14.1.1
|
||||
ddsketch==3.0.1
|
||||
depyf==0.19.0
|
||||
detect-installer==0.1.0
|
||||
dill==0.4.1
|
||||
diskcache==5.6.3
|
||||
distro==1.9.0
|
||||
dnspython==2.8.0
|
||||
einops==0.8.2
|
||||
email-validator==2.3.0
|
||||
fastapi==0.139.0
|
||||
fastapi-cli==0.0.29
|
||||
fastapi-cloud-cli==0.22.2
|
||||
fastar==0.11.0
|
||||
fasteners==0.20
|
||||
filelock==3.29.7
|
||||
flashinfer-python==0.3.1.post1
|
||||
-e file:///home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6
|
||||
frozendict==2.4.7
|
||||
frozenlist==1.8.0
|
||||
fsspec==2026.6.0
|
||||
gguf==0.19.0
|
||||
h11==0.16.0
|
||||
hf-xet==1.5.1
|
||||
httpcore==1.0.9
|
||||
httptools==0.8.0
|
||||
httpx==0.28.1
|
||||
huggingface-hub==0.36.2
|
||||
idna==3.18
|
||||
iniconfig==2.3.0
|
||||
interegular==0.3.3
|
||||
jinja2==3.1.6
|
||||
jiter==0.16.0
|
||||
joblib==1.5.3
|
||||
jsonschema==4.26.0
|
||||
jsonschema-specifications==2025.9.1
|
||||
lark==1.2.2
|
||||
llguidance==0.7.30
|
||||
llvmlite==0.44.0
|
||||
lm-format-enforcer==0.11.3
|
||||
markdown-it-py==4.2.0
|
||||
markupsafe==3.0.3
|
||||
mdurl==0.1.2
|
||||
mistral-common==1.11.5
|
||||
mpmath==1.3.0
|
||||
msgpack==1.2.1
|
||||
msgspec==0.21.1
|
||||
multidict==6.7.1
|
||||
narwhals==2.24.0
|
||||
networkx==3.6.1
|
||||
ninja==1.13.0
|
||||
numba==0.61.2
|
||||
numpy==2.2.6
|
||||
nvidia-cublas-cu12==12.8.4.1
|
||||
nvidia-cuda-cupti-cu12==12.8.90
|
||||
nvidia-cuda-nvrtc-cu12==12.8.93
|
||||
nvidia-cuda-runtime-cu12==12.8.90
|
||||
nvidia-cudnn-cu12==9.10.2.21
|
||||
nvidia-cudnn-frontend==1.26.0
|
||||
nvidia-cufft-cu12==11.3.3.83
|
||||
nvidia-cufile-cu12==1.13.1.3
|
||||
nvidia-curand-cu12==10.3.9.90
|
||||
nvidia-cusolver-cu12==11.7.3.90
|
||||
nvidia-cusparse-cu12==12.5.8.93
|
||||
nvidia-cusparselt-cu12==0.7.1
|
||||
nvidia-ml-py==13.610.43
|
||||
nvidia-nccl-cu12==2.27.3
|
||||
nvidia-nvjitlink-cu12==12.8.93
|
||||
nvidia-nvtx-cu12==12.8.90
|
||||
openai==2.45.0
|
||||
openai-harmony==0.0.8
|
||||
opencv-python-headless==5.0.0.93
|
||||
outlines-core==0.2.11
|
||||
packaging==26.2
|
||||
pandas==3.0.3
|
||||
partial-json-parser==0.2.1.1.post7
|
||||
pillow==12.3.0
|
||||
plotly==6.9.0
|
||||
pluggy==1.6.0
|
||||
prometheus-client==0.25.0
|
||||
prometheus-fastapi-instrumentator==8.0.2
|
||||
propcache==0.5.2
|
||||
protobuf==7.35.1
|
||||
psutil==7.2.2
|
||||
py-cpuinfo==9.0.0
|
||||
pybase64==1.4.3
|
||||
pycountry==26.2.16
|
||||
pycparser==3.0
|
||||
pydantic==2.13.4
|
||||
pydantic-core==2.46.4
|
||||
pydantic-extra-types==2.11.1
|
||||
pydantic-settings==2.14.2
|
||||
pygments==2.20.0
|
||||
pynvml==13.0.1
|
||||
pytest==9.1.1
|
||||
python-dateutil==2.9.0.post0
|
||||
python-dotenv==1.2.2
|
||||
python-json-logger==4.1.0
|
||||
python-multipart==0.0.32
|
||||
pyyaml==6.0.3
|
||||
pyzmq==27.1.0
|
||||
ray==2.56.0
|
||||
referencing==0.37.0
|
||||
regex==2026.7.10
|
||||
requests==2.34.2
|
||||
rich==15.0.0
|
||||
rich-toolkit==0.20.3
|
||||
rignore==0.7.6
|
||||
rpds-py==2026.6.3
|
||||
safetensors==0.8.0
|
||||
scikit-learn==1.9.0
|
||||
scipy==1.18.0
|
||||
sentencepiece==0.2.2
|
||||
sentry-sdk==2.65.0
|
||||
setproctitle==1.3.7
|
||||
setuptools==79.0.1
|
||||
shellingham==1.5.4
|
||||
six==1.17.0
|
||||
sniffio==1.3.1
|
||||
soundfile==0.14.0
|
||||
soxr==1.1.0
|
||||
starlette==1.3.1
|
||||
sympy==1.14.0
|
||||
tabulate==0.10.0
|
||||
threadpoolctl==3.6.0
|
||||
tiktoken==0.13.0
|
||||
tokenizers==0.21.4
|
||||
torch==2.8.0
|
||||
torchaudio==2.8.0
|
||||
torchvision==0.23.0
|
||||
tqdm==4.68.4
|
||||
transformers==4.55.2
|
||||
triton==3.4.0
|
||||
typer==0.26.8
|
||||
typing-extensions==4.16.0
|
||||
typing-inspection==0.4.2
|
||||
urllib3==2.7.0
|
||||
uvicorn==0.51.0
|
||||
uvloop==0.22.1
|
||||
vllm==0.10.2
|
||||
watchfiles==1.2.0
|
||||
websockets==16.1
|
||||
xformers==0.0.32.post1
|
||||
xgrammar==0.1.23
|
||||
yarl==1.24.2
|
||||
@@ -0,0 +1,163 @@
|
||||
aiohappyeyeballs==2.7.1
|
||||
aiohttp==3.14.1
|
||||
aiosignal==1.4.0
|
||||
annotated-doc==0.0.4
|
||||
annotated-types==0.7.0
|
||||
anyio==4.14.2
|
||||
astor==0.8.1
|
||||
attrs==26.1.0
|
||||
blake3==1.0.9
|
||||
cachetools==7.1.4
|
||||
cbor2==6.1.3
|
||||
certifi==2026.6.17
|
||||
cffi==2.1.0
|
||||
charset-normalizer==3.4.9
|
||||
click==8.4.2
|
||||
cloudpickle==3.1.2
|
||||
compressed-tensors==0.11.0
|
||||
cuda-pathfinder==1.5.6
|
||||
cupy-cuda12x==14.1.1
|
||||
ddsketch==3.0.1
|
||||
depyf==0.19.0
|
||||
detect-installer==0.1.0
|
||||
dill==0.4.1
|
||||
diskcache==5.6.3
|
||||
distro==1.9.0
|
||||
dnspython==2.8.0
|
||||
einops==0.8.2
|
||||
email-validator==2.3.0
|
||||
fastapi==0.139.0
|
||||
fastapi-cli==0.0.29
|
||||
fastapi-cloud-cli==0.22.2
|
||||
fastar==0.11.0
|
||||
fasteners==0.20
|
||||
filelock==3.29.7
|
||||
flashinfer-python==0.3.1.post1
|
||||
-e file:///home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6
|
||||
frozendict==2.4.7
|
||||
frozenlist==1.8.0
|
||||
fsspec==2026.6.0
|
||||
gguf==0.19.0
|
||||
h11==0.16.0
|
||||
hf-xet==1.5.1
|
||||
httpcore==1.0.9
|
||||
httptools==0.8.0
|
||||
httpx==0.28.1
|
||||
huggingface-hub==1.23.0
|
||||
idna==3.18
|
||||
iniconfig==2.3.0
|
||||
interegular==0.3.3
|
||||
jinja2==3.1.6
|
||||
jiter==0.16.0
|
||||
joblib==1.5.3
|
||||
jsonschema==4.26.0
|
||||
jsonschema-specifications==2025.9.1
|
||||
lark==1.2.2
|
||||
llguidance==0.7.30
|
||||
llvmlite==0.44.0
|
||||
lm-format-enforcer==0.11.3
|
||||
markdown-it-py==4.2.0
|
||||
markupsafe==3.0.3
|
||||
mdurl==0.1.2
|
||||
mistral-common==1.11.5
|
||||
mpmath==1.3.0
|
||||
msgpack==1.2.1
|
||||
msgspec==0.21.1
|
||||
multidict==6.7.1
|
||||
narwhals==2.24.0
|
||||
networkx==3.6.1
|
||||
ninja==1.13.0
|
||||
numba==0.61.2
|
||||
numpy==2.2.6
|
||||
nvidia-cublas-cu12==12.8.4.1
|
||||
nvidia-cuda-cupti-cu12==12.8.90
|
||||
nvidia-cuda-nvrtc-cu12==12.8.93
|
||||
nvidia-cuda-runtime-cu12==12.8.90
|
||||
nvidia-cudnn-cu12==9.10.2.21
|
||||
nvidia-cudnn-frontend==1.26.0
|
||||
nvidia-cufft-cu12==11.3.3.83
|
||||
nvidia-cufile-cu12==1.13.1.3
|
||||
nvidia-curand-cu12==10.3.9.90
|
||||
nvidia-cusolver-cu12==11.7.3.90
|
||||
nvidia-cusparse-cu12==12.5.8.93
|
||||
nvidia-cusparselt-cu12==0.7.1
|
||||
nvidia-ml-py==13.610.43
|
||||
nvidia-nccl-cu12==2.27.3
|
||||
nvidia-nvjitlink-cu12==12.8.93
|
||||
nvidia-nvtx-cu12==12.8.90
|
||||
openai==2.45.0
|
||||
openai-harmony==0.0.8
|
||||
opencv-python-headless==5.0.0.93
|
||||
outlines-core==0.2.11
|
||||
packaging==26.2
|
||||
pandas==3.0.3
|
||||
partial-json-parser==0.2.1.1.post7
|
||||
pillow==12.3.0
|
||||
plotly==6.9.0
|
||||
pluggy==1.6.0
|
||||
prometheus-client==0.25.0
|
||||
prometheus-fastapi-instrumentator==8.0.2
|
||||
propcache==0.5.2
|
||||
protobuf==7.35.1
|
||||
psutil==7.2.2
|
||||
py-cpuinfo==9.0.0
|
||||
pybase64==1.4.3
|
||||
pycountry==26.2.16
|
||||
pycparser==3.0
|
||||
pydantic==2.13.4
|
||||
pydantic-core==2.46.4
|
||||
pydantic-extra-types==2.11.1
|
||||
pydantic-settings==2.14.2
|
||||
pygments==2.20.0
|
||||
pynvml==13.0.1
|
||||
pytest==9.1.1
|
||||
python-dateutil==2.9.0.post0
|
||||
python-dotenv==1.2.2
|
||||
python-json-logger==4.1.0
|
||||
python-multipart==0.0.32
|
||||
pyyaml==6.0.3
|
||||
pyzmq==27.1.0
|
||||
ray==2.56.0
|
||||
referencing==0.37.0
|
||||
regex==2026.7.10
|
||||
requests==2.34.2
|
||||
rich==15.0.0
|
||||
rich-toolkit==0.20.3
|
||||
rignore==0.7.6
|
||||
rpds-py==2026.6.3
|
||||
safetensors==0.8.0
|
||||
scikit-learn==1.9.0
|
||||
scipy==1.18.0
|
||||
sentencepiece==0.2.2
|
||||
sentry-sdk==2.65.0
|
||||
setproctitle==1.3.7
|
||||
setuptools==79.0.1
|
||||
shellingham==1.5.4
|
||||
six==1.17.0
|
||||
sniffio==1.3.1
|
||||
soundfile==0.14.0
|
||||
soxr==1.1.0
|
||||
starlette==1.3.1
|
||||
sympy==1.14.0
|
||||
tabulate==0.10.0
|
||||
threadpoolctl==3.6.0
|
||||
tiktoken==0.13.0
|
||||
tokenizers==0.22.2
|
||||
torch==2.8.0
|
||||
torchaudio==2.8.0
|
||||
torchvision==0.23.0
|
||||
tqdm==4.68.4
|
||||
transformers==5.13.1
|
||||
triton==3.4.0
|
||||
typer==0.26.8
|
||||
typing-extensions==4.16.0
|
||||
typing-inspection==0.4.2
|
||||
urllib3==2.7.0
|
||||
uvicorn==0.51.0
|
||||
uvloop==0.22.1
|
||||
vllm==0.10.2
|
||||
watchfiles==1.2.0
|
||||
websockets==16.1
|
||||
xformers==0.0.32.post1
|
||||
xgrammar==0.1.23
|
||||
yarl==1.24.2
|
||||
@@ -0,0 +1,209 @@
|
||||
# Community Qwen235B / Frontier smoke report
|
||||
|
||||
## Bottom line
|
||||
|
||||
The community-vLLM TP4 feasibility smoke passed, but the unmodified Frontier
|
||||
profiles are not valid inputs for a formal simulator-versus-runtime ranking
|
||||
test yet. The smoke exposed an execution-semantic mismatch inside the MoE
|
||||
profiler: serving selects vLLM's tuned H20 block-FP8 Triton config, while the
|
||||
standalone Frontier profiler omits the FP8 dtype key and silently falls back to
|
||||
a default config. At the tested TP4/EP1/16-token point, fixing this lookup
|
||||
reduces measured grouped-GEMM time by about 19%, with the direction consistent
|
||||
across five paired routing seeds.
|
||||
|
||||
This is precisely why merely installing the same vLLM release on both sides is
|
||||
not a sufficient alignment contract.
|
||||
|
||||
## Change
|
||||
|
||||
- Built an isolated Python 3.12 environment around community vLLM 0.10.2 and
|
||||
Frontier commit `d9cfeb6d8791fbf2f295dd9744c56a666171776e`.
|
||||
- Pinned Transformers 4.55.2 and tokenizers 0.21.4 after reproducing an
|
||||
incompatibility with the unconstrained Transformers 5.13.1 resolver result.
|
||||
- Added Frontier metadata for the local block-FP8 checkpoint without changing
|
||||
the original Hugging Face config.
|
||||
- Collected representative FP8 linear, FlashInfer prefill-attention, FP8 MoE,
|
||||
and TP4 NCCL all-reduce measurements.
|
||||
- Loaded the full Qwen3-235B-A22B-FP8 checkpoint on four H20s through community
|
||||
vLLM and completed one real request.
|
||||
- Ran a 2x2 MoE diagnostic that independently toggles the runtime FP8 tuning
|
||||
key and BF16 compute type.
|
||||
|
||||
## Expected effect
|
||||
|
||||
The smoke was intended to determine whether community vLLM can provide a
|
||||
shared, reproducible profiler/serving substrate before paying for the complete
|
||||
response surface. It was not intended to establish Frontier ranking accuracy.
|
||||
|
||||
Success required all representative operator and collective paths to execute,
|
||||
the TP4 server to become ready, one request to return successfully, and all GPU
|
||||
processes to be released. A discovered profiler/runtime semantic mismatch was
|
||||
treated as a gate on the later simulator comparison rather than calibrated
|
||||
away with end-to-end serving data.
|
||||
|
||||
## Frozen environment
|
||||
|
||||
- Host: `dash0`; 8 NVIDIA H20 GPUs.
|
||||
- Model: `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8`.
|
||||
- Original model-config SHA256:
|
||||
`702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e`.
|
||||
- Python 3.12.3, torch 2.8.0+cu128, vLLM 0.10.2,
|
||||
flashinfer-python 0.3.1.post1, Transformers 4.55.2, tokenizers 0.21.4.
|
||||
- Ubuntu 24.04.2, kernel 5.10.134, CUDA runtime 12.8, CUDA toolkit 12.9,
|
||||
NVIDIA driver 580.95.05, GCC 13.3.0, glibc 2.39.
|
||||
- Two 80-core Intel Xeon sockets (160 online CPUs), 8 H20 GPUs with
|
||||
97,871 MiB each and all-to-all NV18 links; GPU0--3 share NUMA node 0.
|
||||
- Serving: TP4, FlashInfer, eager execution, custom all-reduce disabled,
|
||||
FP8 weights, BF16 KV cache, MNS=64, MBT=8192, max model length 40,960,
|
||||
prefix caching and speculative decoding disabled.
|
||||
|
||||
The first environment resolved Transformers 5.13.1 because vLLM 0.10.2 only
|
||||
declares `transformers>=4.55.2`. That environment failed before model loading
|
||||
because vLLM accesses `all_special_tokens_extended`, an API absent from the
|
||||
new tokenizer implementation. Pinning the declared minimum restored the API;
|
||||
the original and corrected freezes are both retained.
|
||||
|
||||
## Verification
|
||||
|
||||
### End-to-end smoke
|
||||
|
||||
| Stage | Result |
|
||||
|---|---|
|
||||
| FP8 linear, TP4, 16 tokens | Passed; QKV projection mean 0.2232 ms and output projection mean 0.1105 ms |
|
||||
| FlashInfer prefill attention, TP4 | Passed for sequence lengths 32/64/96/128; attention mean 0.0571--0.0665 ms |
|
||||
| FP8 MoE, TP4/EP1, 16 tokens | Passed; unmodified grouped-GEMM mean 0.3075 ms in the initial point |
|
||||
| TP4 NCCL all-reduce, 128 KiB/rank | Passed; mean 0.03694 ms, p50 0.03138 ms, p95 0.05843 ms |
|
||||
| Community-vLLM TP4 model load | Passed; 55.1328 GiB of weights/rank, 627 seconds |
|
||||
| KV-cache sizing | 18.72 GiB/rank, 417,616 tokens; reported 10.20x concurrency at length 40,960 |
|
||||
| API readiness and one-token completion | Passed; `/v1/models` and `/v1/completions` returned HTTP 200 |
|
||||
| Cleanup | Passed; all eight GPUs returned to zero compute processes and zero MiB used |
|
||||
|
||||
The successful serving-only run started at 09:09:52 UTC and completed at
|
||||
09:22:29 UTC (12 minutes 37 seconds). The one-request response contains one
|
||||
completion choice and one completion token. The copied artifacts match the
|
||||
remote SHA256 manifests.
|
||||
|
||||
### FP8 path audit
|
||||
|
||||
The Frontier confirmation UI prints `FP8 Quantization: N/A` and labels the
|
||||
operations BF16 because it displays the unset CLI override rather than the
|
||||
model-config-derived operation precision. This is misleading, not evidence of
|
||||
a BF16-weight profile:
|
||||
|
||||
- the quantization manager configured `attn_pre_proj`, `attn_post_proj`, and
|
||||
`moe_grouped_gemm` as FP8 from the model config;
|
||||
- the linear implementation invokes `apply_w8a8_block_fp8_linear` with a
|
||||
128x128 weight block;
|
||||
- the MoE implementation quantizes weights and activations and invokes vLLM's
|
||||
fused kernel with `use_fp8_w8a8=True`;
|
||||
- `FRONTIER_FP8_GEMM_SURROGATE` was not enabled;
|
||||
- each CSV carries
|
||||
`method=fp8|act=dynamic|serialized=True|block=128x128` separately from its
|
||||
BF16 output/compute dtype metadata.
|
||||
|
||||
### MoE runtime-alignment factorial
|
||||
|
||||
Point: Qwen3-235B-A22B-FP8, TP4, EP1, 16 input tokens, uniform routing seeds
|
||||
0--4. Values are the mean of each row's CUDA-event samples, then averaged
|
||||
across the five paired routing seeds.
|
||||
|
||||
| Variant | Grouped GEMM mean (ms) | Paired delta vs original | Routing-seed 95% CI |
|
||||
|---|---:|---:|---:|
|
||||
| Original: default config + FP16 compute type | 0.3100 | 0.00% | -- |
|
||||
| FP8 config key only + FP16 compute type | 0.2508 | -19.12% | [-21.26%, -16.98%] |
|
||||
| Default config + BF16 compute type only | 0.3088 | -0.40% | [-1.31%, 0.51%] |
|
||||
| FP8 config key + BF16 compute type | 0.2512 | -18.99% | [-21.14%, -16.85%] |
|
||||
|
||||
These intervals describe variation across the five routing seeds, not
|
||||
independent process/server-run uncertainty. Even with that limitation, the
|
||||
factorial localizes the dominant error at this point to kernel tuning-config
|
||||
selection rather than FP16-versus-BF16 compute type.
|
||||
|
||||
The concrete lookup difference is:
|
||||
|
||||
- Frontier calls `get_config_dtype_str(base_dtype)`. For BF16 this returns
|
||||
`None`, so it searches for a config without an FP8 dtype component and uses
|
||||
the default when that file is absent.
|
||||
- vLLM serving calls the same helper with `use_fp8_w8a8=True`, obtains
|
||||
`fp8_w8a8`, and loads
|
||||
`E=128,N=384,device_name=NVIDIA_H20,dtype=fp8_w8a8,block_shape=[128,128].json`.
|
||||
|
||||
Frontier also hard-codes `tl.float16` as the FP8 kernel compute type, while the
|
||||
serving path uses BF16 for this model. That difference was not material at this
|
||||
single point, but it should still be corrected to make the counterfactual
|
||||
execution semantics identical.
|
||||
|
||||
## Validity audit
|
||||
|
||||
### Headline claims and benchmark surface
|
||||
|
||||
| Claim | Verdict | Scope |
|
||||
|---|---|---|
|
||||
| Community vLLM 0.10.2 can load and serve this model on TP4 H20 | PASS | One frozen TP4 configuration and one real request |
|
||||
| Unmodified Frontier chooses a different MoE tuning config, causing a material operator-time error at the tested point | PASS | TP4/EP1, 16 tokens, five paired routing seeds |
|
||||
| Frontier is or is not sufficient as a Qwen235B config ranker | NEEDS EVIDENCE | No aligned full profiles, frozen simulator surface, or real config surface yet |
|
||||
|
||||
The mechanism diagnostic uses the unmodified Frontier wrapper as baseline,
|
||||
grouped-GEMM CUDA-event latency as its direct metric, and the same model,
|
||||
hardware, vLLM binary, token count, TP/EP point, and routing seeds in all four
|
||||
cells. Absolute times, paired relative differences, across-seed standard
|
||||
deviations, and a paired routing-seed interval are retained. The data range is
|
||||
deliberately narrow and cannot support an end-to-end or cross-config claim.
|
||||
|
||||
| Benchmark issue | Verdict | Severity | Evidence / required action |
|
||||
|---|---|---|---|
|
||||
| Microbenchmark presented as end-to-end performance | PASS | -- | The 19% number is reported only as mechanism evidence; ranking remains blocked. |
|
||||
| Simplified profiler matches the real execution path | FAIL | Blocking | Tuning-key and compute-type semantics differ. Apply the alignment patch and test exact config dictionaries before simulation. |
|
||||
| Statistical significance / repeat protocol | NEEDS EVIDENCE | Major | Five paired routing seeds are not independent process repeats. Repeat in reverse/randomized variant order and at separated times. |
|
||||
| Selective parameter range | NEEDS EVIDENCE | Major | Only TP4/EP1/16 tokens is measured. Cover the full decision-relevant token and TP range, including boundary values. |
|
||||
| Calibration set equals evaluation set | PASS | -- | No serving result calibrated the profile; the protocol freezes simulator outputs before ground truth. |
|
||||
| Proper diagnostic baseline | PASS | -- | Each ablation is compared with the exact unmodified Frontier commit on the same point. |
|
||||
| Platform and absolute values missing | PASS | -- | OS, kernel, CPU, GPU topology, driver/toolkit/runtime versions, commands, absolute metrics, logs, and hashes are retained. |
|
||||
| Full ranking baseline/SOTA comparison | N/A | -- | This smoke makes no ranking or tuner-superiority claim; it becomes required in the formal evaluation. |
|
||||
|
||||
Overall audit decision: the feasibility smoke and localized mismatch result are
|
||||
reportable, but any Frontier-sufficiency or config-ranking claim is **Block**
|
||||
until the required reruns and end-to-end comparison are complete.
|
||||
|
||||
## Result
|
||||
|
||||
The shared community stack is feasible: profiles can be collected and the
|
||||
235B checkpoint can serve on TP4 within memory. The stronger result, however,
|
||||
is a falsification of the current alignment assumption. Same model, hardware,
|
||||
vLLM package, quantization signature, and nominal backend did not imply the
|
||||
same kernel configuration. A small wrapper-level omission produced a roughly
|
||||
19% operator-time error before any scheduling approximation was involved.
|
||||
|
||||
Therefore no Frontier config ranking should be generated from the original
|
||||
MoE CSV and presented as a fair simulator-versus-real comparison. The correct
|
||||
next boundary is: first make the profiler reproduce the serving kernel
|
||||
selection, then freeze profile-only simulator outputs, and only then collect
|
||||
the real response surface.
|
||||
|
||||
## Remaining risk and next gate
|
||||
|
||||
- The 19% result is one TP4/EP1/16-token point. It must be checked across the
|
||||
decision-relevant token counts and TP4/TP8 before claiming its effect on
|
||||
config ordering.
|
||||
- Frontier's shuffling helper still performs a separate BF16/full-expert-width
|
||||
config lookup and reports a missing `E=128,N=1536` H20 file. At 16 tokens its
|
||||
default `BLOCK_SIZE_M` matches the tuned path, but this must be validated over
|
||||
the full token grid.
|
||||
- The derived Frontier model config adds
|
||||
`is_checkpoint_fp8_serialized=true`, which Frontier requires to parse the
|
||||
checkpoint metadata. This adaptation must remain explicit and hash-tracked.
|
||||
- The representative attention and collective smokes do not provide the
|
||||
profile coverage needed by the prefill candidate grid. Decode-dominant EP8
|
||||
remains blocked on all-to-all measurement/consumption.
|
||||
- A TP4 cold start spends about 10.5 minutes reading weights. Using that time as
|
||||
a lower-bound proxy, eight TP4/TP8 cells already cost roughly 8.4
|
||||
H20-GPU-hours in weight loading alone; a real one-pass response surface will
|
||||
exceed 10 H20-GPU-hours after initialization and load probes. It requires a
|
||||
separate launch approval after simulator outputs are frozen.
|
||||
|
||||
Recommended next step: upstream the two semantic fixes with regression tests
|
||||
that compare the exact vLLM config dictionary selected by profiler and serving
|
||||
for each `(M, TP, EP, quantization)` point; align the shuffling block-size
|
||||
lookup; regenerate TP4/TP8 profile closure; then run and hash the Frontier
|
||||
prefill predictions. Do not start the eight-cell real serving sweep before
|
||||
that gate passes.
|
||||
@@ -0,0 +1 @@
|
||||
{"object":"list","data":[{"id":"qwen3-235b-community-smoke","object":"model","created":1784107344,"owned_by":"vllm","root":"/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8","parent":null,"max_model_len":40960,"permission":[{"id":"modelperm-1715f628151642daa743a9bc47bffc88","object":"model_permission","created":1784107344,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]}]}
|
||||
@@ -0,0 +1,6 @@
|
||||
b186a5da9809c3f2a3ac8e008837719af2f0dd65c583e022d44183b3bd7587d0 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/moe_factorial_r5_summary.json
|
||||
ef9e8136bc6fc6d97a675cab95cb5b0425ac727d6d1e77598beadbf3d9cf4331 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/moe_factorial_r5_cells.csv
|
||||
8d9a6aad4ee9fbb3ed054ef1798850042077df40f16c4cf3e490abe7d416bf6e /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/aligned/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
bc2bf6fdb6987dc73d07c17034037f539ab2ba79901ee3ff6d33094c6beb2ffd /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/compute_type/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
9769cd09cb35060ce6e3ed6610700485edf0ab00eb74c97199b200a0f90bf3bc /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/config_key/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
0607dcb87a952ded5679f74ce151ca1624aa8f0484a40924920f0fd2ccb85634 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/original/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
@@ -0,0 +1,21 @@
|
||||
variant,seed,grouped_gemm_mean_ms,paired_delta_vs_original_pct
|
||||
original,0,0.3051120042800903,0.0
|
||||
original,1,0.305161565542221,0.0
|
||||
original,2,0.3159376084804535,0.0
|
||||
original,3,0.3228943943977356,0.0
|
||||
original,4,0.301118403673172,0.0
|
||||
config_key,0,0.2425376176834106,-20.508660989699024
|
||||
config_key,1,0.2544096112251282,-16.631175104543427
|
||||
config_key,2,0.2587344050407409,-18.105854416901945
|
||||
config_key,3,0.2593088150024414,-19.692376361594743
|
||||
config_key,4,0.2388928234577179,-20.664821364752093
|
||||
compute_type,0,0.3045775890350342,-0.17515379190571334
|
||||
compute_type,1,0.3057951927185058,0.20763662526077642
|
||||
compute_type,2,0.3109855651855469,-1.567411780675354
|
||||
compute_type,3,0.3209056258201599,-0.6159192020924187
|
||||
compute_type,4,0.3015664219856262,0.14878476605517665
|
||||
aligned,0,0.2454447746276855,-19.55584467847773
|
||||
aligned,1,0.2543264031410217,-16.658441999690808
|
||||
aligned,2,0.2585648000240326,-18.159537489811207
|
||||
aligned,3,0.2606095969676971,-19.289525773964723
|
||||
aligned,4,0.2369616031646728,-21.306170504986376
|
||||
|
@@ -0,0 +1,55 @@
|
||||
{
|
||||
"ci_scope": "paired t interval across routing seeds; not independent server-run uncertainty",
|
||||
"metric": "time_stats.moe_grouped_gemm.mean",
|
||||
"point": {
|
||||
"ep": 1,
|
||||
"model": "Qwen3-235B-A22B-FP8",
|
||||
"routing_seeds": [
|
||||
0,
|
||||
1,
|
||||
2,
|
||||
3,
|
||||
4
|
||||
],
|
||||
"tokens": 16,
|
||||
"tp": 4
|
||||
},
|
||||
"variants": {
|
||||
"aligned": {
|
||||
"mean_ms": 0.25118143558502193,
|
||||
"paired_delta_pct_95ci": [
|
||||
-21.135576186756534,
|
||||
-16.8522319920158
|
||||
],
|
||||
"paired_delta_pct_mean": -18.993904089386167,
|
||||
"sd_across_seeds_ms": 0.009854035105320383
|
||||
},
|
||||
"compute_type": {
|
||||
"mean_ms": 0.3087660789489746,
|
||||
"paired_delta_pct_95ci": [
|
||||
-1.3065750560434517,
|
||||
0.5057497027004384
|
||||
],
|
||||
"paired_delta_pct_mean": -0.4004126766715066,
|
||||
"sd_across_seeds_ms": 0.007591103189515662
|
||||
},
|
||||
"config_key": {
|
||||
"mean_ms": 0.2507766544818878,
|
||||
"paired_delta_pct_95ci": [
|
||||
-21.259089864169376,
|
||||
-16.98206543082712
|
||||
],
|
||||
"paired_delta_pct_mean": -19.120577647498248,
|
||||
"sd_across_seeds_ms": 0.009466111912783425
|
||||
},
|
||||
"original": {
|
||||
"mean_ms": 0.3100447952747345,
|
||||
"paired_delta_pct_95ci": [
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"paired_delta_pct_mean": 0.0,
|
||||
"sd_across_seeds_ms": 0.009051191520196651
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"id":"cmpl-78d1852cd51e445dab3372e1e2025d75","object":"text_completion","created":1784107344,"model":"qwen3-235b-community-smoke","choices":[{"index":0,"text":" ","logprobs":null,"finish_reason":"length","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt_token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":1,"total_tokens":2,"completion_tokens":1,"prompt_tokens_details":null},"kv_transfer_params":null}
|
||||
@@ -0,0 +1,3 @@
|
||||
a44ff3d6a0998324b52df358baf778b375f98be3e03c24b07f903f9c51525beb /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/allreduce_tp4.jsonl
|
||||
b6edf2c29065cdd303ec642bac85ceafcec30494dcce7ddf3d401c36de5162fe /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/models.json
|
||||
921505259d86a9037f2241cf5d611942cdf2985352c7080d1af0d39750fab211 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/one_request.json
|
||||
@@ -0,0 +1,17 @@
|
||||
case,cell_id,tp,dp,ep,mns,mbt,gpu_count,score_req_s_per_gpu,capacity_lower_bound_req_s_per_gpu,capacity_upper_bound_req_s_per_gpu,capacity_bracket_width_req_s_per_gpu,best_request_rate_req_s,best_sampling_u,best_pass_rate,probe_count,best_source,completed_with_probe_failure,fully_valid,result_path,result_sha256
|
||||
qwen235b_prefill_only,tp4_mns64_mbt8192,4,1,1,64,8192,4,0.1175,0.1175,0.13333333333333333,0.015833333333333338,0.47,0.021484375,0.9609929078014184,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/store/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/trials/trial-0001/result.json,cc9d8e3611c9719b5d8f8a14d376c24f08b348162cc49022889d9435816da13b
|
||||
qwen235b_prefill_only,tp4_mns64_mbt16384,4,1,1,64,16384,4,0.10666666666666667,0.10666666666666667,0.1175,0.01083333333333332,0.4266666666666667,0.01953125,0.9609375,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/store/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/trials/trial-0002/result.json,768360741f64ab39b3e7af497757c4fd92c66d5a0eff0125cc3d13f2814a0186
|
||||
qwen235b_prefill_only,tp4_mns128_mbt8192,4,1,1,128,8192,4,0.1175,0.1175,0.13333333333333333,0.015833333333333338,0.47,0.021484375,0.9609929078014184,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/store/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/trials/trial-0003/result.json,6cb56fb8ad0f9dc6e68b30528ed8993207c3951ef152ce8705a04c7fcf45a02a
|
||||
qwen235b_prefill_only,tp4_mns128_mbt16384,4,1,1,128,16384,4,0.10666666666666667,0.10666666666666667,0.1175,0.01083333333333332,0.4266666666666667,0.01953125,0.9609375,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0001/result.json,3c485648244243d529df8d7f16b0fb3d782cef1beec695f67cdb1fd5ea02b440
|
||||
qwen235b_prefill_only,tp8_mns64_mbt8192,8,1,1,64,8192,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9638118214716526,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0002/result.json,95c38d14ddc88f7f08730b7486ebb11538da6cec9ede11f5d16862b683290850
|
||||
qwen235b_prefill_only,tp8_mns64_mbt16384,8,1,1,64,16384,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9565741857659831,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0003/result.json,97b96526c568a572f10de602075c59a0a14555114f65d7c8e928b51fcf1cb787
|
||||
qwen235b_prefill_only,tp8_mns128_mbt8192,8,1,1,128,8192,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9601930036188179,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0004/result.json,f2695efe64c9c728f381e0d39ee42e7553b771928547f7a74a2cb2dafdc892f4
|
||||
qwen235b_prefill_only,tp8_mns128_mbt16384,8,1,1,128,16384,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9589867310012062,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0005/result.json,1bc4d53f5ecbf4fbbd7c749a29ecf0f62e8382b84d899cc366d5bb485d78e261
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns64_mbt256,4,2,8,64,256,8,0.05354166666666667,0.05354166666666667,0.058958333333333335,0.005416666666666667,0.42833333333333334,0.01953125,1.0,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0001/result.json,f5a1c9ff3e8f98c37f8d3623e0376f34f67990f2e71e3b012beb8ac8ed447367
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns64_mbt384,4,2,8,64,384,8,0.05354166666666667,0.05354166666666667,0.058958333333333335,0.005416666666666667,0.42833333333333334,0.01953125,0.9922178988326849,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0002/result.json,adaafb060dc3ba642da34af7b69ee09ee0d04d857b8d4b5121a4b124373e1c57
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns128_mbt256,4,2,8,128,256,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9929328621908127,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0003/result.json,3555b3ecdef6337050c4e132e2b6f36a3cb98ea002a96f3efe118d301ad9d258
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns128_mbt384,4,2,8,128,384,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9929328621908127,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0004/result.json,3a010b17bc2cb785a57ef80a08646972109ad2dae28eeb38c153b35a1e3b6be2
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns64_mbt256,2,4,8,64,256,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9752650176678446,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0005/result.json,79649819c5599c8b81a115281a8e73aab5bc1a8188a1f611ace2329f9ae2f418
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns64_mbt384,2,4,8,64,384,8,0.05354166666666667,0.05354166666666667,0.058958333333333335,0.005416666666666667,0.42833333333333334,0.01953125,0.9961089494163424,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0006/result.json,6fdfeccfdbeb92bad8669251b26546966f46b42d667ae70eacdc544b44ad231d
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns128_mbt256,2,4,8,128,256,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9787985865724381,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0007/result.json,8bc1c359e338de1d62bc87c43b1fa270b15023d223450552bffe5d1b778458f3
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns128_mbt384,2,4,8,128,384,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9823321554770318,6,partial_probe_before_failure,True,False,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0008/result.json,c371c2dd38db875db827ab85d4f399d0219cad513dde3c3b4db8cc782ec4debb
|
||||
|
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"cases": [
|
||||
{
|
||||
"best_score_req_s_per_gpu": 0.17270833333333332,
|
||||
"case": "qwen235b_prefill_only",
|
||||
"cell_count": 8,
|
||||
"distinct_score_count": 3,
|
||||
"distinct_scores_req_s_per_gpu": [
|
||||
0.17270833333333332,
|
||||
0.1175,
|
||||
0.10666666666666667
|
||||
],
|
||||
"fully_valid_cell_count": 8,
|
||||
"informative_pair_count": 20,
|
||||
"informative_pair_fraction": 0.7142857142857143,
|
||||
"invalid_cells": [],
|
||||
"pair_count": 28,
|
||||
"possibly_optimal_set_from_search_brackets": [
|
||||
"tp8_mns128_mbt16384",
|
||||
"tp8_mns128_mbt8192",
|
||||
"tp8_mns64_mbt16384",
|
||||
"tp8_mns64_mbt8192"
|
||||
],
|
||||
"possibly_optimal_set_size": 4,
|
||||
"random_top_set_hit_rate": 0.5,
|
||||
"tied_pair_count": 8,
|
||||
"top_set": [
|
||||
"tp8_mns128_mbt16384",
|
||||
"tp8_mns128_mbt8192",
|
||||
"tp8_mns64_mbt16384",
|
||||
"tp8_mns64_mbt8192"
|
||||
],
|
||||
"top_set_size": 4
|
||||
},
|
||||
{
|
||||
"best_score_req_s_per_gpu": 0.058958333333333335,
|
||||
"case": "qwen235b_decode_only",
|
||||
"cell_count": 8,
|
||||
"distinct_score_count": 2,
|
||||
"distinct_scores_req_s_per_gpu": [
|
||||
0.058958333333333335,
|
||||
0.05354166666666667
|
||||
],
|
||||
"fully_valid_cell_count": 7,
|
||||
"informative_pair_count": 15,
|
||||
"informative_pair_fraction": 0.5357142857142857,
|
||||
"invalid_cells": [
|
||||
"tp2_dp4_ep8_mns128_mbt384"
|
||||
],
|
||||
"pair_count": 28,
|
||||
"possibly_optimal_set_from_search_brackets": [
|
||||
"tp2_dp4_ep8_mns128_mbt256",
|
||||
"tp2_dp4_ep8_mns128_mbt384",
|
||||
"tp2_dp4_ep8_mns64_mbt256",
|
||||
"tp2_dp4_ep8_mns64_mbt384",
|
||||
"tp4_dp2_ep8_mns128_mbt256",
|
||||
"tp4_dp2_ep8_mns128_mbt384",
|
||||
"tp4_dp2_ep8_mns64_mbt256",
|
||||
"tp4_dp2_ep8_mns64_mbt384"
|
||||
],
|
||||
"possibly_optimal_set_size": 8,
|
||||
"random_top_set_hit_rate": 0.625,
|
||||
"tied_pair_count": 13,
|
||||
"top_set": [
|
||||
"tp2_dp4_ep8_mns128_mbt256",
|
||||
"tp2_dp4_ep8_mns128_mbt384",
|
||||
"tp2_dp4_ep8_mns64_mbt256",
|
||||
"tp4_dp2_ep8_mns128_mbt256",
|
||||
"tp4_dp2_ep8_mns128_mbt384"
|
||||
],
|
||||
"top_set_size": 5
|
||||
}
|
||||
],
|
||||
"schema": "frontier-multicase-ground-truth-v0"
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
# Qwen235B ground-truth audit
|
||||
|
||||
Objective: maximum SLO-feasible offered request throughput per GPU.
|
||||
This report contains real-machine data only; it makes no Frontier match claim.
|
||||
|
||||
| case | valid cells | score levels | top-set size | random top-set hit | informative pairs |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| qwen235b_prefill_only | 8/8 | 3 | 4/8 | 50.0% | 20/28 (71.4%) |
|
||||
| qwen235b_decode_only | 7/8 | 2 | 5/8 | 62.5% | 15/28 (53.6%) |
|
||||
|
||||
## Cells
|
||||
|
||||
### qwen235b_prefill_only
|
||||
|
||||
| cell | capacity bracket (req/s/GPU) | valid | probes | source |
|
||||
|---|---:|---:|---:|---|
|
||||
| tp4_mns128_mbt16384 | [0.106666667, 0.117500000) | yes | 6 | primary_search |
|
||||
| tp4_mns128_mbt8192 | [0.117500000, 0.133333333) | yes | 6 | primary_search |
|
||||
| tp4_mns64_mbt16384 | [0.106666667, 0.117500000) | yes | 6 | primary_search |
|
||||
| tp4_mns64_mbt8192 | [0.117500000, 0.133333333) | yes | 6 | primary_search |
|
||||
| tp8_mns128_mbt16384 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
| tp8_mns128_mbt8192 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
| tp8_mns64_mbt16384 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
| tp8_mns64_mbt8192 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
|
||||
Top set: `tp8_mns128_mbt16384, tp8_mns128_mbt8192, tp8_mns64_mbt16384, tp8_mns64_mbt8192`.
|
||||
Possibly optimal under binary-search brackets: `tp8_mns128_mbt16384, tp8_mns128_mbt8192, tp8_mns64_mbt16384, tp8_mns64_mbt8192`.
|
||||
|
||||
### qwen235b_decode_only
|
||||
|
||||
| cell | capacity bracket (req/s/GPU) | valid | probes | source |
|
||||
|---|---:|---:|---:|---|
|
||||
| tp2_dp4_ep8_mns128_mbt256 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp2_dp4_ep8_mns128_mbt384 | [0.058958333, 0.066875000) | no | 6 | partial_probe_before_failure |
|
||||
| tp2_dp4_ep8_mns64_mbt256 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp2_dp4_ep8_mns64_mbt384 | [0.053541667, 0.058958333) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns128_mbt256 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns128_mbt384 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns64_mbt256 | [0.053541667, 0.058958333) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns64_mbt384 | [0.053541667, 0.058958333) | yes | 6 | primary_search |
|
||||
|
||||
Top set: `tp2_dp4_ep8_mns128_mbt256, tp2_dp4_ep8_mns128_mbt384, tp2_dp4_ep8_mns64_mbt256, tp4_dp2_ep8_mns128_mbt256, tp4_dp2_ep8_mns128_mbt384`.
|
||||
Possibly optimal under binary-search brackets: `tp2_dp4_ep8_mns128_mbt256, tp2_dp4_ep8_mns128_mbt384, tp2_dp4_ep8_mns64_mbt256, tp2_dp4_ep8_mns64_mbt384, tp4_dp2_ep8_mns128_mbt256, tp4_dp2_ep8_mns128_mbt384, tp4_dp2_ep8_mns64_mbt256, tp4_dp2_ep8_mns64_mbt384`.
|
||||
|
||||
## Interpretation guardrail
|
||||
|
||||
A Frontier top-set hit is insufficient by itself because the surfaces contain large ties. The later comparison must report selected-config regret and tie-aware pairwise ranking, and must keep invalid real cells visible.
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"calibration": {
|
||||
"fit_fixture": "coder_200_ts2",
|
||||
"fitted_a_tp": {
|
||||
"1": 0.7234810457606639,
|
||||
"2": 0.4680889959260082,
|
||||
"4": 0.3521372005220769
|
||||
},
|
||||
"holdout_fixture": "coder_200_ts3",
|
||||
"loss": "[log(G_raw_rerun(tp,scale2;a)/F_raw(tp,scale2))]^2 per TP",
|
||||
"refit_on_holdout": false
|
||||
},
|
||||
"rows": [
|
||||
{
|
||||
"agreement": 37,
|
||||
"false_feasible": 0,
|
||||
"false_infeasible": 55,
|
||||
"kendall_tau_b": 0.0,
|
||||
"mode": "uncalibrated/SLO-gated",
|
||||
"optimistic_real_regret": 0.25634517766497456,
|
||||
"pairwise_exact_sign_accuracy": 0.3787878787878788,
|
||||
"selected_cells": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64"
|
||||
],
|
||||
"worst_case_real_regret": 0.25634517766497456
|
||||
},
|
||||
{
|
||||
"agreement": 64,
|
||||
"false_feasible": 21,
|
||||
"false_infeasible": 7,
|
||||
"kendall_tau_b": 0.9668009539030813,
|
||||
"mode": "frozen-calibrated/SLO-gated",
|
||||
"optimistic_real_regret": 0.0,
|
||||
"pairwise_exact_sign_accuracy": 0.9393939393939394,
|
||||
"selected_cells": [
|
||||
"tp2_mns32",
|
||||
"tp2_mns64"
|
||||
],
|
||||
"worst_case_real_regret": 0.0076142131979695165
|
||||
}
|
||||
],
|
||||
"schema": "frontier-qwen30-calibration-audit-v0"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30B Frontier baseline audit
|
||||
|
||||
| mode | selected cells | worst real regret | Kendall tau-b | pair sign accuracy | feasibility (agree/FP/FN) |
|
||||
|---|---|---:|---:|---:|---:|
|
||||
| uncalibrated/SLO-gated | tp4_mns32, tp4_mns64 | 25.63% | 0.0000 | 37.88% | 37/0/55 |
|
||||
| frozen-calibrated/SLO-gated | tp2_mns32, tp2_mns64 | 0.76% | 0.9668 | 93.94% | 64/21/7 |
|
||||
|
||||
The calibrated mode applies a distinct end-to-end execution-time scale per TP: TP1=0.723481, TP2=0.468089, TP4=0.352137.
|
||||
|
||||
Those scales were fitted against real total throughput on `coder_200_ts2` and checked without refitting on `coder_200_ts3`. This validates within-workload transfer of the calibration, not zero-shot Frontier prediction across TP.
|
||||
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import json
|
||||
import os
|
||||
import statistics
|
||||
|
||||
import torch
|
||||
import torch.distributed as dist
|
||||
|
||||
|
||||
def main() -> None:
|
||||
local_rank = int(os.environ["LOCAL_RANK"])
|
||||
torch.cuda.set_device(local_rank)
|
||||
dist.init_process_group(backend="nccl")
|
||||
|
||||
# 16 tokens x 4096 hidden values in BF16: 128 KiB per rank.
|
||||
tensor = torch.ones((16, 4096), dtype=torch.bfloat16, device="cuda")
|
||||
for _ in range(10):
|
||||
dist.all_reduce(tensor)
|
||||
torch.cuda.synchronize()
|
||||
|
||||
samples_ms = []
|
||||
for _ in range(50):
|
||||
start = torch.cuda.Event(enable_timing=True)
|
||||
end = torch.cuda.Event(enable_timing=True)
|
||||
start.record()
|
||||
dist.all_reduce(tensor)
|
||||
end.record()
|
||||
end.synchronize()
|
||||
samples_ms.append(float(start.elapsed_time(end)))
|
||||
|
||||
if dist.get_rank() == 0:
|
||||
ordered = sorted(samples_ms)
|
||||
result = {
|
||||
"backend": "nccl",
|
||||
"collective": "all_reduce",
|
||||
"dtype": "bfloat16",
|
||||
"elements_per_rank": tensor.numel(),
|
||||
"bytes_per_rank": tensor.numel() * tensor.element_size(),
|
||||
"world_size": dist.get_world_size(),
|
||||
"warmup_iterations": 10,
|
||||
"measured_iterations": len(samples_ms),
|
||||
"mean_ms": statistics.fmean(samples_ms),
|
||||
"p50_ms": statistics.median(samples_ms),
|
||||
"p95_ms": ordered[int(0.95 * (len(ordered) - 1))],
|
||||
"min_ms": min(samples_ms),
|
||||
"max_ms": max(samples_ms),
|
||||
}
|
||||
print(json.dumps(result, sort_keys=True))
|
||||
|
||||
dist.destroy_process_group()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
diff --git a/frontier/profiling/moe/moe_vllm_kernel.py b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
--- a/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
+++ b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
@@ -232 +232 @@ def _invoke_kernel(
|
||||
- compute_type = tl.float16 # FP8 accumulates in FP16
|
||||
+ compute_type = tl.bfloat16
|
||||
@@ -0,0 +1,9 @@
|
||||
diff --git a/frontier/profiling/moe/moe_vllm_kernel.py b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
--- a/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
+++ b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
@@ -510 +510,4 @@ def profile_vllm_fused_moe(
|
||||
- config_dtype = get_config_dtype_str(base_dtype)
|
||||
+ config_dtype = get_config_dtype_str(
|
||||
+ base_dtype,
|
||||
+ use_fp8_w8a8=use_fp8,
|
||||
+ )
|
||||
@@ -0,0 +1,12 @@
|
||||
diff --git a/frontier/profiling/moe/moe_vllm_kernel.py b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
--- a/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
+++ b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
@@ -232 +232 @@ def _invoke_kernel(
|
||||
- compute_type = tl.float16 # FP8 accumulates in FP16
|
||||
+ compute_type = tl.bfloat16
|
||||
@@ -510 +510,4 @@ def profile_vllm_fused_moe(
|
||||
- config_dtype = get_config_dtype_str(base_dtype)
|
||||
+ config_dtype = get_config_dtype_str(
|
||||
+ base_dtype,
|
||||
+ use_fp8_w8a8=use_fp8,
|
||||
+ )
|
||||
189
runs/frontier-multicase-sufficiency-v0/smoke/run_gpu_smoke.sh
Normal file
189
runs/frontier-multicase-sufficiency-v0/smoke/run_gpu_smoke.sh
Normal file
@@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-${OUTPUT_ROOT}/Frontier-d9cfeb6}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8}"
|
||||
PROFILE_ROOT="${OUTPUT_ROOT}/profiles"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
RESULT_DIR="${OUTPUT_ROOT}/results"
|
||||
SERVER_PORT="${SERVER_PORT:-18900}"
|
||||
SKIP_LINEAR="${SKIP_LINEAR:-0}"
|
||||
SERVING_ONLY="${SERVING_ONLY:-0}"
|
||||
SERVED_MODEL="qwen3-235b-community-smoke"
|
||||
SERVER_PID=""
|
||||
|
||||
mkdir -p "${PROFILE_ROOT}" "${LOG_DIR}" "${RESULT_DIR}"
|
||||
exec > >(tee -a "${LOG_DIR}/gpu_smoke.log") 2>&1
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
for _ in $(seq 1 30); do
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
if [[ -z "${CUDA_VISIBLE_DEVICES:-}" ]]; then
|
||||
echo "ERROR: CUDA_VISIBLE_DEVICES must name exactly four allocated GPUs" >&2
|
||||
exit 1
|
||||
fi
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne 4 ]]; then
|
||||
echo "ERROR: expected four allocated GPUs, got CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "LAUNCH_ECHO host=$(hostname) gpus=${CUDA_VISIBLE_DEVICES} model=${MODEL_ROOT} frontier=d9cfeb6 vllm=community-0.10.2 transformers=4.55.2 backend=FLASHINFER execution=eager kv=BF16 spec=off tasks=representative-FP8-linear/attention/MoE+TP4-allreduce+TP4-model-load+one-request skip_linear=${SKIP_LINEAR} serving_only=${SERVING_ONLY} hard_wall_cap=1800s hard_gpu_cap=2_H20h"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
|
||||
test -x "${VENV_ROOT}/bin/python"
|
||||
test -f "${FRONTIER_ROOT}/pyproject.toml"
|
||||
test -f "${MODEL_ROOT}/config.json"
|
||||
|
||||
export PYTHONPATH="${FRONTIER_ROOT}${PYTHONPATH:+:${PYTHONPATH}}"
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export VLLM_USE_V1=1
|
||||
export VLLM_ATTENTION_BACKEND=FLASHINFER
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
|
||||
cd "${FRONTIER_ROOT}"
|
||||
|
||||
if [[ "${SERVING_ONLY}" -eq 0 && "${SKIP_LINEAR}" -eq 0 ]]; then
|
||||
echo "STAGE linear_op"
|
||||
timeout 300 "${VENV_ROOT}/bin/python" -m frontier.profiling.linear_op.main \
|
||||
--disable_ray \
|
||||
--models Qwen3-235B-A22B-FP8 \
|
||||
--num_gpus 1 \
|
||||
--max_tokens 16 \
|
||||
--num_tokens_list 16 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--profile_method cuda_event \
|
||||
--device h20 \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--is_moe \
|
||||
--yes
|
||||
elif [[ "${SERVING_ONLY}" -eq 0 ]]; then
|
||||
echo "STAGE linear_op SKIPPED (existing artifact retained)"
|
||||
fi
|
||||
|
||||
if [[ "${SERVING_ONLY}" -eq 0 ]]; then
|
||||
echo "STAGE attention"
|
||||
timeout 300 "${VENV_ROOT}/bin/python" -m frontier.profiling.attention.main \
|
||||
--disable_ray \
|
||||
--models Qwen3-235B-A22B-FP8 \
|
||||
--num_gpus 1 \
|
||||
--max_model_len 40960 \
|
||||
--max_seq_len 128 \
|
||||
--min_batch_size 1 \
|
||||
--max_batch_size 1 \
|
||||
--batch_size_list 1 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--max_pipeline_parallel_size 1 \
|
||||
--attention_backend FLASHINFER \
|
||||
--block_size 16 \
|
||||
--profile_only_prefill \
|
||||
--fixed_chunked_prefill_size 128 \
|
||||
--device h20 \
|
||||
--profile_method cuda_event \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--yes
|
||||
|
||||
echo "STAGE moe"
|
||||
timeout 300 "${VENV_ROOT}/bin/python" -m frontier.profiling.moe.main \
|
||||
--disable_ray \
|
||||
--models Qwen3-235B-A22B-FP8 \
|
||||
--device h20 \
|
||||
--num_gpus 1 \
|
||||
--max_tokens 16 \
|
||||
--num_tokens_list 16 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--expert_parallel_sizes 1 \
|
||||
--load_distributions uniform \
|
||||
--num_samples_per_distribution 1 \
|
||||
--routing_runtime_path standard_fused_topk \
|
||||
--gating_runtime_context prefill_hot \
|
||||
--profile_method cuda_event \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--yes
|
||||
|
||||
echo "STAGE allreduce"
|
||||
timeout 180 "${VENV_ROOT}/bin/torchrun" \
|
||||
--standalone \
|
||||
--nnodes=1 \
|
||||
--nproc-per-node=4 \
|
||||
"${OUTPUT_ROOT}/scripts/allreduce_smoke.py" \
|
||||
| tee "${RESULT_DIR}/allreduce_tp4.jsonl"
|
||||
else
|
||||
echo "STAGES Frontier profiles and allreduce SKIPPED (existing artifacts retained)"
|
||||
fi
|
||||
|
||||
echo "STAGE serving"
|
||||
setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
|
||||
--host 127.0.0.1 \
|
||||
--port "${SERVER_PORT}" \
|
||||
--served-model-name "${SERVED_MODEL}" \
|
||||
--tensor-parallel-size 4 \
|
||||
--disable-custom-all-reduce \
|
||||
--quantization fp8 \
|
||||
--gpu-memory-utilization 0.80 \
|
||||
--kv-cache-dtype auto \
|
||||
--max-model-len 40960 \
|
||||
--max-num-batched-tokens 8192 \
|
||||
--max-num-seqs 64 \
|
||||
--no-enable-prefix-caching \
|
||||
--enable-chunked-prefill \
|
||||
--enforce-eager \
|
||||
--disable-log-requests \
|
||||
> "${LOG_DIR}/server.log" 2>&1 &
|
||||
SERVER_PID=$!
|
||||
|
||||
READY=0
|
||||
for _ in $(seq 1 180); do
|
||||
if curl -fsS --max-time 2 "http://127.0.0.1:${SERVER_PORT}/v1/models" \
|
||||
> "${RESULT_DIR}/models.json" 2>/dev/null; then
|
||||
READY=1
|
||||
break
|
||||
fi
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
echo "ERROR: vLLM server exited before readiness" >&2
|
||||
tail -200 "${LOG_DIR}/server.log" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
if [[ "${READY}" -ne 1 ]]; then
|
||||
echo "ERROR: vLLM server did not become ready within 900 seconds" >&2
|
||||
tail -200 "${LOG_DIR}/server.log" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
curl -fsS --max-time 120 \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"qwen3-235b-community-smoke","prompt":"Hello","max_tokens":1,"temperature":0}' \
|
||||
"http://127.0.0.1:${SERVER_PORT}/v1/completions" \
|
||||
| tee "${RESULT_DIR}/one_request.json"
|
||||
echo
|
||||
jq -e '.choices | length == 1' "${RESULT_DIR}/one_request.json" >/dev/null
|
||||
|
||||
cleanup
|
||||
SERVER_PID=""
|
||||
|
||||
find "${PROFILE_ROOT}" -type f -maxdepth 5 -print -exec sha256sum {} \;
|
||||
sha256sum \
|
||||
"${RESULT_DIR}/allreduce_tp4.jsonl" \
|
||||
"${RESULT_DIR}/models.json" \
|
||||
"${RESULT_DIR}/one_request.json" \
|
||||
> "${RESULT_DIR}/results.sha256"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "GPU_SMOKE_COMPLETE"
|
||||
66
runs/frontier-multicase-sufficiency-v0/smoke/setup_env.sh
Executable file
66
runs/frontier-multicase-sufficiency-v0/smoke/setup_env.sh
Executable file
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-${OUTPUT_ROOT}/Frontier-d9cfeb6}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
|
||||
export UV_HTTP_TIMEOUT="${UV_HTTP_TIMEOUT:-300}"
|
||||
|
||||
mkdir -p "${LOG_DIR}" "${OUTPUT_ROOT}/provenance"
|
||||
exec > >(tee -a "${LOG_DIR}/setup_env.log") 2>&1
|
||||
|
||||
echo "SETUP_ENV output=${OUTPUT_ROOT} frontier=${FRONTIER_ROOT} venv=${VENV_ROOT} python=/usr/local/bin/python3.12 vllm=0.10.2 transformers=4.55.2"
|
||||
|
||||
if [[ ! -f "${FRONTIER_ROOT}/pyproject.toml" ]]; then
|
||||
echo "ERROR: Frontier source is missing at ${FRONTIER_ROOT}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
uv venv --clear --python /usr/local/bin/python3.12 "${VENV_ROOT}"
|
||||
uv pip install \
|
||||
--python "${VENV_ROOT}/bin/python" \
|
||||
"vllm==0.10.2" \
|
||||
"transformers==4.55.2" \
|
||||
"flashinfer-python>=0.3,<0.4" \
|
||||
-e "${FRONTIER_ROOT}[test]"
|
||||
|
||||
"${VENV_ROOT}/bin/python" - <<'PY' | tee "${OUTPUT_ROOT}/provenance/environment.json"
|
||||
import importlib.metadata as metadata
|
||||
import json
|
||||
import platform
|
||||
|
||||
import flashinfer
|
||||
import frontier
|
||||
import torch
|
||||
import vllm
|
||||
|
||||
record = {
|
||||
"python": platform.python_version(),
|
||||
"torch": torch.__version__,
|
||||
"torch_cuda": torch.version.cuda,
|
||||
"vllm_import_version": vllm.__version__,
|
||||
"vllm_metadata_version": metadata.version("vllm"),
|
||||
"vllm_path": vllm.__file__,
|
||||
"transformers_metadata_version": metadata.version("transformers"),
|
||||
"tokenizers_metadata_version": metadata.version("tokenizers"),
|
||||
"flashinfer_metadata_version": metadata.version("flashinfer-python"),
|
||||
"flashinfer_path": flashinfer.__file__,
|
||||
"frontier_metadata_version": metadata.version("frontier-simulator"),
|
||||
"frontier_path": frontier.__path__[0],
|
||||
}
|
||||
print(json.dumps(record, indent=2, sort_keys=True))
|
||||
assert record["vllm_import_version"] == "0.10.2", record
|
||||
assert record["vllm_metadata_version"] == "0.10.2", record
|
||||
assert record["transformers_metadata_version"] == "4.55.2", record
|
||||
PY
|
||||
|
||||
uv pip freeze --python "${VENV_ROOT}/bin/python" > "${OUTPUT_ROOT}/provenance/requirements.freeze.txt"
|
||||
sha256sum \
|
||||
"${OUTPUT_ROOT}/provenance/environment.json" \
|
||||
"${OUTPUT_ROOT}/provenance/requirements.freeze.txt" \
|
||||
> "${OUTPUT_ROOT}/provenance/environment.sha256"
|
||||
|
||||
echo "SETUP_ENV_COMPLETE"
|
||||
@@ -0,0 +1,71 @@
|
||||
import importlib.util
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
MODULE_PATH = Path(__file__).with_name("audit_ground_truth.py")
|
||||
SPEC = importlib.util.spec_from_file_location("audit_ground_truth", MODULE_PATH)
|
||||
audit = importlib.util.module_from_spec(SPEC)
|
||||
assert SPEC.loader is not None
|
||||
SPEC.loader.exec_module(audit)
|
||||
|
||||
|
||||
class AuditGroundTruthTest(unittest.TestCase):
|
||||
def test_case_summary_exposes_tied_top_set(self):
|
||||
rows = [
|
||||
{
|
||||
"cell_id": "a",
|
||||
"score_req_s_per_gpu": 2.0,
|
||||
"capacity_lower_bound_req_s_per_gpu": 2.0,
|
||||
"capacity_upper_bound_req_s_per_gpu": 2.1,
|
||||
"fully_valid": True,
|
||||
},
|
||||
{
|
||||
"cell_id": "b",
|
||||
"score_req_s_per_gpu": 2.0,
|
||||
"capacity_lower_bound_req_s_per_gpu": 2.0,
|
||||
"capacity_upper_bound_req_s_per_gpu": 2.2,
|
||||
"fully_valid": True,
|
||||
},
|
||||
{
|
||||
"cell_id": "c",
|
||||
"score_req_s_per_gpu": 1.0,
|
||||
"capacity_lower_bound_req_s_per_gpu": 1.0,
|
||||
"capacity_upper_bound_req_s_per_gpu": 1.5,
|
||||
"fully_valid": False,
|
||||
},
|
||||
]
|
||||
|
||||
summary = audit.summarize_case("test", rows)
|
||||
|
||||
self.assertEqual(summary["top_set"], ["a", "b"])
|
||||
self.assertEqual(summary["distinct_score_count"], 2)
|
||||
self.assertEqual(summary["tied_pair_count"], 1)
|
||||
self.assertEqual(summary["informative_pair_count"], 2)
|
||||
self.assertAlmostEqual(summary["random_top_set_hit_rate"], 2 / 3)
|
||||
self.assertEqual(summary["invalid_cells"], ["c"])
|
||||
self.assertEqual(
|
||||
summary["possibly_optimal_set_from_search_brackets"], ["a", "b"]
|
||||
)
|
||||
|
||||
def test_config_gpu_count_includes_data_parallelism(self):
|
||||
result = {
|
||||
"config_patch": {
|
||||
"flag_patch": {
|
||||
"tensor-parallel-size": 2,
|
||||
"data-parallel-size": 4,
|
||||
"expert-parallel-size": 8,
|
||||
"max-num-seqs": 128,
|
||||
"max-num-batched-tokens": 384,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
config = audit.config_from_result(result)
|
||||
|
||||
self.assertEqual(config["gpu_count"], 8)
|
||||
self.assertEqual(audit.cell_id(config), "tp2_dp4_ep8_mns128_mbt384")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
321
runs/frontier-multicase-sufficiency-v1/analyze_t0_surface.py
Normal file
321
runs/frontier-multicase-sufficiency-v1/analyze_t0_surface.py
Normal file
@@ -0,0 +1,321 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate and compare frozen T0 Frontier and two-round real surfaces."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
SLOS = ("tpot_40ms", "tpot_120ms", "tpot_150ms", "tpot_180ms")
|
||||
RATE_LATTICE = (0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20)
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
|
||||
|
||||
def rate_key(rate: float) -> str:
|
||||
return f"r{rate:.2f}".replace(".", "p")
|
||||
|
||||
|
||||
def find_real_result(roots: list[Path], name: str, round_id: int, rate: float) -> Path:
|
||||
relative = Path(name) / f"round{round_id}/results" / f"{rate_key(rate)}.json"
|
||||
matches = [root / relative for root in roots if (root / relative).is_file()]
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f"expected one real result for {relative}, got {matches}")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def capacity(loads: list[dict[str, Any]], slo: str, field: str) -> float | None:
|
||||
values = [float(load["rate"]) for load in loads if bool(load[field][slo]["feasible"])]
|
||||
return max(values) if values else None
|
||||
|
||||
|
||||
def real_boundary_status(loads: list[dict[str, Any]], slo: str) -> str:
|
||||
labels = {
|
||||
float(load["rate"]): bool(load["real_conservative"][slo]["feasible"])
|
||||
for load in loads
|
||||
}
|
||||
ordered = [(rate, labels[rate]) for rate in RATE_LATTICE if rate in labels]
|
||||
if any(not left and right for (_, left), (_, right) in zip(ordered, ordered[1:])):
|
||||
return "non_monotonic_requires_full_lattice"
|
||||
if len(labels) == len(RATE_LATTICE):
|
||||
return "complete_lattice"
|
||||
for lower, upper in zip(RATE_LATTICE, RATE_LATTICE[1:]):
|
||||
if labels.get(lower) is True and labels.get(upper) is False:
|
||||
return "adjacent_transition_bracketed"
|
||||
if labels.get(RATE_LATTICE[-1]) is True:
|
||||
return "upper_lattice_reached"
|
||||
if labels.get(RATE_LATTICE[0]) is False and not any(labels.values()):
|
||||
return "lowest_anchor_infeasible"
|
||||
return "unbracketed_requires_expansion"
|
||||
|
||||
|
||||
def kendall_tau_b(left: list[float], right: list[float]) -> float | None:
|
||||
concordant = discordant = left_ties = right_ties = 0
|
||||
for i in range(len(left)):
|
||||
for j in range(i + 1, len(left)):
|
||||
x = (left[i] > left[j]) - (left[i] < left[j])
|
||||
y = (right[i] > right[j]) - (right[i] < right[j])
|
||||
if x == 0 and y == 0:
|
||||
continue
|
||||
if x == 0:
|
||||
left_ties += 1
|
||||
elif y == 0:
|
||||
right_ties += 1
|
||||
elif x == y:
|
||||
concordant += 1
|
||||
else:
|
||||
discordant += 1
|
||||
denominator = math.sqrt(
|
||||
(concordant + discordant + left_ties)
|
||||
* (concordant + discordant + right_ties)
|
||||
)
|
||||
return (concordant - discordant) / denominator if denominator else None
|
||||
|
||||
|
||||
def sign(left: float, right: float) -> int:
|
||||
return (left > right) - (left < right)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-freeze", type=Path, required=True)
|
||||
parser.add_argument("--real-plan", type=Path, required=True)
|
||||
parser.add_argument("--real-root", type=Path, action="append", required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
freeze = json.loads(args.frontier_freeze.read_text())
|
||||
plan = json.loads(args.real_plan.read_text())
|
||||
real_roots = [root.resolve() for root in args.real_root]
|
||||
if freeze.get("status") != "frozen_before_real_surface" or len(freeze.get("config_results") or []) != 8:
|
||||
raise ValueError("Frontier freeze is incomplete")
|
||||
if plan.get("frontier_freeze", {}).get("sha256") != sha256(args.frontier_freeze):
|
||||
raise ValueError("real plan does not point to this Frontier freeze")
|
||||
sim_by_name = {item["config"]["name"]: item for item in freeze["config_results"]}
|
||||
|
||||
cells = []
|
||||
for cell in plan["cells"]:
|
||||
config = cell["config"]
|
||||
name = config["name"]
|
||||
sim_loads = {float(load["offered_request_rate"]): load for load in sim_by_name[name]["loads"]}
|
||||
loads = []
|
||||
for rate in cell["rates"]:
|
||||
round_summaries = []
|
||||
files = []
|
||||
for round_id in (1, 2):
|
||||
path = find_real_result(real_roots, name, round_id, float(rate))
|
||||
payload = json.loads(path.read_text())
|
||||
if payload.get("schema") != "qwen235b-t0-rate-anchor-v1":
|
||||
raise ValueError(f"bad real result schema: {path}")
|
||||
if payload["summary"]["completed"] != 64 or payload["summary"]["failed"] != 0:
|
||||
raise ValueError(f"incomplete real anchor: {path}")
|
||||
if float(payload["workload"]["offered_request_rate"]) != float(rate):
|
||||
raise ValueError(f"offered-rate drift: {path}")
|
||||
round_summaries.append(payload["summary"]["slos"])
|
||||
files.append({"path": str(path), "sha256": sha256(path)})
|
||||
conservative = {
|
||||
slo: {
|
||||
"feasible": all(summary[slo]["feasible"] for summary in round_summaries),
|
||||
"round_pass_rates": [summary[slo]["pass_rate"] for summary in round_summaries],
|
||||
}
|
||||
for slo in SLOS
|
||||
}
|
||||
loads.append(
|
||||
{
|
||||
"rate": float(rate),
|
||||
"real_conservative": conservative,
|
||||
"sim": sim_loads[float(rate)]["slos"],
|
||||
"real_files": files,
|
||||
}
|
||||
)
|
||||
cells.append({"config": config, "loads": loads})
|
||||
|
||||
comparisons = {}
|
||||
for slo in SLOS:
|
||||
records = []
|
||||
for cell in cells:
|
||||
real = capacity(cell["loads"], slo, "real_conservative")
|
||||
sim_values = [
|
||||
float(load["offered_request_rate"])
|
||||
for load in sim_by_name[cell["config"]["name"]]["loads"]
|
||||
if bool(load["slos"][slo]["feasible"])
|
||||
]
|
||||
sim = max(sim_values) if sim_values else None
|
||||
tp = int(cell["config"]["tp"])
|
||||
boundary = real_boundary_status(cell["loads"], slo)
|
||||
records.append(
|
||||
{
|
||||
"config": cell["config"],
|
||||
"real_capacity": real,
|
||||
"sim_capacity": sim,
|
||||
"real_capacity_per_gpu": real / tp if real is not None else None,
|
||||
"sim_capacity_per_gpu": sim / tp if sim is not None else None,
|
||||
"real_boundary_status": boundary,
|
||||
"expansion_required": boundary in {
|
||||
"non_monotonic_requires_full_lattice",
|
||||
"unbracketed_requires_expansion",
|
||||
},
|
||||
}
|
||||
)
|
||||
rankable = [
|
||||
row
|
||||
for row in records
|
||||
if not row["expansion_required"]
|
||||
and row["real_capacity_per_gpu"] is not None
|
||||
and row["sim_capacity_per_gpu"] is not None
|
||||
]
|
||||
tau = kendall_tau_b(
|
||||
[row["real_capacity_per_gpu"] for row in rankable],
|
||||
[row["sim_capacity_per_gpu"] for row in rankable],
|
||||
)
|
||||
pairwise = {
|
||||
"all_pairs": 0,
|
||||
"exact_sign_matches": 0,
|
||||
"real_non_tie_pairs": 0,
|
||||
"real_non_tie_direction_matches": 0,
|
||||
}
|
||||
for i, left in enumerate(rankable):
|
||||
for right in rankable[i + 1 :]:
|
||||
real_sign = sign(left["real_capacity_per_gpu"], right["real_capacity_per_gpu"])
|
||||
sim_sign = sign(left["sim_capacity_per_gpu"], right["sim_capacity_per_gpu"])
|
||||
pairwise["all_pairs"] += 1
|
||||
pairwise["exact_sign_matches"] += real_sign == sim_sign
|
||||
if real_sign:
|
||||
pairwise["real_non_tie_pairs"] += 1
|
||||
pairwise["real_non_tie_direction_matches"] += real_sign == sim_sign
|
||||
real_best = max((row["real_capacity_per_gpu"] for row in records if row["real_capacity_per_gpu"] is not None), default=None)
|
||||
sim_best = max((row["sim_capacity_per_gpu"] for row in records if row["sim_capacity_per_gpu"] is not None), default=None)
|
||||
sim_top = [row for row in records if sim_best is not None and row["sim_capacity_per_gpu"] == sim_best]
|
||||
real_top = [row for row in records if real_best is not None and row["real_capacity_per_gpu"] == real_best]
|
||||
optimistic_regret = worst_regret = None
|
||||
if real_best is not None and sim_top and all(row["real_capacity_per_gpu"] is not None for row in sim_top):
|
||||
regrets = [(real_best - row["real_capacity_per_gpu"]) / real_best for row in sim_top]
|
||||
optimistic_regret = min(regrets)
|
||||
worst_regret = max(regrets)
|
||||
confusion = {"anchors": 0, "match": 0, "false_feasible": 0, "false_infeasible": 0}
|
||||
for cell in cells:
|
||||
for load in cell["loads"]:
|
||||
real_feasible = bool(load["real_conservative"][slo]["feasible"])
|
||||
sim_feasible = bool(load["sim"][slo]["feasible"])
|
||||
confusion["anchors"] += 1
|
||||
confusion["match"] += real_feasible == sim_feasible
|
||||
confusion["false_feasible"] += sim_feasible and not real_feasible
|
||||
confusion["false_infeasible"] += real_feasible and not sim_feasible
|
||||
comparisons[slo] = {
|
||||
"records": records,
|
||||
"kendall_tau_b": tau,
|
||||
"pairwise": pairwise,
|
||||
"anchor_confusion": confusion,
|
||||
"real_top_set": [row["config"]["name"] for row in real_top],
|
||||
"sim_top_set": [row["config"]["name"] for row in sim_top],
|
||||
"top_set_intersection": sorted(
|
||||
{row["config"]["name"] for row in real_top}
|
||||
& {row["config"]["name"] for row in sim_top}
|
||||
),
|
||||
"top_set_exact_match": {
|
||||
row["config"]["name"] for row in real_top
|
||||
} == {row["config"]["name"] for row in sim_top},
|
||||
"optimistic_regret": optimistic_regret,
|
||||
"worst_tie_break_regret": worst_regret,
|
||||
}
|
||||
|
||||
run_costs = []
|
||||
for root in real_roots:
|
||||
config_names = [child.name for child in root.iterdir() if child.is_dir() and child.name in sim_by_name]
|
||||
if len(config_names) != 1:
|
||||
raise ValueError(f"expected one config directory in real root {root}, got {config_names}")
|
||||
config = sim_by_name[config_names[0]]["config"]
|
||||
remote_run = root.parents[2] / "remote_run"
|
||||
started = datetime.fromisoformat((remote_run / "started_at").read_text().strip())
|
||||
finished = datetime.fromisoformat((remote_run / "finished_at").read_text().strip())
|
||||
wall_seconds = (finished - started).total_seconds()
|
||||
run_costs.append(
|
||||
{
|
||||
"run_id": root.parents[2].name,
|
||||
"config": config_names[0],
|
||||
"gpu_count": int(config["tp"]),
|
||||
"wall_seconds": wall_seconds,
|
||||
"h20_gpu_hours": wall_seconds * int(config["tp"]) / 3600,
|
||||
"started_at": started.isoformat(),
|
||||
"finished_at": finished.isoformat(),
|
||||
}
|
||||
)
|
||||
fresh_server_anchors = 2 * sum(len(cell["loads"]) for cell in cells)
|
||||
real_execution_cost = {
|
||||
"accepted_fleet_jobs": len(run_costs),
|
||||
"fresh_server_anchors": fresh_server_anchors,
|
||||
"measured_requests": fresh_server_anchors * 64,
|
||||
"warmup_requests": 2
|
||||
* sum(
|
||||
min(32, max(4, math.ceil(float(load["rate"]) * 20)))
|
||||
for cell in cells
|
||||
for load in cell["loads"]
|
||||
),
|
||||
"accepted_h20_gpu_hours": sum(run["h20_gpu_hours"] for run in run_costs),
|
||||
"campaign_wall_span_seconds": (
|
||||
max(datetime.fromisoformat(run["finished_at"]) for run in run_costs)
|
||||
- min(datetime.fromisoformat(run["started_at"]) for run in run_costs)
|
||||
).total_seconds(),
|
||||
"runs": run_costs,
|
||||
}
|
||||
|
||||
output = {
|
||||
"schema": "qwen235b-t0-simulator-real-comparison-v1",
|
||||
"frontier_freeze_sha256": sha256(args.frontier_freeze),
|
||||
"real_plan_sha256": sha256(args.real_plan),
|
||||
"real_execution_cost": real_execution_cost,
|
||||
"cells": cells,
|
||||
"comparisons": comparisons,
|
||||
}
|
||||
args.output_root.mkdir(parents=True, exist_ok=True)
|
||||
write_json(args.output_root / "comparison.json", output)
|
||||
with (args.output_root / "capacity.csv").open("w", newline="") as target:
|
||||
writer = csv.DictWriter(
|
||||
target,
|
||||
fieldnames=[
|
||||
"slo",
|
||||
"config",
|
||||
"tp",
|
||||
"mns",
|
||||
"mbt",
|
||||
"real_capacity_per_gpu",
|
||||
"sim_capacity_per_gpu",
|
||||
"real_boundary_status",
|
||||
"expansion_required",
|
||||
],
|
||||
)
|
||||
writer.writeheader()
|
||||
for slo, comparison in comparisons.items():
|
||||
for row in comparison["records"]:
|
||||
writer.writerow(
|
||||
{
|
||||
"slo": slo,
|
||||
"config": row["config"]["name"],
|
||||
"tp": row["config"]["tp"],
|
||||
"mns": row["config"]["mns"],
|
||||
"mbt": row["config"]["mbt"],
|
||||
"real_capacity_per_gpu": row["real_capacity_per_gpu"],
|
||||
"sim_capacity_per_gpu": row["sim_capacity_per_gpu"],
|
||||
"real_boundary_status": row["real_boundary_status"],
|
||||
"expansion_required": row["expansion_required"],
|
||||
}
|
||||
)
|
||||
print(args.output_root / "comparison.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,61 @@
|
||||
{
|
||||
"schema": "qwen235b-t0-real-exclusions-v1",
|
||||
"policy": "Exclude the complete output directory whenever another process shares any allocated GPU or benchmark endpoint during warmup or a measured anchor.",
|
||||
"excluded_attempts": [
|
||||
{
|
||||
"reason": "Fleet monitor re-probed during model load and oversubscribed both TP4 allocations; the later MNS128 attempts overlapped the MNS64 warmup and start of r0p10.",
|
||||
"remote_quarantine": "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/contaminated-queue-race-20260716T045500Z",
|
||||
"run_ids": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T045338281933Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T045339503446Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1-20260716T045458846499Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1-20260716T045500253803Z"
|
||||
]
|
||||
},
|
||||
{
|
||||
"reason": "A dispatch command still copying the source was mistakenly resubmitted. The duplicate controllers reused the first pair's open ports and issued concurrent warmup requests, so both the original and duplicate outputs were rejected.",
|
||||
"remote_quarantine": "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/contaminated-duplicate-dispatch-20260716T0501Z",
|
||||
"run_ids": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T050017477963Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T050018748070Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T050123485776Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T050124800483Z"
|
||||
]
|
||||
},
|
||||
{
|
||||
"reason": "Clean diagnostic run exposed cross-anchor execution-state leakage: TPOT at 1.60 req/s failed after a sustained 0.10 anchor but passed after a 2.40 anchor. The multi-rate-per-server contract does not match Frontier's independent rate runs, so the complete attempt is diagnostic-only.",
|
||||
"remote_quarantine": "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/diagnostic-cross-anchor-state-20260716T0504Z",
|
||||
"run_ids": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T050405703312Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T050407104293Z"
|
||||
]
|
||||
}
|
||||
],
|
||||
"accepted_attempts": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T054640909739Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T054642115787Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-expansion-r3p20-20260716-v1-20260716T062904775741Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-expansion-r3p20-20260716-v1-20260716T062906244819Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1-20260716T063826749477Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1-20260716T063827805737Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt8192-expansion-r3p20-20260716-v1-20260716T072121119478Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt16384-expansion-r3p20-20260716-v1-20260716T072122501851Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1-20260716T073151443129Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r2p40-20260716-v1-20260716T081536141326Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r3p20-20260716-v1-20260716T082603837807Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt16384-20260716-v1-20260716T083632620015Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt16384-expansion-r2p40-20260716-v1-20260716T092040851202Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt8192-20260716-v1-20260716T093245831612Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt8192-expansion-r2p40-20260716-v1-20260716T101734683241Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt16384-20260716-v1-20260716T102855604708Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r2p40-20260716-v1-20260716T111437770615Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r3p20-20260716-v1-20260716T112608807037Z"
|
||||
],
|
||||
"accepted_warnings": [
|
||||
{
|
||||
"run_id": "qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1-20260716T073151443129Z",
|
||||
"anchor": "round1/r1p20",
|
||||
"reason": "One rank-7 TCPStore heartbeat warning occurred 1.1 s after the complete 64-request result was written, while all eight workers were terminating after the runner intentionally stopped the fresh server. The application then shut down normally; no request, engine, OOM, or in-window NCCL failure occurred."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,299 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_protocol_amendment": {
|
||||
"contract": "fresh server and target-rate warmup for every config-rate-round",
|
||||
"reason": "observed cross-anchor GPU/kernel/batch warm-state leakage",
|
||||
"timing": "after excluded multi-rate-per-server diagnostic and before any accepted real surface cell"
|
||||
},
|
||||
"expansion_artifact_roots": [
|
||||
"artifacts/t0-real-expansion-r2p40-v1",
|
||||
"artifacts/t0-real-expansion-r3p20-v1"
|
||||
],
|
||||
"expected_total_h20_gpu_hours": 39.98888888888888,
|
||||
"frontier_freeze": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/frontier-t0-surface-v1/frontier_surface_frozen.json",
|
||||
"sha256": "801aa36451c8647f71cc87011144622d2203786e82f189ed1375d964399b106a"
|
||||
},
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
"post_pilot_sensitivities": [
|
||||
"tpot_120ms",
|
||||
"tpot_150ms",
|
||||
"tpot_180ms"
|
||||
],
|
||||
"preregistered_plan": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/t0-real-surface-v1/real-plan.json",
|
||||
"sha256": "f7acb45d1183a6026dba4bbf9bc2d7578452b815b8de0035ca732512f3d9d47a"
|
||||
},
|
||||
"schema": "qwen235b-t0-real-executed-plan-v1",
|
||||
"selection_slo": "tpot_150ms",
|
||||
"selection_timing": "rate anchors frozen after complete simulator surface and before any accepted real surface cell",
|
||||
"status": "complete",
|
||||
"strict_preregistered_slo": "tpot_40ms"
|
||||
}
|
||||
@@ -0,0 +1,187 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
}
|
||||
],
|
||||
"expected_total_h20_gpu_hours": 27.016666666666666,
|
||||
"frontier_freeze": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/frontier-t0-surface-v1/frontier_surface_frozen.json",
|
||||
"sha256": "801aa36451c8647f71cc87011144622d2203786e82f189ed1375d964399b106a"
|
||||
},
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
"post_pilot_sensitivities": [
|
||||
"tpot_120ms",
|
||||
"tpot_150ms",
|
||||
"tpot_180ms"
|
||||
],
|
||||
"schema": "qwen235b-t0-real-plan-v1",
|
||||
"selection_slo": "tpot_150ms",
|
||||
"selection_timing": "after_complete_simulator_freeze_before_any_real_surface_cell",
|
||||
"strict_preregistered_slo": "tpot_40ms"
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_protocol_amendment": {
|
||||
"contract": "fresh server and target-rate warmup for every config-rate-round",
|
||||
"reason": "observed cross-anchor GPU/kernel/batch warm-state leakage",
|
||||
"timing": "after excluded multi-rate-per-server diagnostic and before any accepted real surface cell"
|
||||
},
|
||||
"expected_total_h20_gpu_hours": 39.98888888888888,
|
||||
"frontier_freeze": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/frontier-t0-surface-v1/frontier_surface_frozen.json",
|
||||
"sha256": "801aa36451c8647f71cc87011144622d2203786e82f189ed1375d964399b106a"
|
||||
},
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
"post_pilot_sensitivities": [
|
||||
"tpot_120ms",
|
||||
"tpot_150ms",
|
||||
"tpot_180ms"
|
||||
],
|
||||
"schema": "qwen235b-t0-real-plan-v1",
|
||||
"selection_slo": "tpot_150ms",
|
||||
"selection_timing": "rate anchors frozen after complete simulator surface and before any accepted real surface cell",
|
||||
"strict_preregistered_slo": "tpot_40ms"
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Create an immutable Frontier profile root with measured decode rows."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODEL = "Qwen3-235B-A22B-FP8"
|
||||
ATTENTION_RELATIVE_PATH = Path("compute/h20") / MODEL / "attention.csv"
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--base-profile-root", type=Path, required=True)
|
||||
parser.add_argument("--decode-attention-csv", type=Path, required=True)
|
||||
parser.add_argument("--true-mixed-attention-csv", type=Path)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def read_csv(path: Path) -> tuple[list[str], list[dict[str, str]]]:
|
||||
with path.open(newline="") as source:
|
||||
reader = csv.DictReader(source)
|
||||
if reader.fieldnames is None:
|
||||
raise ValueError(f"missing CSV header: {path}")
|
||||
return reader.fieldnames, list(reader)
|
||||
|
||||
|
||||
def is_true(value: str) -> bool:
|
||||
return value.strip().lower() == "true"
|
||||
|
||||
|
||||
def profile_hashes(root: Path) -> dict[str, str]:
|
||||
return {
|
||||
str(path.relative_to(root)): sha256(path)
|
||||
for path in sorted(root.rglob("*"))
|
||||
if path.is_file() and path.name != "profile_closure_manifest.json"
|
||||
}
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
base_root = args.base_profile_root.resolve()
|
||||
decode_csv = args.decode_attention_csv.resolve()
|
||||
output_root = args.output_root.resolve()
|
||||
base_attention = base_root / ATTENTION_RELATIVE_PATH
|
||||
|
||||
if output_root.exists():
|
||||
raise FileExistsError(f"refusing to overwrite profile root: {output_root}")
|
||||
for path in (base_attention, decode_csv):
|
||||
if not path.is_file():
|
||||
raise FileNotFoundError(path)
|
||||
|
||||
base_fields, base_rows = read_csv(base_attention)
|
||||
decode_fields, decode_source_rows = read_csv(decode_csv)
|
||||
if base_fields != decode_fields:
|
||||
raise ValueError("base and decode attention CSV schemas differ")
|
||||
if not base_rows or any(not is_true(row["is_prefill"]) for row in base_rows):
|
||||
raise ValueError("base attention profile must contain only prefill rows")
|
||||
decode_rows = [row for row in decode_source_rows if not is_true(row["is_prefill"])]
|
||||
if not decode_rows:
|
||||
raise ValueError("decode attention profile contains no decode rows")
|
||||
if any(not row["time_stats.attn_decode.median"] for row in decode_rows):
|
||||
raise ValueError("decode attention profile has an empty median")
|
||||
|
||||
dimensions = {
|
||||
(
|
||||
int(row["num_tensor_parallel_workers"]),
|
||||
int(row["batch_size"]),
|
||||
int(row["kv_cache_size"]),
|
||||
row["attention_backend"],
|
||||
)
|
||||
for row in decode_rows
|
||||
}
|
||||
if len(dimensions) != len(decode_rows):
|
||||
raise ValueError("decode attention profile has duplicate coverage coordinates")
|
||||
if any(dimension[-1] != "FLASHINFER" for dimension in dimensions):
|
||||
raise ValueError("decode attention profile is not entirely FlashInfer")
|
||||
|
||||
true_mixed_csv = (
|
||||
args.true_mixed_attention_csv.resolve()
|
||||
if args.true_mixed_attention_csv is not None
|
||||
else None
|
||||
)
|
||||
true_mixed_rows: list[dict[str, str]] = []
|
||||
output_fields = list(base_fields)
|
||||
if true_mixed_csv is not None:
|
||||
if not true_mixed_csv.is_file():
|
||||
raise FileNotFoundError(true_mixed_csv)
|
||||
true_mixed_fields, true_mixed_rows = read_csv(true_mixed_csv)
|
||||
required = {
|
||||
"is_true_mixed_batch",
|
||||
"decode_batch_size",
|
||||
"decode_avg_kv_cache_size",
|
||||
"num_prefill_seqs",
|
||||
"time_stats.attn_decode.median",
|
||||
"time_stats.attn_prefill.median",
|
||||
}
|
||||
missing = required - set(true_mixed_fields)
|
||||
if missing:
|
||||
raise ValueError(f"true-mixed attention CSV lacks columns: {sorted(missing)}")
|
||||
if not true_mixed_rows or any(
|
||||
not is_true(row["is_true_mixed_batch"]) for row in true_mixed_rows
|
||||
):
|
||||
raise ValueError("true-mixed attention CSV has non-mixed rows")
|
||||
if any(
|
||||
not row["time_stats.attn_decode.median"]
|
||||
or not row["time_stats.attn_prefill.median"]
|
||||
for row in true_mixed_rows
|
||||
):
|
||||
raise ValueError("true-mixed attention profile has an empty median")
|
||||
if {
|
||||
int(row["num_tensor_parallel_workers"]) for row in true_mixed_rows
|
||||
} != {4, 8}:
|
||||
raise ValueError("true-mixed attention profile must cover TP4 and TP8")
|
||||
output_fields.extend(
|
||||
field for field in true_mixed_fields if field not in output_fields
|
||||
)
|
||||
|
||||
for row in [*base_rows, *decode_rows]:
|
||||
if "is_true_mixed_batch" in output_fields:
|
||||
row["is_true_mixed_batch"] = "False"
|
||||
|
||||
output_root.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix=f".{output_root.name}.", dir=output_root.parent
|
||||
) as temporary:
|
||||
temporary_root = Path(temporary) / output_root.name
|
||||
shutil.copytree(base_root, temporary_root)
|
||||
merged_attention = temporary_root / ATTENTION_RELATIVE_PATH
|
||||
with merged_attention.open("w", newline="") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=output_fields, lineterminator="\n")
|
||||
writer.writeheader()
|
||||
writer.writerows([*base_rows, *decode_rows, *true_mixed_rows])
|
||||
|
||||
payload = {
|
||||
"schema": "frontier-profile-closure-v1",
|
||||
"model": MODEL,
|
||||
"base_profile_root": str(base_root),
|
||||
"base_attention_sha256": sha256(base_attention),
|
||||
"decode_attention_csv": str(decode_csv),
|
||||
"decode_attention_sha256": sha256(decode_csv),
|
||||
"base_prefill_rows": len(base_rows),
|
||||
"decode_source_rows": len(decode_source_rows),
|
||||
"added_decode_rows": len(decode_rows),
|
||||
"true_mixed_attention_csv": (
|
||||
str(true_mixed_csv) if true_mixed_csv is not None else None
|
||||
),
|
||||
"true_mixed_attention_sha256": (
|
||||
sha256(true_mixed_csv) if true_mixed_csv is not None else None
|
||||
),
|
||||
"added_true_mixed_rows": len(true_mixed_rows),
|
||||
"merged_attention_rows": (
|
||||
len(base_rows) + len(decode_rows) + len(true_mixed_rows)
|
||||
),
|
||||
"decode_dimensions": [list(values) for values in sorted(dimensions)],
|
||||
"output_files_sha256": profile_hashes(temporary_root),
|
||||
}
|
||||
write_json(temporary_root / "profile_closure_manifest.json", payload)
|
||||
temporary_root.rename(output_root)
|
||||
|
||||
print(json.dumps(payload, indent=2, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
183
runs/frontier-multicase-sufficiency-v1/audit_trace_contract.py
Normal file
183
runs/frontier-multicase-sufficiency-v1/audit_trace_contract.py
Normal file
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audit Qwen235B trace token lengths and source prefix-hash identities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import platform
|
||||
import socket
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--trace", type=Path, required=True)
|
||||
parser.add_argument("--model", type=Path, required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--max-model-len", type=int, default=40960)
|
||||
parser.add_argument("--source-block-size", type=int, default=64)
|
||||
parser.add_argument("--batch-size", type=int, default=16)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
if args.max_model_len <= 0 or args.source_block_size <= 0 or args.batch_size <= 0:
|
||||
raise ValueError("length and batch-size arguments must be positive")
|
||||
|
||||
import transformers
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
rows = [json.loads(line) for line in args.trace.open() if line.strip()]
|
||||
context_exceeded = [
|
||||
row
|
||||
for row in rows
|
||||
if int(row["input_length"]) + int(row["output_length"]) > args.max_model_len
|
||||
]
|
||||
zero_output = [row for row in rows if int(row["output_length"]) == 0]
|
||||
eligible = [
|
||||
row
|
||||
for row in rows
|
||||
if int(row["input_length"]) + int(row["output_length"]) <= args.max_model_len
|
||||
and int(row["output_length"]) > 0
|
||||
]
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
|
||||
length_mismatch_count = 0
|
||||
hash_count_mismatch_count = 0
|
||||
hash_to_key: dict[str, bytes] = {}
|
||||
key_to_hash: dict[bytes, str] = {}
|
||||
hash_to_key_conflict_count = 0
|
||||
key_to_hash_conflict_count = 0
|
||||
total_tokens = 0
|
||||
full_blocks = 0
|
||||
partial_blocks = 0
|
||||
length_digest = hashlib.sha256()
|
||||
token_digest = hashlib.sha256()
|
||||
started = time.time()
|
||||
|
||||
for start in range(0, len(eligible), args.batch_size):
|
||||
batch = eligible[start : start + args.batch_size]
|
||||
encoded = tokenizer(
|
||||
[row["prompt"] for row in batch],
|
||||
add_special_tokens=False,
|
||||
padding=False,
|
||||
truncation=False,
|
||||
)["input_ids"]
|
||||
for offset, (row, token_ids) in enumerate(zip(batch, encoded, strict=True)):
|
||||
row_index = start + offset
|
||||
actual_length = len(token_ids)
|
||||
expected_length = int(row["input_length"])
|
||||
total_tokens += actual_length
|
||||
length_digest.update(f"{row_index}:{actual_length}\n".encode())
|
||||
if actual_length != expected_length:
|
||||
length_mismatch_count += 1
|
||||
|
||||
source_hashes = row["hash_ids"]
|
||||
expected_hashes = math.ceil(actual_length / args.source_block_size)
|
||||
if len(source_hashes) != expected_hashes:
|
||||
hash_count_mismatch_count += 1
|
||||
continue
|
||||
|
||||
request_token_digest = hashlib.sha256()
|
||||
parent = b"ROOT"
|
||||
for block_index, source_hash in enumerate(source_hashes):
|
||||
begin = block_index * args.source_block_size
|
||||
chunk = token_ids[begin : begin + args.source_block_size]
|
||||
token_payload = b"".join(
|
||||
int(token_id).to_bytes(4, "little", signed=False)
|
||||
for token_id in chunk
|
||||
)
|
||||
request_token_digest.update(token_payload)
|
||||
chunk_digest = hashlib.blake2b(token_payload, digest_size=16).digest()
|
||||
key_digest = hashlib.blake2b(
|
||||
parent + b"\0" + chunk_digest, digest_size=16
|
||||
).digest()
|
||||
if len(chunk) == args.source_block_size:
|
||||
full_blocks += 1
|
||||
else:
|
||||
partial_blocks += 1
|
||||
|
||||
hash_id = str(source_hash)
|
||||
previous_key = hash_to_key.setdefault(hash_id, key_digest)
|
||||
if previous_key != key_digest:
|
||||
hash_to_key_conflict_count += 1
|
||||
previous_hash = key_to_hash.setdefault(key_digest, hash_id)
|
||||
if previous_hash != hash_id:
|
||||
key_to_hash_conflict_count += 1
|
||||
parent = hash_id.encode()
|
||||
|
||||
token_digest.update(row_index.to_bytes(4, "little"))
|
||||
token_digest.update(request_token_digest.digest())
|
||||
|
||||
payload: dict[str, Any] = {
|
||||
"schema": "qwen235b-trace-contract-audit-v1",
|
||||
"status": "pass_offline_source_contract"
|
||||
if not any(
|
||||
(
|
||||
length_mismatch_count,
|
||||
hash_count_mismatch_count,
|
||||
hash_to_key_conflict_count,
|
||||
key_to_hash_conflict_count,
|
||||
)
|
||||
)
|
||||
else "fail",
|
||||
"execution": {
|
||||
"host": socket.gethostname(),
|
||||
"device": "cpu_only",
|
||||
"elapsed_seconds": round(time.time() - started, 3),
|
||||
"python_version": platform.python_version(),
|
||||
"tokenizer_class": type(tokenizer).__name__,
|
||||
"transformers_version": transformers.__version__,
|
||||
"model_path": str(args.model.resolve()),
|
||||
},
|
||||
"trace": {
|
||||
"path": str(args.trace.resolve()),
|
||||
"sha256": sha256_file(args.trace),
|
||||
"source_request_count": len(rows),
|
||||
"context_exceeded_count": len(context_exceeded),
|
||||
"zero_output_count": len(zero_output),
|
||||
"exclusion_overlap_count": sum(row in zero_output for row in context_exceeded),
|
||||
"eligible_request_count": len(eligible),
|
||||
},
|
||||
"tokenization": {
|
||||
"total_token_count": total_tokens,
|
||||
"input_length_mismatch_count": length_mismatch_count,
|
||||
"length_order_sha256": length_digest.hexdigest(),
|
||||
"per_request_token_digest_sha256": token_digest.hexdigest(),
|
||||
},
|
||||
"source_hash_contract": {
|
||||
"source_block_size_tokens": args.source_block_size,
|
||||
"hash_count_mismatch_count": hash_count_mismatch_count,
|
||||
"full_block_count": full_blocks,
|
||||
"partial_block_count": partial_blocks,
|
||||
"unique_hash_id_count": len(hash_to_key),
|
||||
"unique_parent_chunk_key_count": len(key_to_hash),
|
||||
"hash_id_to_parent_chunk_conflict_count": hash_to_key_conflict_count,
|
||||
"parent_chunk_to_hash_id_conflict_count": key_to_hash_conflict_count,
|
||||
"key_definition": (
|
||||
"(parent source hash id, BLAKE2b-128 of the tokenizer token-id chunk)"
|
||||
),
|
||||
},
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
if payload["status"] != "pass_offline_source_contract":
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
24
runs/frontier-multicase-sufficiency-v1/fleet.toml
Normal file
24
runs/frontier-multicase-sufficiency-v1/fleet.toml
Normal file
@@ -0,0 +1,24 @@
|
||||
version = 1
|
||||
|
||||
[paths]
|
||||
state_dir = "runs/frontier-multicase-sufficiency-v1/fleet-state"
|
||||
artifacts_dir = "runs/frontier-multicase-sufficiency-v1/fleet-artifacts"
|
||||
|
||||
[ssh]
|
||||
connect_timeout_sec = 10
|
||||
|
||||
[scheduler]
|
||||
gpu_free_memory_mb = 1024
|
||||
gpu_free_utilization_pct = 10
|
||||
prefer_pack = true
|
||||
|
||||
[sync]
|
||||
mode = "scp"
|
||||
local_path = "runs/frontier-multicase-sufficiency-v1"
|
||||
|
||||
[[hosts]]
|
||||
name = "dash0"
|
||||
ssh_alias = "dash0"
|
||||
enabled = true
|
||||
sync_remote_path = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1"
|
||||
fleet_root = "/home/admin/cpfs/wjh/aituner/gpu-fleet-fidelity-v1"
|
||||
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Render the fleet queue for a frozen T0 real-surface plan."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def quoted(value: object) -> str:
|
||||
return json.dumps(str(value))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--plan", type=Path, required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--config", action="append", dest="configs")
|
||||
parser.add_argument("--rates", nargs="+", type=float)
|
||||
parser.add_argument("--artifact-root", default="artifacts/t0-real-surface-v1")
|
||||
parser.add_argument("--name-suffix", default="")
|
||||
args = parser.parse_args()
|
||||
plan = json.loads(args.plan.read_text())
|
||||
if plan.get("schema") != "qwen235b-t0-real-plan-v1" or len(plan.get("cells") or []) != 8:
|
||||
raise ValueError("invalid or incomplete T0 real plan")
|
||||
lattice = {0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20}
|
||||
if args.rates and any(rate not in lattice for rate in args.rates):
|
||||
raise ValueError("rate override must stay on the frozen T0 lattice")
|
||||
|
||||
indexed_cells = list(enumerate(plan["cells"]))
|
||||
if args.configs:
|
||||
requested = set(args.configs)
|
||||
known = {cell["config"]["name"] for _, cell in indexed_cells}
|
||||
if not requested <= known:
|
||||
raise ValueError(f"unknown configs: {sorted(requested - known)}")
|
||||
indexed_cells = [
|
||||
(index, cell)
|
||||
for index, cell in indexed_cells
|
||||
if cell["config"]["name"] in requested
|
||||
]
|
||||
|
||||
lines = [
|
||||
"# Generated from the frozen T0 real plan; do not edit rates in place.",
|
||||
"version = 1",
|
||||
"",
|
||||
]
|
||||
for index, cell in indexed_cells:
|
||||
config = cell["config"]
|
||||
suffix = f"-{args.name_suffix}" if args.name_suffix else ""
|
||||
name = f"qwen235b-t0-real-{config['name']}{suffix}-20260716-v1"
|
||||
artifact = f"{args.artifact_root.rstrip('/')}/{config['name']}"
|
||||
rates = args.rates or cell["rates"]
|
||||
lines.extend(
|
||||
[
|
||||
"[[jobs]]",
|
||||
f"name = {quoted(name)}",
|
||||
f"gpus = {int(config['tp'])}",
|
||||
'gpu_model = "H20"',
|
||||
'hosts = ["dash0"]',
|
||||
'command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"',
|
||||
f"artifacts = [{quoted(artifact)}]",
|
||||
"",
|
||||
"[jobs.env]",
|
||||
f"OUTPUT_ROOT = {quoted('/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/' + artifact)}",
|
||||
f"TP = {quoted(config['tp'])}",
|
||||
f"MNS = {quoted(config['mns'])}",
|
||||
f"MBT = {quoted(config['mbt'])}",
|
||||
f"RATES = {quoted(' '.join(f'{rate:.2f}' for rate in rates))}",
|
||||
f"SERVER_PORT = {quoted(18920 + index)}",
|
||||
'VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"',
|
||||
'MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"',
|
||||
"",
|
||||
]
|
||||
)
|
||||
args.output.write_text("\n".join(lines))
|
||||
print(args.output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
43
runs/frontier-multicase-sufficiency-v1/jobs.toml
Normal file
43
runs/frontier-multicase-sufficiency-v1/jobs.toml
Normal file
@@ -0,0 +1,43 @@
|
||||
# Append-only queue for the simulator-fidelity v1 campaign.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-tp4-smoke-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 1800 bash run_t0_smoke.sh"
|
||||
artifacts = ["artifacts/t0-smoke-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-smoke-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6"
|
||||
SERVER_PORT = "18910"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-decode-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_decode_attention_profile.sh"
|
||||
artifacts = ["artifacts/decode-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/decode-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-full-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_t0_full_attention_profile.sh"
|
||||
artifacts = ["artifacts/t0-full-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-full-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
@@ -0,0 +1,16 @@
|
||||
# One-shot queue view for the next authorized profile job. The canonical
|
||||
# append-only campaign queue remains jobs.toml.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-decode-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_decode_attention_profile.sh"
|
||||
artifacts = ["artifacts/decode-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/decode-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
@@ -0,0 +1,37 @@
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-anchor-isolation-smoke-tp4-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 1800 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "1.60"
|
||||
SERVER_PORT = "18930"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-anchor-isolation-smoke-tp4-mbt16k-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 1800 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "1.60"
|
||||
SERVER_PORT = "18931"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,15 @@
|
||||
# One-shot queue view for the T0 full-coverage attention profile.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-full-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_t0_full_attention_profile.sh"
|
||||
artifacts = ["artifacts/t0-full-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-full-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18922"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18923"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18920"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18921"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18926"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18925"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt8192-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18922"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt16384-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18923"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt8192-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18920"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt16384-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18921"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r3p20-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt8192-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18926"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt16384-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18925"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r3p20-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
146
runs/frontier-multicase-sufficiency-v1/jobs_t0_real_surface.toml
Normal file
146
runs/frontier-multicase-sufficiency-v1/jobs_t0_real_surface.toml
Normal file
@@ -0,0 +1,146 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18920"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18921"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18922"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18923"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18925"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18926"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
128
runs/frontier-multicase-sufficiency-v1/prepare_t0_real_plan.py
Normal file
128
runs/frontier-multicase-sufficiency-v1/prepare_t0_real_plan.py
Normal file
@@ -0,0 +1,128 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Select predeclared simulator-lattice anchors for blind real confirmation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
RATES = (0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20)
|
||||
CONFIG_NAMES = {
|
||||
f"tp{tp}_mns{mns}_mbt{mbt}"
|
||||
for tp in (4, 8)
|
||||
for mns in (64, 128)
|
||||
for mbt in (8192, 16384)
|
||||
}
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-freeze", type=Path, required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--selection-slo", default="tpot_150ms")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def select_rates(loads: list[dict[str, Any]], slo: str) -> list[float]:
|
||||
ordered = sorted(loads, key=lambda item: float(item["offered_request_rate"]))
|
||||
rates = [float(item["offered_request_rate"]) for item in ordered]
|
||||
feasible = [bool(item["slos"][slo]["feasible"]) for item in ordered]
|
||||
selected = {rates[0]}
|
||||
for index in range(len(rates) - 1):
|
||||
if feasible[index] != feasible[index + 1]:
|
||||
selected.update((rates[index], rates[index + 1]))
|
||||
if len(selected) == 1:
|
||||
if all(feasible):
|
||||
selected.update(rates[-2:])
|
||||
elif not any(feasible):
|
||||
selected.update(rates[:2])
|
||||
return sorted(selected)
|
||||
|
||||
|
||||
def warmup_requests(rate: float) -> int:
|
||||
return min(32, max(4, math.ceil(rate * 20.0)))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
freeze_path = args.frontier_freeze.resolve()
|
||||
freeze = json.loads(freeze_path.read_text())
|
||||
if freeze.get("schema") != "frontier-qwen235b-t0-surface-v1":
|
||||
raise ValueError("unexpected Frontier freeze schema")
|
||||
if freeze.get("status") != "frozen_before_real_surface":
|
||||
raise ValueError("Frontier surface is not frozen")
|
||||
results = freeze.get("config_results") or []
|
||||
if len(results) != 8 or any(len(item.get("loads") or []) != 8 for item in results):
|
||||
raise ValueError("Frontier surface is incomplete")
|
||||
names = {item.get("config", {}).get("name") for item in results}
|
||||
if names != CONFIG_NAMES:
|
||||
raise ValueError(f"Frontier config set mismatch: {names}")
|
||||
for item in results:
|
||||
rates = tuple(sorted(float(load["offered_request_rate"]) for load in item["loads"]))
|
||||
if rates != RATES:
|
||||
raise ValueError(f"Frontier rate lattice mismatch for {item['config']['name']}: {rates}")
|
||||
|
||||
cells = []
|
||||
total_expected_seconds = 0.0
|
||||
for item in results:
|
||||
config = item["config"]
|
||||
rates = select_rates(item["loads"], args.selection_slo)
|
||||
# Every anchor gets an independent server in both rounds. The estimate
|
||||
# includes server startup, target-rate warmup and conservative drain
|
||||
# allowances for both the discarded and measured request streams.
|
||||
expected_seconds = 2 * sum(
|
||||
120.0
|
||||
+ (warmup_requests(rate) - 1) / rate
|
||||
+ 60.0
|
||||
+ 63.0 / rate
|
||||
+ 60.0
|
||||
for rate in rates
|
||||
)
|
||||
total_expected_seconds += expected_seconds * int(config["tp"])
|
||||
cells.append(
|
||||
{
|
||||
"config": config,
|
||||
"rates": rates,
|
||||
"rounds": 2,
|
||||
"requests_per_anchor": 64,
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"target_rate_warmup_requests": {
|
||||
f"{rate:.2f}": warmup_requests(rate) for rate in rates
|
||||
},
|
||||
"expected_wall_seconds": expected_seconds,
|
||||
"expected_h20_gpu_hours": expected_seconds * int(config["tp"]) / 3600.0,
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
}
|
||||
)
|
||||
payload = {
|
||||
"schema": "qwen235b-t0-real-plan-v1",
|
||||
"frontier_freeze": {"path": str(freeze_path), "sha256": sha256(freeze_path)},
|
||||
"selection_slo": args.selection_slo,
|
||||
"selection_timing": "rate anchors frozen after complete simulator surface and before any accepted real surface cell",
|
||||
"execution_protocol_amendment": {
|
||||
"timing": "after excluded multi-rate-per-server diagnostic and before any accepted real surface cell",
|
||||
"reason": "observed cross-anchor GPU/kernel/batch warm-state leakage",
|
||||
"contract": "fresh server and target-rate warmup for every config-rate-round",
|
||||
},
|
||||
"strict_preregistered_slo": "tpot_40ms",
|
||||
"post_pilot_sensitivities": ["tpot_120ms", "tpot_150ms", "tpot_180ms"],
|
||||
"cells": cells,
|
||||
"expected_total_h20_gpu_hours": total_expected_seconds / 3600.0,
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
print(json.dumps({"cells": len(cells), "expected_total_h20_gpu_hours": payload["expected_total_h20_gpu_hours"]}, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user