Track simulator fidelity experiment artifacts
This commit is contained in:
75
docs/simulator-claim-map-20260716.md
Normal file
75
docs/simulator-claim-map-20260716.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Simulator-for-config-tuning related-work claim map
|
||||
|
||||
日期：2026-07-16。目的：为「Frontier/Vidur-class simulator 能否低成本解决 config tuning」这条主线建立 related-work 边界。Vidur 与 LLMServingSim 的条目基于原文（PDF 全文核读）；SimAI 基于论文页与摘要口径。每项按 Context / Claim / Assumption / Mechanism / Evidence / Boundary / 与本 project 的关系提取。
|
||||
|
||||
## Vidur（MLSys 2024，arXiv:2405.05465）
|
||||
|
||||
| 维度 | 内容 |
|
||||
|---|---|
|
||||
| Context | MSR India。首个面向 LLM inference 的大规模模拟器。Motivation 与我们一致：config search 复杂度 O(\|M\|·\|T\|)，且 optimal config 是 (model, trace) 的函数——Fig 1b 显示跨 trace misconfiguration 代价最高 2×。 |
|
||||
| Claim | (a) request-level 预测误差 <9%：static trace 上 P95 normalized execution latency 误差 ≤3.33%（4 模型 × 3 trace），dynamic trace 在 **85% capacity** 负载下误差 <5%。(b) Vidur-Search 用约 1 小时 96-core CPU（$9.93/h）为 LLaMA2-70B 找到最优 config，对比 deployment-based exploration 估算 42K GPU-hours ≈ $218K。(c) what-if 全量探索 $125 模拟成本 vs 估算 $1.14M 真机成本。 |
|
||||
| Assumption | operator runtime 可由单 GPU profiling + 小型 ML 估计器（random forest）插值；prefill attention 可用等效单序列 sqrt(Σp_i²) 近似；decode attention runtime 只依赖总 KV 读量而非 per-request context 分布；LLM 架构同质（小算子集合跨模型共享）。 |
|
||||
| Mechanism | 声明式 model spec → 算子三分类（token-level / sequence-level / communication）→ 单 GPU CUPTI profiling → RF runtime estimator → event-driven simulator + 三层 hierarchical scheduler（支持 vLLM/Orca+/Sarathi-Serve/FasterTransformer/LightLLM 策略）→ Vidur-Search 对每个 config 二分搜索 max QPS（判据 P99 scheduling delay <5s），目标 QPS/dollar。 |
|
||||
| Evidence | LLaMA2-7B/70B、InternLM-20B、Qwen-72B（全 dense）；Azure A100/H100 4-GPU pairwise-NVLink 节点；Chat-1M / Arxiv-4K / BWB-4K trace，总长截断到 4096 tokens。 |
|
||||
| Boundary | **作者明示**：接近 capacity point 时小误差会因排队失控放大，故 fidelity 评测停在 85% capacity。**结构性**：无 MoE、无 FP8/量化、无 prefix-cache reuse（多轮对话按独立请求处理）、无 speculative decoding（列为 future work）、PP 仅同步、长上下文未覆盖（4K 截断）。metric 口径为 normalized execution latency（static 排除 scheduling delay）。**最关键**：sim 选出的 config 在真机 ground-truth surface 上的 selection regret 从未被验证；42K GPU-h/$218K 是反事实估算，分母是穷举式 exploration 而非 strong sequential tuner。 |
|
||||
| 与本 project 的关系 | Frontier 即 Vidur-class（代码直接使用 vidur backend）+ 我们的 FP8/MoE/EP/decode-profile patches。我们的所有实验恰好工作在 Vidur 声明误差爆炸并回避的 regime（capacity point + SLO gate），补的正是它缺的 selection-regret ground truth。我们的 zero-shot 失败（25–30% regret）与其 <9% 不矛盾——不同 metric、不同 load regime、不同 stack alignment，论文必须主动写明这一点。其 Fig 1b 的 workload-conditioned 结论与我们 P4 sign-flip、P6 churn 互为独立佐证 → 支持 retune 频率 / amortization 论证（C3）。 |
|
||||
|
||||
## LLMServingSim（IISWC 2024，arXiv:2408.05499）
|
||||
|
||||
| 维度 | 内容 |
|
||||
|---|---|
|
||||
| Context | KAIST。scale-out LLM serving 的 HW/SW co-simulation，面向 NPU/PIM/异构加速器设计探索，基于 ASTRA-sim。 |
|
||||
| Claim | 对真实 multi-GPU vLLM serving 平均误差 14.7% 且「趋势一致」；比 mNPUsim/GeneSys/NeuPIMs 快 34.7–491×（摘要口径 91.5×）。 |
|
||||
| Assumption | iteration-level 模拟 + decoder-block 冗余复用（编译一个 block 复制展开、attention/非 attention 分离）可在可行时间内保持足够精度；硬件行为可由可插拔 accelerator compiler+simulator 栈表达（GeneSys 原型）。 |
|
||||
| Mechanism | 逐 iteration：scheduler（iteration-level batching、KV paging、operator mapping）→ per-device 硬件模拟 → graph converter（Chakra）→ ASTRA-sim 网络级模拟 → 循环。 |
|
||||
| Evidence | 与 multi-GPU vLLM 真机对照，变量为 LLM 架构、并行方案、NPU 数量、异构度；报告平均误差与趋势一致性。 |
|
||||
| Boundary | 定位是硬件/系统设计空间探索，不是 engine-knob config tuning；validation 口径是 trend-following，无 SLO-gated capacity、无 selection-regret；14.7% 平均误差大于典型 config 间 capacity margin（我们 12-cell 面上 top-2 差距 0.76%），故该精度不足以支撑近邻 config 选择。 |
|
||||
| 与本 project 的关系 | 说明「模拟保 trend」是社区通行 validation 标准；「trend ≠ selection」这一缺口对它同样成立。不构成直接 baseline，但在 related work 中界定我们评测口径（selection regret at capacity point）的必要性。 |
|
||||
|
||||
## SimAI（NSDI 2025，Alibaba，aliyun/SimAI）
|
||||
|
||||
| 维度 | 内容 |
|
||||
|---|---|
|
||||
| Context | 大规模 LLM **training** 的架构设计与参数调优模拟，生产背景（Alibaba Cloud）。 |
|
||||
| Claim | 各测试场景平均 98.1% 与真实结果对齐；为 host 设计与参数设置提供生产可用 guidance。 |
|
||||
| Assumption | training 过程可由 framework + kernel computation + collective communication 的选择性高保真集成复现。 |
|
||||
| Mechanism | 高保真集成三层栈 + 多线程加速 + lock-free global context sharing。 |
|
||||
| Evidence | 与生产 training 场景对齐（论文口径，未逐一核读实验细节）。 |
|
||||
| Boundary | training-only。training iteration 均匀、batch 组成静态——恰是 Vidur 指出 inference 所缺的性质，因此 98.1% 不可外推到 serving capacity point。 |
|
||||
| 与本 project 的关系 | 「simulator 指导 infra 决策」的工业先例与动机背书；不与 serving config tuning claim 竞争。引用价值在 motivation，不在 evaluation 对照。 |
|
||||
|
||||
## Frontier（本 project 被测对象，非 related work）
|
||||
|
||||
内部 Vidur-class 实现（vidur backend）+ 本 project 的 FP8/MoE tuning-key、Qwen MoE serving plan、TP/EP-aware cache key、critical-lane、decode/true-mixed profile 补丁。我们全部 fidelity 结论限定于该实现与已声明的 patch 集；见 `simulator-fidelity.md`。
|
||||
|
||||
## Consensus / disagreement / uncovered regime
|
||||
|
||||
**Consensus（三方一致或与我们互证）：**
|
||||
|
||||
1. operator/iteration profile + 调度复合的模拟器，在中低负载下能达到 5–15% latency 误差；模拟成本比真机低数个数量级。
|
||||
2. optimal config 是 (model, workload) 的函数，misconfiguration 代价可达 ~2×（Vidur Fig 1b；我们 P4 pattern sign-flip 与 P6 engine-churn 独立复证）。
|
||||
|
||||
**Disagreement：** 无直接冲突数字。我们的 zero-shot 失败与 Vidur <9% 处于不同 metric/regime，论文需主动解释，防止被误读为矛盾或重复。
|
||||
|
||||
**Uncovered regime（本 project 的空间）：**
|
||||
|
||||
1. **capacity-point + SLO-gated selection regret 无人用真机 ground-truth surface 验证。** Vidur 自认该 regime 误差爆炸并把评测停在 85% load；LLMServingSim 只验 trend。而 config tuning 的决策恰好发生在 capacity point。
|
||||
2. MoE、FP8、prefix reuse、speculative decoding、EP topology、长上下文均在已发表 fidelity envelope 之外。
|
||||
3. **alignment/profiling 成本从不与真机 tuning 成本同表比较。** Vidur 的 $218K 对比用穷举做分母；正确分母是 strong sequential tuner（我们实测 0.27–0.45 H20h/task，`runs/tuning-cost/metrics.json`）。
|
||||
4. envelope 失效的低成本检测（workload/runtime/topology 变化后何时还能信 simulator）无人提出。
|
||||
|
||||
## 对本 project claim 的直接影响
|
||||
|
||||
- **C1 定位句**：不是「Vidur 错了」，而是「Vidur-class claim 停在 sub-capacity load 的 prediction fidelity；把它外推到 SLO-gated capacity selection 是社区的隐含用法，我们证明该外推在 zero-shot 下失败（25–30% regret），并给出恢复 ranking 所需的最小真机证据层级」。
|
||||
- **C2**：Vidur 没有 minimum-real-evidence 的概念（要么全模拟、要么全真机）；per-TP calibration / 同栈 profile + KV capacity + patches 的证据层级是新贡献面。
|
||||
- **C3**：省钱叙事必须从「数量级」修正为「仅在 amortization 下成立」，分母换成 strong tuner 实测值；Vidur Fig 1b + 我们 P6 churn 共同支撑 retune 频率前提。
|
||||
|
||||
## 待 triage 的相邻工作（未读原文，暂不写 claim）
|
||||
|
||||
APEX（arXiv:2411.17651，并行执行计划模拟）、LLMServingSim 2.0（arXiv:2602.23036，异构+分离式）、Charon（arXiv:2605.17164，training+inference 统一）、inference-fleet-sim（arXiv:2603.16054，排队论容量规划）、AgentServeSim（arXiv:2606.09613，多轮 agent serving）。若审稿风险评估需要，按本表格式各补一行。
|
||||
|
||||
## Sources
|
||||
|
||||
- Vidur: <https://arxiv.org/abs/2405.05465>（全文核读版本：mlsys24 PDF）
|
||||
- LLMServingSim: <https://arxiv.org/pdf/2408.05499>
|
||||
- SimAI: <https://www.usenix.org/conference/nsdi25/presentation/wang-xizheng-simai>、<https://github.com/aliyun/SimAI>
|
||||
@@ -54,6 +54,11 @@ if [[ ! -x "${TRTLLM_DG_NVCC_COMPILER}" ]]; then
|
||||
echo "ERROR: TRTLLM_DG_NVCC_COMPILER is not executable: ${TRTLLM_DG_NVCC_COMPILER}" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! "${VENV_ROOT}/bin/python" "${CLIENT}" --help 2>&1 \
|
||||
| grep -F --quiet -- '--model-path'; then
|
||||
echo "ERROR: CLIENT must implement the fixed-shape client contract (--model-path, --rate, --input-tokens, --output-tokens): ${CLIENT}" >&2
|
||||
exit 1
|
||||
fi
|
||||
case "${DEEP_GEMM_JIT_DEBUG}" in
|
||||
true|false) ;;
|
||||
*) echo "ERROR: DEEP_GEMM_JIT_DEBUG must be true or false" >&2; exit 1 ;;
|
||||
|
||||
280
runs/frontier-multicase-sufficiency-v0/audit_ground_truth.py
Normal file
280
runs/frontier-multicase-sufficiency-v0/audit_ground_truth.py
Normal file
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audit the Qwen235B real-machine surfaces before comparing Frontier.
|
||||
|
||||
This script intentionally does not consume simulator output. It establishes
|
||||
whether each real response surface is complete and discriminative enough to
|
||||
support a later claim about simulator config selection.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
SCHEMA = "frontier-multicase-ground-truth-v0"
|
||||
EXPECTED_PROBES = 6
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict[str, Any]:
|
||||
with path.open(encoding="utf-8") as source:
|
||||
value = json.load(source)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError(f"expected JSON object: {path}")
|
||||
return value
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def config_from_result(result: dict[str, Any]) -> dict[str, int]:
|
||||
flags = result["config_patch"]["flag_patch"]
|
||||
tp = int(flags["tensor-parallel-size"])
|
||||
dp = int(flags.get("data-parallel-size", 1))
|
||||
return {
|
||||
"tp": tp,
|
||||
"dp": dp,
|
||||
"ep": int(flags.get("expert-parallel-size", 1)),
|
||||
"mns": int(flags["max-num-seqs"]),
|
||||
"mbt": int(flags["max-num-batched-tokens"]),
|
||||
"gpu_count": tp * dp,
|
||||
}
|
||||
|
||||
|
||||
def cell_id(config: dict[str, int]) -> str:
|
||||
topology = f"tp{config['tp']}"
|
||||
if config["dp"] != 1 or config["ep"] != 1:
|
||||
topology += f"_dp{config['dp']}_ep{config['ep']}"
|
||||
return f"{topology}_mns{config['mns']}_mbt{config['mbt']}"
|
||||
|
||||
|
||||
def trial_record(case: str, path: Path) -> dict[str, Any]:
|
||||
result = load_json(path)
|
||||
config = config_from_result(result)
|
||||
score = float(result["best_request_rate"]) / config["gpu_count"]
|
||||
probes = result.get("probes", [])
|
||||
infeasible_above = [
|
||||
float(probe.get("payload", probe)["request_rate"]) / config["gpu_count"]
|
||||
for probe in probes
|
||||
if not probe["feasible"]
|
||||
and float(probe.get("payload", probe)["request_rate"])
|
||||
/ config["gpu_count"]
|
||||
> score
|
||||
]
|
||||
upper_bound = min(infeasible_above) if infeasible_above else None
|
||||
probe_count = len(result.get("probes", []))
|
||||
primary_result = result.get("best_source") == "primary_search"
|
||||
no_probe_failure = not bool(result.get("completed_with_probe_failure", False))
|
||||
fully_valid = (
|
||||
result.get("status") == "completed"
|
||||
and probe_count == EXPECTED_PROBES
|
||||
and primary_result
|
||||
and no_probe_failure
|
||||
)
|
||||
return {
|
||||
"case": case,
|
||||
"cell_id": cell_id(config),
|
||||
**config,
|
||||
"score_req_s_per_gpu": score,
|
||||
"capacity_lower_bound_req_s_per_gpu": score,
|
||||
"capacity_upper_bound_req_s_per_gpu": upper_bound,
|
||||
"capacity_bracket_width_req_s_per_gpu": (
|
||||
upper_bound - score if upper_bound is not None else None
|
||||
),
|
||||
"best_request_rate_req_s": float(result["best_request_rate"]),
|
||||
"best_sampling_u": float(result["best_sampling_u"]),
|
||||
"best_pass_rate": float(result["best_pass_rate"]),
|
||||
"probe_count": probe_count,
|
||||
"best_source": result.get("best_source"),
|
||||
"completed_with_probe_failure": bool(
|
||||
result.get("completed_with_probe_failure", False)
|
||||
),
|
||||
"fully_valid": fully_valid,
|
||||
"result_path": str(path),
|
||||
"result_sha256": sha256_file(path),
|
||||
}
|
||||
|
||||
|
||||
def summarize_case(case: str, rows: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
if not rows:
|
||||
raise ValueError(f"no rows for case: {case}")
|
||||
ids = [row["cell_id"] for row in rows]
|
||||
if len(ids) != len(set(ids)):
|
||||
duplicates = sorted(cell for cell in set(ids) if ids.count(cell) > 1)
|
||||
raise ValueError(f"duplicate cells for {case}: {duplicates}")
|
||||
|
||||
best = max(row["score_req_s_per_gpu"] for row in rows)
|
||||
tolerance = max(1e-12, best * 1e-9)
|
||||
top = [
|
||||
row["cell_id"]
|
||||
for row in rows
|
||||
if math.isclose(row["score_req_s_per_gpu"], best, abs_tol=tolerance)
|
||||
]
|
||||
distinct_scores = []
|
||||
for score in sorted({row["score_req_s_per_gpu"] for row in rows}, reverse=True):
|
||||
if not any(math.isclose(score, seen, abs_tol=tolerance) for seen in distinct_scores):
|
||||
distinct_scores.append(score)
|
||||
|
||||
max_lower_bound = max(row["capacity_lower_bound_req_s_per_gpu"] for row in rows)
|
||||
possibly_optimal = [
|
||||
row["cell_id"]
|
||||
for row in rows
|
||||
if row["capacity_upper_bound_req_s_per_gpu"] is None
|
||||
or row["capacity_upper_bound_req_s_per_gpu"] + tolerance >= max_lower_bound
|
||||
]
|
||||
|
||||
total_pairs = len(rows) * (len(rows) - 1) // 2
|
||||
tied_pairs = sum(
|
||||
1
|
||||
for left_index, left in enumerate(rows)
|
||||
for right in rows[left_index + 1 :]
|
||||
if math.isclose(
|
||||
left["score_req_s_per_gpu"],
|
||||
right["score_req_s_per_gpu"],
|
||||
abs_tol=tolerance,
|
||||
)
|
||||
)
|
||||
return {
|
||||
"case": case,
|
||||
"cell_count": len(rows),
|
||||
"fully_valid_cell_count": sum(bool(row["fully_valid"]) for row in rows),
|
||||
"invalid_cells": [row["cell_id"] for row in rows if not row["fully_valid"]],
|
||||
"best_score_req_s_per_gpu": best,
|
||||
"top_set": sorted(top),
|
||||
"top_set_size": len(top),
|
||||
"random_top_set_hit_rate": len(top) / len(rows),
|
||||
"distinct_score_count": len(distinct_scores),
|
||||
"distinct_scores_req_s_per_gpu": distinct_scores,
|
||||
"possibly_optimal_set_from_search_brackets": sorted(possibly_optimal),
|
||||
"possibly_optimal_set_size": len(possibly_optimal),
|
||||
"pair_count": total_pairs,
|
||||
"tied_pair_count": tied_pairs,
|
||||
"informative_pair_count": total_pairs - tied_pairs,
|
||||
"informative_pair_fraction": (
|
||||
(total_pairs - tied_pairs) / total_pairs if total_pairs else 0.0
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def markdown_report(metrics: dict[str, Any], rows: list[dict[str, Any]]) -> str:
|
||||
lines = [
|
||||
"# Qwen235B ground-truth audit",
|
||||
"",
|
||||
"Objective: maximum SLO-feasible offered request throughput per GPU.",
|
||||
"This report contains real-machine data only; it makes no Frontier match claim.",
|
||||
"",
|
||||
"| case | valid cells | score levels | top-set size | random top-set hit | informative pairs |",
|
||||
"|---|---:|---:|---:|---:|---:|",
|
||||
]
|
||||
for case in metrics["cases"]:
|
||||
lines.append(
|
||||
f"| {case['case']} | {case['fully_valid_cell_count']}/{case['cell_count']} "
|
||||
f"| {case['distinct_score_count']} | {case['top_set_size']}/{case['cell_count']} "
|
||||
f"| {case['random_top_set_hit_rate']:.1%} "
|
||||
f"| {case['informative_pair_count']}/{case['pair_count']} "
|
||||
f"({case['informative_pair_fraction']:.1%}) |"
|
||||
)
|
||||
|
||||
lines.extend(["", "## Cells", ""])
|
||||
for case in metrics["cases"]:
|
||||
lines.extend(
|
||||
[
|
||||
f"### {case['case']}",
|
||||
"",
|
||||
"| cell | capacity bracket (req/s/GPU) | valid | probes | source |",
|
||||
"|---|---:|---:|---:|---|",
|
||||
]
|
||||
)
|
||||
for row in sorted(
|
||||
(row for row in rows if row["case"] == case["case"]),
|
||||
key=lambda row: row["cell_id"],
|
||||
):
|
||||
upper = row["capacity_upper_bound_req_s_per_gpu"]
|
||||
bracket = (
|
||||
f"[{row['capacity_lower_bound_req_s_per_gpu']:.9f}, "
|
||||
f"{upper:.9f})"
|
||||
if upper is not None
|
||||
else f"[{row['capacity_lower_bound_req_s_per_gpu']:.9f}, +inf)"
|
||||
)
|
||||
lines.append(
|
||||
f"| {row['cell_id']} | {bracket} "
|
||||
f"| {'yes' if row['fully_valid'] else 'no'} | {row['probe_count']} "
|
||||
f"| {row['best_source']} |"
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
f"Top set: `{', '.join(case['top_set'])}`.",
|
||||
f"Possibly optimal under binary-search brackets: "
|
||||
f"`{', '.join(case['possibly_optimal_set_from_search_brackets'])}`.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
|
||||
lines.extend(
|
||||
[
|
||||
"## Interpretation guardrail",
|
||||
"",
|
||||
"A Frontier top-set hit is insufficient by itself because the surfaces contain "
|
||||
"large ties. The later comparison must report selected-config regret and "
|
||||
"tie-aware pairwise ranking, and must keep invalid real cells visible.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def write_csv(path: Path, rows: list[dict[str, Any]]) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8", newline="") as target:
|
||||
writer = csv.DictWriter(target, fieldnames=list(rows[0]))
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--prefill-root", action="append", type=Path, required=True)
|
||||
parser.add_argument("--decode-root", action="append", type=Path, required=True)
|
||||
parser.add_argument("--output-dir", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
cases = {"qwen235b_prefill_only": args.prefill_root, "qwen235b_decode_only": args.decode_root}
|
||||
rows = []
|
||||
for case, roots in cases.items():
|
||||
for root in roots:
|
||||
paths = sorted(root.glob("store/*/trials/trial-*/result.json"))
|
||||
if not paths:
|
||||
raise ValueError(f"no result files below {root}")
|
||||
rows.extend(trial_record(case, path) for path in paths)
|
||||
|
||||
summaries = [
|
||||
summarize_case(case, [row for row in rows if row["case"] == case])
|
||||
for case in cases
|
||||
]
|
||||
metrics = {"schema": SCHEMA, "cases": summaries}
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
write_csv(args.output_dir / "cells.csv", rows)
|
||||
(args.output_dir / "metrics.json").write_text(
|
||||
json.dumps(metrics, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
(args.output_dir / "report.md").write_text(
|
||||
markdown_report(metrics, rows), encoding="utf-8"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
110
runs/frontier-multicase-sufficiency-v0/audit_qwen30_baseline.py
Normal file
110
runs/frontier-multicase-sufficiency-v0/audit_qwen30_baseline.py
Normal file
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract the predictive-versus-calibrated Frontier Qwen30B baseline."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODES = ("uncalibrated/SLO-gated", "frozen-calibrated/SLO-gated")
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict[str, Any]:
|
||||
with path.open(encoding="utf-8") as source:
|
||||
value = json.load(source)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError(f"expected JSON object: {path}")
|
||||
return value
|
||||
|
||||
|
||||
def extract(metrics: dict[str, Any], protocol: dict[str, Any]) -> dict[str, Any]:
|
||||
analyses = metrics["analyses"]
|
||||
rows = []
|
||||
for mode in MODES:
|
||||
analysis = analyses[mode]
|
||||
values = analysis["metrics"]
|
||||
top1 = values["top1"]
|
||||
confusion = analysis["false_feasibility"]["overall"]
|
||||
rows.append(
|
||||
{
|
||||
"mode": mode,
|
||||
"selected_cells": top1["candidate_cells"],
|
||||
"optimistic_real_regret": top1["optimistic_regret"],
|
||||
"worst_case_real_regret": top1["worst_case_regret"],
|
||||
"kendall_tau_b": values["kendall_tau_b"]["tau_b"],
|
||||
"pairwise_exact_sign_accuracy": values["pairwise_direction"][
|
||||
"exact_sign_accuracy"
|
||||
],
|
||||
"false_feasible": confusion["false_feasible"],
|
||||
"false_infeasible": confusion["false_infeasible"],
|
||||
"agreement": confusion["agreement"],
|
||||
}
|
||||
)
|
||||
return {
|
||||
"schema": "frontier-qwen30-calibration-audit-v0",
|
||||
"rows": rows,
|
||||
"calibration": {
|
||||
"fitted_a_tp": protocol["fitted_a_tp"],
|
||||
"fit_fixture": protocol["fit_fixture"],
|
||||
"holdout_fixture": protocol["holdout_fixture"],
|
||||
"loss": protocol["loss"],
|
||||
"refit_on_holdout": protocol["refit_on_holdout"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def report(result: dict[str, Any]) -> str:
|
||||
lines = [
|
||||
"# Qwen30B Frontier baseline audit",
|
||||
"",
|
||||
"| mode | selected cells | worst real regret | Kendall tau-b | pair sign accuracy | feasibility (agree/FP/FN) |",
|
||||
"|---|---|---:|---:|---:|---:|",
|
||||
]
|
||||
for row in result["rows"]:
|
||||
lines.append(
|
||||
f"| {row['mode']} | {', '.join(row['selected_cells'])} "
|
||||
f"| {row['worst_case_real_regret']:.2%} | {row['kendall_tau_b']:.4f} "
|
||||
f"| {row['pairwise_exact_sign_accuracy']:.2%} "
|
||||
f"| {row['agreement']}/{row['false_feasible']}/{row['false_infeasible']} |"
|
||||
)
|
||||
calibration = result["calibration"]
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"The calibrated mode applies a distinct end-to-end execution-time scale per TP: "
|
||||
+ ", ".join(
|
||||
f"TP{tp}={value:.6f}"
|
||||
for tp, value in sorted(calibration["fitted_a_tp"].items())
|
||||
)
|
||||
+ ".",
|
||||
"",
|
||||
f"Those scales were fitted against real total throughput on "
|
||||
f"`{calibration['fit_fixture']}` and checked without refitting on "
|
||||
f"`{calibration['holdout_fixture']}`. This validates within-workload transfer of "
|
||||
"the calibration, not zero-shot Frontier prediction across TP.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--metrics", type=Path, required=True)
|
||||
parser.add_argument("--calibration-protocol", type=Path, required=True)
|
||||
parser.add_argument("--output-dir", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
result = extract(load_json(args.metrics), load_json(args.calibration_protocol))
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
(args.output_dir / "metrics.json").write_text(
|
||||
json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
(args.output_dir / "report.md").write_text(report(result), encoding="utf-8")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,166 @@
|
||||
# Community vLLM Qwen235B versus Frontier protocol v0
|
||||
|
||||
## Hypothesis
|
||||
|
||||
I believe that collecting compute and communication profiles from the same
|
||||
community vLLM execution stack used for serving removes the current
|
||||
internal-runtime confounder. I will verify this by freezing Frontier's
|
||||
profile-only predictions before collecting the corresponding serving response
|
||||
surface, then measuring config-selection regret and rank agreement.
|
||||
|
||||
This experiment tests Frontier inside a declared compatibility envelope. It
|
||||
does not claim fidelity for the previous internal vLLM, EAGLE3, DeepEP, or
|
||||
external-KV setup.
|
||||
|
||||
## Frozen system boundary
|
||||
|
||||
- Host: `dash0`, 8 NVIDIA H20 GPUs, driver `580.95.05`.
|
||||
- Model: `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8`.
|
||||
- Model config SHA256:
|
||||
`702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e`.
|
||||
- Model index SHA256:
|
||||
`e03d4abec9611fca05844d2c8b5a08318ddc3c86acfb9481a0ef5692bf76e6d6`.
|
||||
- Frontier: NetX-lab/Frontier commit
|
||||
`d9cfeb6d8791fbf2f295dd9744c56a666171776e`.
|
||||
- vLLM target: exact community release `0.10.2`, installed in a new isolated
|
||||
environment with wheel/source hash recorded. Frontier declares
|
||||
`vllm>=0.10,<0.11`; the model README declares `vllm>=0.8.5`.
|
||||
- Transformers is pinned to `4.55.2` and tokenizers resolves to `0.21.4`.
|
||||
vLLM 0.10.2 only declares a lower bound on Transformers, while current
|
||||
Transformers 5.x removes a tokenizer API used by this vLLM release.
|
||||
- Do not use the shared `/usr/local` vLLM. Its imported version and package
|
||||
metadata disagree, so it is not a reproducible community baseline.
|
||||
|
||||
The profiler and serving process must import the same vLLM installation. The
|
||||
following execution choices are fixed for the first validation pass:
|
||||
|
||||
- FlashInfer attention in both profiler and serving;
|
||||
- eager execution, with CUDA graphs disabled;
|
||||
- no speculative decoding;
|
||||
- no external KV connector;
|
||||
- no prefix reuse;
|
||||
- chunked prefill enabled;
|
||||
- BF16 KV cache (`auto` for this model), while the checkpoint's block-wise FP8
|
||||
weight quantization remains enabled;
|
||||
- community vLLM default MoE implementation unless a backend is explicitly
|
||||
frozen and supported on both sides.
|
||||
|
||||
The SM90 path cannot use `CutlassBlockScaledGroupedGemm` (the vLLM 0.10.2
|
||||
implementation gates that path on SM100), so this system uses Triton block-FP8
|
||||
MoE. However, the two sides do not yet select the same Triton tuning config.
|
||||
Serving resolves the checked-in H20 config named with
|
||||
`dtype=fp8_w8a8`, while Frontier's standalone MoE profiler requests a filename
|
||||
without that FP8 dtype component and falls back to a default config. Formal
|
||||
comparison is gated on repairing this profiler/runtime tuning-config mismatch
|
||||
and re-profiling; recording the same vLLM package version is not sufficient.
|
||||
|
||||
These controls isolate operator composition and scheduling. CUDA graphs,
|
||||
FlashAttention, speculative decoding, external KV, and optimized expert
|
||||
communication become separate stress cases after the controlled pass.
|
||||
|
||||
## Case P: prefill-only
|
||||
|
||||
Reuse the original trace window and SLO because the community model's 40,960
|
||||
token limit covers the filtered input plus the one-token completion.
|
||||
|
||||
- Window: `thinking_w20260327_1000`.
|
||||
- Input filter: 0--32,768 tokens.
|
||||
- Output override: 1 token.
|
||||
- Replay scale: 1.0.
|
||||
- SLO pass rate: at least 0.95.
|
||||
- TTFT SLO: 1 s for input <=8,191; 2 s otherwise.
|
||||
- Objective: maximum SLO-feasible offered requests/s/GPU.
|
||||
- Candidate grid: TP `{4,8}` x MNS `{64,128}` x MBT `{8192,16384}`.
|
||||
- DP=1 and expert parallel disabled.
|
||||
|
||||
This case can test the TP4-versus-TP8 decision and batching effects. It must not
|
||||
reuse performance values from the internal 256k model as ground truth.
|
||||
|
||||
## Case D: decode-dominant
|
||||
|
||||
Do not initially reproduce the previous strict decode-only case. It depends on
|
||||
`DecodeBenchConnector`, EAGLE3, FP8 KV, DeepEP/NVSHMEM, and decode CUDA graphs,
|
||||
which are outside the controlled Frontier profile contract.
|
||||
|
||||
Construct a community-only decode-dominant case that both systems can express:
|
||||
|
||||
- same trace window and timestamp/sampling fields;
|
||||
- input filter: 1--512 tokens;
|
||||
- output override: 512 tokens with EOS ignored;
|
||||
- SLO pass rate: at least 0.95;
|
||||
- TPOT SLO: 40 ms;
|
||||
- objective: maximum SLO-feasible offered requests/s/GPU;
|
||||
- topology grid: `(TP=4, DP=2, EP=8)` and `(TP=2, DP=4, EP=8)`;
|
||||
- batching grid: MNS `{64,128}` x MBT `{256,384}`.
|
||||
|
||||
This is deliberately named decode-dominant, not decode-only. A strict
|
||||
decode-only claim requires an initial-KV state contract in Frontier.
|
||||
|
||||
## Required profile closure
|
||||
|
||||
Profile data are measurement inputs, not end-to-end calibration. No serving
|
||||
throughput or latency from either case may scale the profiles.
|
||||
|
||||
1. Linear/operator profiles for the TP degrees consumed by the two grids.
|
||||
2. FlashInfer attention profiles covering the observed prefill/decode batch,
|
||||
context-length, and chunk-size ranges.
|
||||
3. FP8 MoE profiles for the actually consumed parallel pairs:
|
||||
`(MoE TP=4, EP=1)`, `(MoE TP=8, EP=1)`, and `(MoE TP=1, EP=8)`.
|
||||
4. H20 intra-node collective profiles for TP all-reduce at world sizes 2, 4,
|
||||
and 8 and the EP8 all-to-all path.
|
||||
|
||||
Frontier already provides an H20 device description but no checked-in H20
|
||||
network profiles. Its public collective profiler covers all-reduce and
|
||||
send/recv, not all-to-all. Therefore Case P may proceed after H20 all-reduce
|
||||
closure; Case D remains blocked until EP8 all-to-all is either measured and
|
||||
consumed or the selected communication model is independently validated
|
||||
against those measurements.
|
||||
|
||||
## Blind run order
|
||||
|
||||
1. Build the isolated environment and record package/binary hashes.
|
||||
2. Run one-row compute and collective smokes, then one TP4 server-load/request
|
||||
smoke. A smoke failure stops the campaign.
|
||||
3. Collect profiles and validate CSV metadata/coverage.
|
||||
4. Run Frontier for every candidate and offered-load anchor.
|
||||
5. Freeze simulator outputs and their SHA256 checksums.
|
||||
6. Only then collect community-vLLM serving ground truth. Randomize the first
|
||||
trial order and reverse it for the second trial.
|
||||
7. Refine only decision-relevant capacity intervals that still overlap.
|
||||
|
||||
## Metrics and decision rule
|
||||
|
||||
Report per case:
|
||||
|
||||
- absolute simulated and real TTFT/TPOT/throughput values;
|
||||
- anchor-level SLO feasibility confusion;
|
||||
- selected-config real regret, including interval-robust regret;
|
||||
- Kendall tau-b with ties preserved;
|
||||
- informative-pair direction accuracy;
|
||||
- top-set hit and random top-set hit probability;
|
||||
- profile and real-GPU measurement cost.
|
||||
|
||||
Frontier is sufficient as a config ranker for this controlled family only if
|
||||
every completed case has:
|
||||
|
||||
- worst selected-config real regret <=5%;
|
||||
- Kendall tau-b >=0.8 on enough informative pairs;
|
||||
- no unresolved ground-truth interval capable of reversing the decision;
|
||||
- no per-case or per-action end-to-end calibration.
|
||||
|
||||
If the controlled pass succeeds, enable one omitted mechanism at a time. If it
|
||||
fails, use operator/communication/stage residuals to localize which composition
|
||||
assumption reverses the ranking before proposing a new tuner mechanism.
|
||||
|
||||
## Launch gates and initial cost cap
|
||||
|
||||
The first authorized GPU action should be smoke-only:
|
||||
|
||||
- isolated vLLM import and Qwen235B TP4 load;
|
||||
- one request through the community server;
|
||||
- one representative FP8 linear, attention, MoE, and all-reduce profile point;
|
||||
- expected wall time: 20--40 minutes;
|
||||
- hard GPU budget: 2 H20-GPU-hours.
|
||||
|
||||
No full response-surface sweep is authorized by this protocol. Its cost and
|
||||
anchor count must be resolved from the smoke timings and echoed separately.
|
||||
114
runs/frontier-multicase-sufficiency-v0/findings.md
Normal file
114
runs/frontier-multicase-sufficiency-v0/findings.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Frontier multi-case sufficiency: current finding
|
||||
|
||||
## Bottom line
|
||||
|
||||
The existing evidence does **not** show that Frontier already solves config
|
||||
selection. The strongest Qwen30B match is an action-conditioned calibrated
|
||||
result, while the zero-shot/profile-only result selects the wrong TP family.
|
||||
The Qwen235B surfaces are useful follow-up cases, but decode ground truth and
|
||||
Frontier's execution semantics must be repaired before a match claim is valid.
|
||||
|
||||
## Qwen30B: calibration is decision-bearing
|
||||
|
||||
Under the aligned maximum-SLO-feasible-throughput objective:
|
||||
|
||||
| reading | Frontier selection | worst real regret | Kendall tau-b |
|
||||
|---|---|---:|---:|
|
||||
| profile-only | TP4/MNS32 or TP4/MNS64 | 25.63% | 0.0000 |
|
||||
| per-TP calibrated | TP2/MNS32 or TP2/MNS64 | 0.76% | 0.9668 |
|
||||
|
||||
The calibrated run multiplies all simulated execution times by a separately
|
||||
fitted factor for each TP: TP1=0.723481, TP2=0.468089, TP4=0.352137. These
|
||||
factors were fitted to real end-to-end throughput on the same model/workload
|
||||
family. The large, TP-dependent residual is therefore evidence that the
|
||||
profile-only simulator misses action-dependent execution behavior; it is not
|
||||
evidence that the unmodified simulator predicts the action correctly.
|
||||
|
||||
## Qwen235B real response surfaces
|
||||
|
||||
### Prefill-only
|
||||
|
||||
- 8/8 cells are complete primary searches.
|
||||
- The point-estimate top set contains all four TP8 cells, so a random cell has a
|
||||
50% top-set hit probability.
|
||||
- Capacity brackets still separate all TP8 cells from all TP4 cells. Thus this
|
||||
case can test the **TP4 versus TP8** decision, but the current six probes do
|
||||
not distinguish MNS/MBT within TP8.
|
||||
|
||||
### Decode-only
|
||||
|
||||
- Only 7/8 cells are fully valid; `TP2/DP4/EP8, MNS128, MBT384` is marked
|
||||
`partial_probe_before_failure` after an engine restart/port failure.
|
||||
- There are only two point-estimate score levels, and the top set contains 5/8
|
||||
cells (62.5% random hit probability).
|
||||
- More importantly, all eight binary-search capacity brackets overlap the best
|
||||
observed lower bound. The current data cannot rule out any cell as optimal.
|
||||
|
||||
Consequently, a decode top-set hit on these results is not a match. After the
|
||||
simulator selects a cell, the cheapest rigorous next step is to refine only
|
||||
that cell and one competing topology/batching cell until their capacity
|
||||
intervals separate or remain statistically indistinguishable.
|
||||
|
||||
## Frontier semantic coverage for Qwen235B
|
||||
|
||||
The topology itself is expressible: Frontier exposes attention TP/DP and MoE
|
||||
TP/EP separately, so TP4/DP2/EP8 and TP2/DP4/EP8 need not be decomposed into
|
||||
independent scheduling and execution problems.
|
||||
|
||||
The current execution model is not yet aligned, however:
|
||||
|
||||
- no checked-in H20 Qwen235B `linear_op.csv`, `attention.csv`, or `moe.csv`;
|
||||
- checked-in Qwen235B config is BF16 with max position 40960, whereas the real
|
||||
run uses FP8 weights, FP8 KV, and max model length 262144;
|
||||
- real prefill uses FlashAttention and internal BLADNN kernels, while Frontier's
|
||||
attention profiler exposes only FlashInfer and NO-OP backends;
|
||||
- real decode starts with a dummy-filled external KV cache through
|
||||
`DecodeBenchConnector`; Frontier's trace request generator requires positive
|
||||
prefill tokens and has no equivalent initial-KV trace contract;
|
||||
- real decode combines DeepEP/NVSHMEM, EAGLE3, and FULL_DECODE_ONLY CUDA graphs;
|
||||
Frontier explicitly treats speculative decode plus decode CUDA graphs as a
|
||||
conflicting/diagnostic combination.
|
||||
|
||||
Running dummy profiles or silently substituting FlashInfer/eager decode would
|
||||
produce a number, but it would not test whether Frontier matches this system.
|
||||
|
||||
## Community-vLLM Qwen235B smoke
|
||||
|
||||
The controlled community stack is feasible: vLLM 0.10.2 loaded the
|
||||
Qwen3-235B-A22B-FP8 checkpoint on TP4, allocated a BF16 KV cache, and completed
|
||||
a real request. Representative FP8 linear/MoE, FlashInfer attention, and TP4
|
||||
NCCL paths also executed successfully.
|
||||
|
||||
The smoke nevertheless found a profiler/runtime mismatch before any scheduler
|
||||
model was involved. Frontier's MoE wrapper calls `get_config_dtype_str` without
|
||||
`use_fp8_w8a8=True`, so it misses vLLM's tuned H20 block-FP8 Triton config and
|
||||
uses a default. At TP4/EP1 with 16 tokens, a paired five-routing-seed factorial
|
||||
measured:
|
||||
|
||||
| variant | grouped-GEMM mean | paired delta vs original |
|
||||
|---|---:|---:|
|
||||
| original default config + FP16 compute type | 0.3100 ms | 0.00% |
|
||||
| FP8 config key only | 0.2508 ms | -19.12% |
|
||||
| BF16 compute type only | 0.3088 ms | -0.40% |
|
||||
| both aligned | 0.2512 ms | -18.99% |
|
||||
|
||||
The config-key-only routing-seed 95% interval is [-21.26%, -16.98%]; the
|
||||
compute-type-only interval crosses zero. This localizes the dominant error at
|
||||
this point to kernel tuning-config selection. The original MoE CSV must not be
|
||||
used for a formal Frontier ranking until this path and the related shuffling
|
||||
block-size lookup are aligned and re-profiled. Full evidence and hashes are in
|
||||
`results/community-qwen235b-smoke/`.
|
||||
|
||||
## Research implication
|
||||
|
||||
The most interesting observation is already visible: Frontier's error is not a
|
||||
single global time bias. It changes strongly with the execution action (TP in
|
||||
Qwen30B), and a per-action residual can reverse the selected config. Qwen235B
|
||||
prefill and decode offer held-out tests of whether those residuals are explained
|
||||
by measurable execution-state features (kernel family, communication mode,
|
||||
graph mode, speculative width, and KV initial state) or require case-specific
|
||||
end-to-end fitting.
|
||||
|
||||
That is a systems question: **which execution-state transitions make operator
|
||||
profile composition non-invariant across configurations, and what is the
|
||||
minimum real evidence needed to recover the counterfactual ordering?**
|
||||
79
runs/frontier-multicase-sufficiency-v0/protocol.md
Normal file
79
runs/frontier-multicase-sufficiency-v0/protocol.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Frontier multi-case sufficiency protocol v0
|
||||
|
||||
## Claim under test
|
||||
|
||||
Can Frontier select a low-regret configuration for the frozen candidate grid
|
||||
under the objective **maximum SLO-feasible offered request throughput per GPU**?
|
||||
|
||||
The allowed conclusion is scoped to the tested model/runtime/hardware/workload
|
||||
family. Three matches cannot establish universal simulator fidelity.
|
||||
|
||||
## Cases
|
||||
|
||||
1. Qwen3-30B-A3B mixed chat, H20, TP x MNS grid (existing data).
|
||||
2. Qwen3-235B-A22B prefill-only, H20, TP x MNS x MBT grid.
|
||||
3. Qwen3-235B-A22B decode-only, H20, TP/DP/EP x MNS x MBT grid.
|
||||
|
||||
For Qwen235B, keep the real trace window, request filtering, SLO, model build,
|
||||
runtime flags, and candidate cells recorded by the original trials. Frontier is
|
||||
evaluated at the same offered-load anchors; no capacity extrapolation beyond
|
||||
the common anchors is allowed.
|
||||
|
||||
## Two separate simulator readings
|
||||
|
||||
- **Zero-shot/profile-only:** operator profiles and documented hardware/model
|
||||
inputs are allowed; no end-to-end measurement from the evaluated workload is
|
||||
used to scale Frontier.
|
||||
- **Calibrated:** every real measurement used to fit a scalar or residual is
|
||||
charged and reported. Calibration is fitted on a declared train fixture and
|
||||
evaluated on a disjoint workload/config holdout.
|
||||
|
||||
These readings must never be merged. A per-TP scale fitted from the same
|
||||
workload does not count as zero-shot simulator accuracy.
|
||||
|
||||
## Ground-truth gate
|
||||
|
||||
Before a match decision:
|
||||
|
||||
- every result must be a completed primary search without probe failure;
|
||||
- capacity is an interval from the largest feasible anchor to the next
|
||||
infeasible anchor, not merely the feasible lower bound;
|
||||
- refine probes when the possibly-optimal set induced by those intervals is too
|
||||
broad to distinguish the simulator's selected cell;
|
||||
- record the random top-set hit rate and the number of informative (non-tied)
|
||||
pairs.
|
||||
|
||||
## Primary metrics and predeclared decision rule
|
||||
|
||||
Per case, report:
|
||||
|
||||
1. selected-config real regret (point estimate and interval-robust bound);
|
||||
2. Kendall tau-b and exact pair-direction accuracy with real ties preserved;
|
||||
3. top-set hit and its random-hit baseline;
|
||||
4. anchor-level SLO feasibility confusion;
|
||||
5. profile/calibration cost and all right-censored cells.
|
||||
|
||||
The profile-only simulator is considered sufficient as a **config ranker for
|
||||
the tested family** only if every case has:
|
||||
|
||||
- worst selected-config real regret <= 5%;
|
||||
- Kendall tau-b >= 0.8 on a response surface with enough informative pairs;
|
||||
- no unresolved ground-truth interval that can reverse the selected decision;
|
||||
- no per-case or per-action end-to-end calibration.
|
||||
|
||||
SLO-oracle sufficiency is a stronger claim and additionally requires low false
|
||||
feasible/false infeasible rates; ranker success alone does not establish it.
|
||||
|
||||
## Current representational gaps to disclose
|
||||
|
||||
- The checked-in Frontier Qwen235B model config is BF16 and has no H20
|
||||
Qwen235B operator profiles, while the real runtime uses FP8 weights and FP8 KV.
|
||||
- Prefill uses internal vLLM/BLADNN paths and PIECEWISE CUDA graphs.
|
||||
- Decode uses TP/DP with EP8, DeepEP/NVSHMEM, EAGLE3, DecodeBenchConnector, and
|
||||
FULL_DECODE_ONLY CUDA graphs.
|
||||
- Frontier currently rejects the faithful combination of speculative decoding
|
||||
and decode CUDA-graph modeling unless using a diagnostic opt-in; its public
|
||||
examples disable decode CUDA graphs for speculative decoding.
|
||||
|
||||
Therefore an as-is Frontier run and an upgraded/semantically aligned run, if
|
||||
implemented, must be reported separately.
|
||||
@@ -0,0 +1,333 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3MoeForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"decoder_sparse_step": 1,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 12288,
|
||||
"max_position_embeddings": 40960,
|
||||
"max_window_layers": 94,
|
||||
"mlp_only_layers": [],
|
||||
"model_type": "qwen3_moe",
|
||||
"moe_intermediate_size": 1536,
|
||||
"norm_topk_prob": true,
|
||||
"num_attention_heads": 64,
|
||||
"num_experts": 128,
|
||||
"num_experts_per_tok": 8,
|
||||
"num_hidden_layers": 94,
|
||||
"num_key_value_heads": 4,
|
||||
"output_router_logits": false,
|
||||
"rms_norm_eps": 0.000001,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000.0,
|
||||
"router_aux_loss_coef": 0.001,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.0",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936,
|
||||
"quantization_config": {
|
||||
"activation_scheme": "dynamic",
|
||||
"modules_to_not_convert": [
|
||||
"lm_head",
|
||||
"model.layers.0.input_layernorm",
|
||||
"model.layers.0.mlp.gate",
|
||||
"model.layers.0.post_attention_layernorm",
|
||||
"model.layers.1.input_layernorm",
|
||||
"model.layers.1.mlp.gate",
|
||||
"model.layers.1.post_attention_layernorm",
|
||||
"model.layers.2.input_layernorm",
|
||||
"model.layers.2.mlp.gate",
|
||||
"model.layers.2.post_attention_layernorm",
|
||||
"model.layers.3.input_layernorm",
|
||||
"model.layers.3.mlp.gate",
|
||||
"model.layers.3.post_attention_layernorm",
|
||||
"model.layers.4.input_layernorm",
|
||||
"model.layers.4.mlp.gate",
|
||||
"model.layers.4.post_attention_layernorm",
|
||||
"model.layers.5.input_layernorm",
|
||||
"model.layers.5.mlp.gate",
|
||||
"model.layers.5.post_attention_layernorm",
|
||||
"model.layers.6.input_layernorm",
|
||||
"model.layers.6.mlp.gate",
|
||||
"model.layers.6.post_attention_layernorm",
|
||||
"model.layers.7.input_layernorm",
|
||||
"model.layers.7.mlp.gate",
|
||||
"model.layers.7.post_attention_layernorm",
|
||||
"model.layers.8.input_layernorm",
|
||||
"model.layers.8.mlp.gate",
|
||||
"model.layers.8.post_attention_layernorm",
|
||||
"model.layers.9.input_layernorm",
|
||||
"model.layers.9.mlp.gate",
|
||||
"model.layers.9.post_attention_layernorm",
|
||||
"model.layers.10.input_layernorm",
|
||||
"model.layers.10.mlp.gate",
|
||||
"model.layers.10.post_attention_layernorm",
|
||||
"model.layers.11.input_layernorm",
|
||||
"model.layers.11.mlp.gate",
|
||||
"model.layers.11.post_attention_layernorm",
|
||||
"model.layers.12.input_layernorm",
|
||||
"model.layers.12.mlp.gate",
|
||||
"model.layers.12.post_attention_layernorm",
|
||||
"model.layers.13.input_layernorm",
|
||||
"model.layers.13.mlp.gate",
|
||||
"model.layers.13.post_attention_layernorm",
|
||||
"model.layers.14.input_layernorm",
|
||||
"model.layers.14.mlp.gate",
|
||||
"model.layers.14.post_attention_layernorm",
|
||||
"model.layers.15.input_layernorm",
|
||||
"model.layers.15.mlp.gate",
|
||||
"model.layers.15.post_attention_layernorm",
|
||||
"model.layers.16.input_layernorm",
|
||||
"model.layers.16.mlp.gate",
|
||||
"model.layers.16.post_attention_layernorm",
|
||||
"model.layers.17.input_layernorm",
|
||||
"model.layers.17.mlp.gate",
|
||||
"model.layers.17.post_attention_layernorm",
|
||||
"model.layers.18.input_layernorm",
|
||||
"model.layers.18.mlp.gate",
|
||||
"model.layers.18.post_attention_layernorm",
|
||||
"model.layers.19.input_layernorm",
|
||||
"model.layers.19.mlp.gate",
|
||||
"model.layers.19.post_attention_layernorm",
|
||||
"model.layers.20.input_layernorm",
|
||||
"model.layers.20.mlp.gate",
|
||||
"model.layers.20.post_attention_layernorm",
|
||||
"model.layers.21.input_layernorm",
|
||||
"model.layers.21.mlp.gate",
|
||||
"model.layers.21.post_attention_layernorm",
|
||||
"model.layers.22.input_layernorm",
|
||||
"model.layers.22.mlp.gate",
|
||||
"model.layers.22.post_attention_layernorm",
|
||||
"model.layers.23.input_layernorm",
|
||||
"model.layers.23.mlp.gate",
|
||||
"model.layers.23.post_attention_layernorm",
|
||||
"model.layers.24.input_layernorm",
|
||||
"model.layers.24.mlp.gate",
|
||||
"model.layers.24.post_attention_layernorm",
|
||||
"model.layers.25.input_layernorm",
|
||||
"model.layers.25.mlp.gate",
|
||||
"model.layers.25.post_attention_layernorm",
|
||||
"model.layers.26.input_layernorm",
|
||||
"model.layers.26.mlp.gate",
|
||||
"model.layers.26.post_attention_layernorm",
|
||||
"model.layers.27.input_layernorm",
|
||||
"model.layers.27.mlp.gate",
|
||||
"model.layers.27.post_attention_layernorm",
|
||||
"model.layers.28.input_layernorm",
|
||||
"model.layers.28.mlp.gate",
|
||||
"model.layers.28.post_attention_layernorm",
|
||||
"model.layers.29.input_layernorm",
|
||||
"model.layers.29.mlp.gate",
|
||||
"model.layers.29.post_attention_layernorm",
|
||||
"model.layers.30.input_layernorm",
|
||||
"model.layers.30.mlp.gate",
|
||||
"model.layers.30.post_attention_layernorm",
|
||||
"model.layers.31.input_layernorm",
|
||||
"model.layers.31.mlp.gate",
|
||||
"model.layers.31.post_attention_layernorm",
|
||||
"model.layers.32.input_layernorm",
|
||||
"model.layers.32.mlp.gate",
|
||||
"model.layers.32.post_attention_layernorm",
|
||||
"model.layers.33.input_layernorm",
|
||||
"model.layers.33.mlp.gate",
|
||||
"model.layers.33.post_attention_layernorm",
|
||||
"model.layers.34.input_layernorm",
|
||||
"model.layers.34.mlp.gate",
|
||||
"model.layers.34.post_attention_layernorm",
|
||||
"model.layers.35.input_layernorm",
|
||||
"model.layers.35.mlp.gate",
|
||||
"model.layers.35.post_attention_layernorm",
|
||||
"model.layers.36.input_layernorm",
|
||||
"model.layers.36.mlp.gate",
|
||||
"model.layers.36.post_attention_layernorm",
|
||||
"model.layers.37.input_layernorm",
|
||||
"model.layers.37.mlp.gate",
|
||||
"model.layers.37.post_attention_layernorm",
|
||||
"model.layers.38.input_layernorm",
|
||||
"model.layers.38.mlp.gate",
|
||||
"model.layers.38.post_attention_layernorm",
|
||||
"model.layers.39.input_layernorm",
|
||||
"model.layers.39.mlp.gate",
|
||||
"model.layers.39.post_attention_layernorm",
|
||||
"model.layers.40.input_layernorm",
|
||||
"model.layers.40.mlp.gate",
|
||||
"model.layers.40.post_attention_layernorm",
|
||||
"model.layers.41.input_layernorm",
|
||||
"model.layers.41.mlp.gate",
|
||||
"model.layers.41.post_attention_layernorm",
|
||||
"model.layers.42.input_layernorm",
|
||||
"model.layers.42.mlp.gate",
|
||||
"model.layers.42.post_attention_layernorm",
|
||||
"model.layers.43.input_layernorm",
|
||||
"model.layers.43.mlp.gate",
|
||||
"model.layers.43.post_attention_layernorm",
|
||||
"model.layers.44.input_layernorm",
|
||||
"model.layers.44.mlp.gate",
|
||||
"model.layers.44.post_attention_layernorm",
|
||||
"model.layers.45.input_layernorm",
|
||||
"model.layers.45.mlp.gate",
|
||||
"model.layers.45.post_attention_layernorm",
|
||||
"model.layers.46.input_layernorm",
|
||||
"model.layers.46.mlp.gate",
|
||||
"model.layers.46.post_attention_layernorm",
|
||||
"model.layers.47.input_layernorm",
|
||||
"model.layers.47.mlp.gate",
|
||||
"model.layers.47.post_attention_layernorm",
|
||||
"model.layers.48.input_layernorm",
|
||||
"model.layers.48.mlp.gate",
|
||||
"model.layers.48.post_attention_layernorm",
|
||||
"model.layers.49.input_layernorm",
|
||||
"model.layers.49.mlp.gate",
|
||||
"model.layers.49.post_attention_layernorm",
|
||||
"model.layers.50.input_layernorm",
|
||||
"model.layers.50.mlp.gate",
|
||||
"model.layers.50.post_attention_layernorm",
|
||||
"model.layers.51.input_layernorm",
|
||||
"model.layers.51.mlp.gate",
|
||||
"model.layers.51.post_attention_layernorm",
|
||||
"model.layers.52.input_layernorm",
|
||||
"model.layers.52.mlp.gate",
|
||||
"model.layers.52.post_attention_layernorm",
|
||||
"model.layers.53.input_layernorm",
|
||||
"model.layers.53.mlp.gate",
|
||||
"model.layers.53.post_attention_layernorm",
|
||||
"model.layers.54.input_layernorm",
|
||||
"model.layers.54.mlp.gate",
|
||||
"model.layers.54.post_attention_layernorm",
|
||||
"model.layers.55.input_layernorm",
|
||||
"model.layers.55.mlp.gate",
|
||||
"model.layers.55.post_attention_layernorm",
|
||||
"model.layers.56.input_layernorm",
|
||||
"model.layers.56.mlp.gate",
|
||||
"model.layers.56.post_attention_layernorm",
|
||||
"model.layers.57.input_layernorm",
|
||||
"model.layers.57.mlp.gate",
|
||||
"model.layers.57.post_attention_layernorm",
|
||||
"model.layers.58.input_layernorm",
|
||||
"model.layers.58.mlp.gate",
|
||||
"model.layers.58.post_attention_layernorm",
|
||||
"model.layers.59.input_layernorm",
|
||||
"model.layers.59.mlp.gate",
|
||||
"model.layers.59.post_attention_layernorm",
|
||||
"model.layers.60.input_layernorm",
|
||||
"model.layers.60.mlp.gate",
|
||||
"model.layers.60.post_attention_layernorm",
|
||||
"model.layers.61.input_layernorm",
|
||||
"model.layers.61.mlp.gate",
|
||||
"model.layers.61.post_attention_layernorm",
|
||||
"model.layers.62.input_layernorm",
|
||||
"model.layers.62.mlp.gate",
|
||||
"model.layers.62.post_attention_layernorm",
|
||||
"model.layers.63.input_layernorm",
|
||||
"model.layers.63.mlp.gate",
|
||||
"model.layers.63.post_attention_layernorm",
|
||||
"model.layers.64.input_layernorm",
|
||||
"model.layers.64.mlp.gate",
|
||||
"model.layers.64.post_attention_layernorm",
|
||||
"model.layers.65.input_layernorm",
|
||||
"model.layers.65.mlp.gate",
|
||||
"model.layers.65.post_attention_layernorm",
|
||||
"model.layers.66.input_layernorm",
|
||||
"model.layers.66.mlp.gate",
|
||||
"model.layers.66.post_attention_layernorm",
|
||||
"model.layers.67.input_layernorm",
|
||||
"model.layers.67.mlp.gate",
|
||||
"model.layers.67.post_attention_layernorm",
|
||||
"model.layers.68.input_layernorm",
|
||||
"model.layers.68.mlp.gate",
|
||||
"model.layers.68.post_attention_layernorm",
|
||||
"model.layers.69.input_layernorm",
|
||||
"model.layers.69.mlp.gate",
|
||||
"model.layers.69.post_attention_layernorm",
|
||||
"model.layers.70.input_layernorm",
|
||||
"model.layers.70.mlp.gate",
|
||||
"model.layers.70.post_attention_layernorm",
|
||||
"model.layers.71.input_layernorm",
|
||||
"model.layers.71.mlp.gate",
|
||||
"model.layers.71.post_attention_layernorm",
|
||||
"model.layers.72.input_layernorm",
|
||||
"model.layers.72.mlp.gate",
|
||||
"model.layers.72.post_attention_layernorm",
|
||||
"model.layers.73.input_layernorm",
|
||||
"model.layers.73.mlp.gate",
|
||||
"model.layers.73.post_attention_layernorm",
|
||||
"model.layers.74.input_layernorm",
|
||||
"model.layers.74.mlp.gate",
|
||||
"model.layers.74.post_attention_layernorm",
|
||||
"model.layers.75.input_layernorm",
|
||||
"model.layers.75.mlp.gate",
|
||||
"model.layers.75.post_attention_layernorm",
|
||||
"model.layers.76.input_layernorm",
|
||||
"model.layers.76.mlp.gate",
|
||||
"model.layers.76.post_attention_layernorm",
|
||||
"model.layers.77.input_layernorm",
|
||||
"model.layers.77.mlp.gate",
|
||||
"model.layers.77.post_attention_layernorm",
|
||||
"model.layers.78.input_layernorm",
|
||||
"model.layers.78.mlp.gate",
|
||||
"model.layers.78.post_attention_layernorm",
|
||||
"model.layers.79.input_layernorm",
|
||||
"model.layers.79.mlp.gate",
|
||||
"model.layers.79.post_attention_layernorm",
|
||||
"model.layers.80.input_layernorm",
|
||||
"model.layers.80.mlp.gate",
|
||||
"model.layers.80.post_attention_layernorm",
|
||||
"model.layers.81.input_layernorm",
|
||||
"model.layers.81.mlp.gate",
|
||||
"model.layers.81.post_attention_layernorm",
|
||||
"model.layers.82.input_layernorm",
|
||||
"model.layers.82.mlp.gate",
|
||||
"model.layers.82.post_attention_layernorm",
|
||||
"model.layers.83.input_layernorm",
|
||||
"model.layers.83.mlp.gate",
|
||||
"model.layers.83.post_attention_layernorm",
|
||||
"model.layers.84.input_layernorm",
|
||||
"model.layers.84.mlp.gate",
|
||||
"model.layers.84.post_attention_layernorm",
|
||||
"model.layers.85.input_layernorm",
|
||||
"model.layers.85.mlp.gate",
|
||||
"model.layers.85.post_attention_layernorm",
|
||||
"model.layers.86.input_layernorm",
|
||||
"model.layers.86.mlp.gate",
|
||||
"model.layers.86.post_attention_layernorm",
|
||||
"model.layers.87.input_layernorm",
|
||||
"model.layers.87.mlp.gate",
|
||||
"model.layers.87.post_attention_layernorm",
|
||||
"model.layers.88.input_layernorm",
|
||||
"model.layers.88.mlp.gate",
|
||||
"model.layers.88.post_attention_layernorm",
|
||||
"model.layers.89.input_layernorm",
|
||||
"model.layers.89.mlp.gate",
|
||||
"model.layers.89.post_attention_layernorm",
|
||||
"model.layers.90.input_layernorm",
|
||||
"model.layers.90.mlp.gate",
|
||||
"model.layers.90.post_attention_layernorm",
|
||||
"model.layers.91.input_layernorm",
|
||||
"model.layers.91.mlp.gate",
|
||||
"model.layers.91.post_attention_layernorm",
|
||||
"model.layers.92.input_layernorm",
|
||||
"model.layers.92.mlp.gate",
|
||||
"model.layers.92.post_attention_layernorm",
|
||||
"model.layers.93.input_layernorm",
|
||||
"model.layers.93.mlp.gate",
|
||||
"model.layers.93.post_attention_layernorm"
|
||||
],
|
||||
"fmt": "e4m3",
|
||||
"quant_method": "fp8",
|
||||
"weight_block_size": [
|
||||
128,
|
||||
128
|
||||
],
|
||||
"is_checkpoint_fp8_serialized": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03094400092959404,0.052000001072883606,0.03423200035467744,0.032816000282764435,0.004586225105504425,0.05225599929690361,0.08137600123882294,0.06316960025578737,0.0586559996008873,0.009046516570964667,0.025087999179959297,0.05766399949789047,0.030939200054854156,0.028768000192940235,0.006844666670168347,0.23388800024986267,0.31091201305389404,0.24544477462768555,0.23836800456047058,0.018666831776499748,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030368000268936157,0.05008000135421753,0.03488799966871738,0.03270399942994118,0.004964435488742052,0.05142400041222572,0.07401599735021591,0.05652640014886856,0.054847998544573784,0.005118024227402908,0.02534399926662445,0.0424639992415905,0.02885119989514351,0.028447999618947506,0.004027125677475938,0.24774399399757385,0.26895999908447266,0.25432640314102173,0.25200000405311584,0.00582256680354476,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.0307839997112751,0.04447999969124794,0.03474240032956004,0.03252799995243549,0.00435773849976965,0.051231998950242996,0.07468800246715546,0.057651200145483014,0.0561280008405447,0.006050027441177822,0.02579200081527233,0.033504001796245575,0.02867360021919012,0.02796800062060356,0.00238178652420847,0.2533760070800781,0.2958720028400421,0.2585648000240326,0.25679999589920044,0.009045161306858063,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030592000111937523,0.04108799993991852,0.03223840007558465,0.031279999762773514,0.002451635813407811,0.05177599936723709,0.077504001557827,0.05718399975448847,0.0533440001308918,0.00711033940595454,0.02457600086927414,0.051552001386880875,0.03192960014566779,0.028431999497115612,0.007824328074264588,0.2531839907169342,0.28329598903656006,0.26060959696769714,0.2584640085697174,0.006803486030548811,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030527999624609947,0.03500799834728241,0.03184479987248778,0.03139200061559677,0.001123070738289876,0.05104000121355057,0.07103999704122543,0.05895199999213219,0.05676800012588501,0.0068221931474717916,0.024383999407291412,0.04211200028657913,0.02812959998846054,0.026959999464452267,0.003887262202895282,0.2314240038394928,0.2559039890766144,0.23696160316467285,0.23375999927520752,0.006377317477017641,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/aligned
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03232000023126602,0.07411199808120728,0.040055999718606475,0.03792000003159046,0.010027335937042725,0.05955199897289276,0.09151999652385712,0.06805919948965311,0.06542399898171425,0.009156033646696494,0.02735999971628189,0.04198399931192398,0.031628800183534624,0.03081599995493889,0.0038650835976484594,0.29440000653266907,0.37036800384521484,0.3045775890350342,0.3012160062789917,0.01622786745429039,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.03190400078892708,0.0639680027961731,0.036531200259923936,0.03387199901044369,0.007307958553439168,0.05407999828457832,0.08684799820184708,0.0631104001775384,0.059248000383377075,0.008437822166776614,0.02537599951028824,0.045343998819589615,0.029841599892824887,0.028095999732613564,0.004578510902843549,0.29868799448013306,0.3216319978237152,0.30579519271850586,0.30294400453567505,0.006667278707027435,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.031199999153614044,0.06745599955320358,0.03718719966709614,0.03391999937593937,0.008084069620200455,0.05353600159287453,0.06828799843788147,0.059427200257778166,0.05902400054037571,0.0036973349098869714,0.026688000187277794,0.03788800165057182,0.030459199845790864,0.02908799983561039,0.0032231049972104124,0.30588799715042114,0.3216319978237152,0.3109855651855469,0.31036800146102905,0.004134960938245058,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.031488001346588135,0.04790399968624115,0.03488959986716509,0.033615998923778534,0.003995903359891917,0.05503999814391136,0.08563199639320374,0.06150399968028068,0.057312000542879105,0.007950341155323642,0.026335999369621277,0.04598399996757507,0.030313600040972232,0.02792000025510788,0.005356499011224523,0.31516799330711365,0.32972800731658936,0.3209056258201599,0.3196159899234772,0.004338567610830069,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030751999467611313,0.044895999133586884,0.03544640000909567,0.03302400000393391,0.00457756723742277,0.05331199988722801,0.07606399804353714,0.05961279980838299,0.05753600038588047,0.006503363955077464,0.02630399912595749,0.04368000105023384,0.03112160013988614,0.028815999627113342,0.005258638278682459,0.2905920147895813,0.3538239896297455,0.3015664219856262,0.29631999135017395,0.014866933226585388,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/compute_type
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.029888000339269638,0.05558399856090546,0.036182400118559596,0.03598400019109249,0.00590375348379892,0.05135999992489815,0.07407999783754349,0.05939359981566668,0.05599999986588955,0.007274243962538696,0.024831999093294144,0.060575999319553375,0.03204159988090396,0.029280000366270542,0.008986441399241034,0.23401600122451782,0.2914240062236786,0.24253761768341064,0.23852799832820892,0.012656064704060555,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.0306560005992651,0.04927999898791313,0.0345616003498435,0.032096000388264656,0.005272357401818311,0.0514880008995533,0.08454400300979614,0.05950720049440861,0.055904000997543335,0.008599599958802731,0.024639999493956566,0.04726399853825569,0.029726399946957825,0.028032000176608562,0.005891950556036845,0.24633599817752838,0.276095986366272,0.2544096112251282,0.25091201066970825,0.006771203130483627,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030208000913262367,0.062144000083208084,0.03510080017149449,0.03203200176358223,0.007286512102806094,0.050592001527547836,0.07017599791288376,0.05696159955114126,0.054735999554395676,0.006055532934843585,0.0244159996509552,0.04364800080657005,0.02875520009547472,0.02700799982994795,0.00429941663275523,0.252703994512558,0.27452799677848816,0.25873440504074097,0.2567040026187897,0.006180537864565849,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.028863999992609024,0.04185599833726883,0.03244960019364953,0.03129600081592798,0.00309197601199972,0.05049600079655647,0.07241600006818771,0.05591519977897406,0.05273599922657013,0.006825017255220366,0.024000000208616257,0.04150399938225746,0.027609600126743315,0.02619200013577938,0.0041488046270812895,0.25491198897361755,0.27008000016212463,0.2593088150024414,0.25811201333999634,0.003932233899831772,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030400000512599945,0.039135999977588654,0.03235360030084848,0.031888000667095184,0.002007792126883597,0.050944000482559204,0.06815999746322632,0.054641599953174594,0.053279999643564224,0.003969876178544029,0.024064000695943832,0.03145600110292435,0.027134399861097336,0.026559999212622643,0.0021875101737656014,0.23164799809455872,0.2622720003128052,0.2388928234577179,0.23686400055885315,0.007063580676913261,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/config_key
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,6 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03094400092959404,0.05427199974656105,0.03751839986070991,0.03566399961709976,0.0063355673062837495,0.052960000932216644,0.10150399804115295,0.06679840013384819,0.06265599839389324,0.012010412593353396,0.025087999179959297,0.0607680007815361,0.03164320001378655,0.028655999340116978,0.008879480144565048,0.29548799991607666,0.3161599934101105,0.30511200428009033,0.30371201038360596,0.006655826233327389,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030368000268936157,0.045184001326560974,0.03435200024396181,0.03299199976027012,0.003866832846889978,0.053247999399900436,0.10054399818181992,0.061161600053310394,0.05702400021255016,0.010951609505981603,0.025248000398278236,0.039872001856565475,0.029195200372487306,0.027520000003278255,0.003716525371772022,0.29631999135017395,0.3163839876651764,0.30516156554222107,0.30246400833129883,0.0062532913871109486,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.6171875,0.0,0.9682458365518543,4.0,6.171569533299451,0.521240234375,uniform,1,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030688000842928886,0.04044799879193306,0.03336800048127771,0.032368000596761703,0.002819613112867307,0.053568001836538315,0.06889600306749344,0.05800320040434599,0.055616000667214394,0.00500911399891226,0.025631999596953392,0.040991999208927155,0.028961599990725517,0.027951999567449093,0.003321893930324548,0.30828800797462463,0.3516480028629303,0.3159376084804535,0.3127039968967438,0.009800842963159084,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.640625,0.0,0.9842509842514764,4.0,6.193092091810443,0.5164794921875,uniform,2,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.031007999554276466,0.044319998472929,0.03418559962883592,0.03271999955177307,0.0038270996330861703,0.0541439987719059,0.07507199794054031,0.06345439981669188,0.06393599882721901,0.006346258200489106,0.025407999753952026,0.03494400158524513,0.028180800192058087,0.027375999838113785,0.0024874374625372497,0.3179520070552826,0.331712007522583,0.3228943943977356,0.32150399684906006,0.004389750771224499,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.671875,0.0,0.9354143466934853,5.0,6.268369216857352,0.48486328125,uniform,3,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
0.030912000685930252,0.04745600000023842,0.03517600009217858,0.03243200108408928,0.005035846835878437,0.053279999643564224,0.09200000017881393,0.06018720027059317,0.05721599981188774,0.008845081624923161,0.025151999667286873,0.04851200059056282,0.0323488000780344,0.02820800058543682,0.0077821073314592575,0.2917119860649109,0.3189440071582794,0.301118403673172,0.29817599058151245,0.008174914866685867,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.59375,0.0,1.1319231422671772,6.0,6.039891775809659,0.5689697265625,uniform,4,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 5
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/original
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,5 @@
|
||||
time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type
|
||||
0.010463999584317207,0.04681599885225296,0.018502399697899817,0.01104000024497509,0.014205381674858167,0.021247999742627144,0.033055998384952545,0.025439999625086786,0.024639999493956566,0.004202660707027564,0.05206400156021118,0.09040000289678574,0.0665344014763832,0.06451199948787689,0.013110543934264527,0.007327999919652939,0.011680000461637974,0.008480000123381615,0.0077760000713169575,0.0016149067494504018,0.009472000412642956,0.011103999800980091,0.010118400119245053,0.009600000455975533,0.0007061491557578713,4096,64,4,16,4,40960,1,128,0,True,FLASHINFER,False,even,[128],128,128,128,128.0,128,0.0,0.0,0.0,False,0,128,128,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
0.010751999914646149,0.012896000407636166,0.011795200034976005,0.011359999887645245,0.0009191526000700493,0.020191999152302742,0.02755199931561947,0.02319999970495701,0.021247999742627144,0.002944486774412461,0.049215998500585556,0.06700800359249115,0.05813760012388229,0.058240000158548355,0.006895964777123649,0.007519999984651804,0.009727999567985535,0.008140799775719642,0.007807999849319458,0.0008127242879711131,0.009472000412642956,0.025248000398278236,0.01303040012717247,0.010015999898314476,0.0061233119538994085,4096,64,4,16,4,40960,1,32,0,True,FLASHINFER,False,even,[32],32,32,32,32.0,32,0.0,0.0,0.0,False,0,32,32,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
0.010239999741315842,0.011008000001311302,0.010515199974179268,0.010432000271975994,0.0002635681936991548,0.02112000063061714,0.03843199834227562,0.025894399732351303,0.023231999948620796,0.006378760808481684,0.05417599901556969,0.06438399851322174,0.05709439888596535,0.055615998804569244,0.003697870830170538,0.007135999854654074,0.00774399982765317,0.007500800024718046,0.007552000228315592,0.0002009640969704077,0.009279999881982803,0.012736000120639801,0.01031040009111166,0.009568000212311745,0.0012940667684018582,4096,64,4,16,4,40960,1,64,0,True,FLASHINFER,False,even,[64],64,64,64,64.0,64,0.0,0.0,0.0,False,0,64,64,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
0.010367999784648418,0.02332800067961216,0.013459200039505959,0.010847999714314938,0.004984552697426283,0.02054399996995926,0.0261439997702837,0.02255360037088394,0.022112000733613968,0.001928344367854761,0.04851200059056282,0.07968000322580338,0.058873600512743,0.056352000683546066,0.010813603870069636,0.00723200011998415,0.008352000266313553,0.007686400134116411,0.007648000027984381,0.00039524862182609345,0.008960000239312649,0.011103999800980091,0.009881599992513656,0.00940799992531538,0.0009090251198168286,4096,64,4,16,4,40960,1,96,0,True,FLASHINFER,False,even,[96],96,96,96,96.0,96,0.0,0.0,0.0,False,0,96,96,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT
|
||||
|
@@ -0,0 +1,5 @@
|
||||
time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type,is_true_mixed_batch
|
||||
0.010463999584317207,0.04681599885225296,0.018502399697899817,0.01104000024497509,0.014205381674858167,0.021247999742627144,0.033055998384952545,0.025439999625086786,0.024639999493956566,0.004202660707027564,0.05206400156021118,0.09040000289678574,0.0665344014763832,0.06451199948787689,0.013110543934264527,0.007327999919652939,0.011680000461637974,0.008480000123381615,0.0077760000713169575,0.0016149067494504018,0.009472000412642956,0.011103999800980091,0.010118400119245053,0.009600000455975533,0.0007061491557578713,4096,64,4,16,4,40960,1,128,0,True,FLASHINFER,False,even,[128],128,128,128,128.0,128,0.0,0.0,0.0,False,0,128,128,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
0.010751999914646149,0.012896000407636166,0.011795200034976005,0.011359999887645245,0.0009191526000700493,0.020191999152302742,0.02755199931561947,0.02319999970495701,0.021247999742627144,0.002944486774412461,0.049215998500585556,0.06700800359249115,0.05813760012388229,0.058240000158548355,0.006895964777123649,0.007519999984651804,0.009727999567985535,0.008140799775719642,0.007807999849319458,0.0008127242879711131,0.009472000412642956,0.025248000398278236,0.01303040012717247,0.010015999898314476,0.0061233119538994085,4096,64,4,16,4,40960,1,32,0,True,FLASHINFER,False,even,[32],32,32,32,32.0,32,0.0,0.0,0.0,False,0,32,32,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
0.010239999741315842,0.011008000001311302,0.010515199974179268,0.010432000271975994,0.0002635681936991548,0.02112000063061714,0.03843199834227562,0.025894399732351303,0.023231999948620796,0.006378760808481684,0.05417599901556969,0.06438399851322174,0.05709439888596535,0.055615998804569244,0.003697870830170538,0.007135999854654074,0.00774399982765317,0.007500800024718046,0.007552000228315592,0.0002009640969704077,0.009279999881982803,0.012736000120639801,0.01031040009111166,0.009568000212311745,0.0012940667684018582,4096,64,4,16,4,40960,1,64,0,True,FLASHINFER,False,even,[64],64,64,64,64.0,64,0.0,0.0,0.0,False,0,64,64,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
0.010367999784648418,0.02332800067961216,0.013459200039505959,0.010847999714314938,0.004984552697426283,0.02054399996995926,0.0261439997702837,0.02255360037088394,0.022112000733613968,0.001928344367854761,0.04851200059056282,0.07968000322580338,0.058873600512743,0.056352000683546066,0.010813603870069636,0.00723200011998415,0.008352000266313553,0.007686400134116411,0.007648000027984381,0.00039524862182609345,0.008960000239312649,0.011103999800980091,0.009881599992513656,0.00940799992531538,0.0009090251198168286,4096,64,4,16,4,40960,1,96,0,True,FLASHINFER,False,even,[96],96,96,96,96.0,96,0.0,0.0,0.0,False,0,96,96,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128,CUDA_EVENT,False
|
||||
|
@@ -0,0 +1,3 @@
|
||||
time_stats.attn_pre_proj.min,time_stats.attn_pre_proj.max,time_stats.attn_pre_proj.mean,time_stats.attn_pre_proj.median,time_stats.attn_pre_proj.std,time_stats.attn_rope.min,time_stats.attn_rope.max,time_stats.attn_rope.mean,time_stats.attn_rope.median,time_stats.attn_rope.std,time_stats.attn_post_proj.min,time_stats.attn_post_proj.max,time_stats.attn_post_proj.mean,time_stats.attn_post_proj.median,time_stats.attn_post_proj.std,time_stats.emb.min,time_stats.emb.max,time_stats.emb.mean,time_stats.emb.median,time_stats.emb.std,time_stats.input_layernorm.min,time_stats.input_layernorm.max,time_stats.input_layernorm.mean,time_stats.input_layernorm.median,time_stats.input_layernorm.std,time_stats.post_attention_layernorm.min,time_stats.post_attention_layernorm.max,time_stats.post_attention_layernorm.mean,time_stats.post_attention_layernorm.median,time_stats.post_attention_layernorm.std,n_head,n_kv_head,n_embd,n_expanded_embd,vocab_size,use_gated_mlp,use_qk_norm,attn_output_gate,num_tokens,num_tensor_parallel_workers,padded_n_embd,padded_n_expanded_embd,model_arch,is_step2_mini,share_expert_dim,share_q_dim,measurement_type,profiling_precision,quant_signature
|
||||
0.19225600361824036,0.26678401231765747,0.22318400144577027,0.22147200256586075,0.020290217906394733,0.024639999493956566,0.04320000112056732,0.028563199937343596,0.026335999369621277,0.006067654243700026,0.09071999788284302,0.12992000579833984,0.11053120009601117,0.11033599823713303,0.011489030217015594,,,,,,,,,,,,,,,,64,4,4096,1536,151936,True,True,False,16,4,4096,1536,generic,False,,,CUDA_EVENT,BF16,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
,,,,,,,,,,,,,,,0.05990400165319443,0.1345279961824417,0.08581680012866855,0.0899839997291565,0.020920650895404905,0.020160000771284103,0.0360959991812706,0.02369120018556714,0.021824000403285027,0.00439515404502141,0.01833599992096424,0.02425600029528141,0.020108799915760756,0.019407999701797962,0.001409577392533308,64,4,4096,1536,151936,True,True,False,16,1,4096,1536,generic,False,,,CUDA_EVENT,BF16,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,2 @@
|
||||
time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature
|
||||
0.03097599931061268,0.049056001007556915,0.03467839974910021,0.03254400007426739,0.005093522706269737,0.05193600058555603,0.08419200032949448,0.06054240055382252,0.05641600117087364,0.009051489911083033,0.025919999927282333,0.04064000025391579,0.030527999717742206,0.030608000233769417,0.0040231266205605675,0.29603201150894165,0.3494400084018707,0.3075023889541626,0.30137598514556885,0.014090820215642452,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,prefill_hot,ffn_like_prefix_20x,8,4096,1536,True,4,128,0.375,1.0,1.0,0.609375,0.0,1.0231690964840563,4.0,6.122626857503489,0.5433349609375,uniform,0,vllm_fused,CUDA_EVENT,BF16,generic,method=fp8|act=dynamic|serialized=True|block=128x128
|
||||
|
@@ -0,0 +1,65 @@
|
||||
attention_backend: FLASHINFER
|
||||
batch_size_list:
|
||||
- 1
|
||||
block_shape: null
|
||||
block_size: 16
|
||||
decode_kv_cache_size_list: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_chunked_prefill_grid_search: false
|
||||
enable_mixed_prefill: false
|
||||
enable_true_mixed: false
|
||||
fixed_chunked_prefill_size: 128
|
||||
max_batch_size: 1
|
||||
max_mixed_batch_size: 8
|
||||
max_model_len: 40960
|
||||
max_pipeline_parallel_size: 1
|
||||
max_seq_len: 128
|
||||
min_batch_size: 1
|
||||
mixed_batch_size_list: null
|
||||
mixed_batch_size_max: 32
|
||||
mixed_batch_size_min: 2
|
||||
mixed_kv_cache_size_list:
|
||||
- 0
|
||||
mixed_mode: both
|
||||
mixed_num_samples: 3
|
||||
mixed_profile_strategy: default
|
||||
mixed_shapes_per_point: 2
|
||||
mixed_total_tokens_list: null
|
||||
mixed_total_tokens_max: 1055
|
||||
mixed_total_tokens_min: 1025
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
profile_only_decode: false
|
||||
profile_only_prefill: true
|
||||
skip_confirmation: true
|
||||
true_mixed_decode_batch_sizes:
|
||||
- 1
|
||||
- 2
|
||||
- 4
|
||||
- 8
|
||||
true_mixed_decode_kv_cache_sizes:
|
||||
- 128
|
||||
- 256
|
||||
- 512
|
||||
- 1024
|
||||
- 2048
|
||||
true_mixed_prefill_batch_sizes:
|
||||
- 1
|
||||
- 2
|
||||
- 4
|
||||
true_mixed_prefill_chunk_sizes:
|
||||
- 64
|
||||
- 128
|
||||
- 256
|
||||
- 512
|
||||
- 1024
|
||||
true_mixed_prefill_kv_cache_size: 0
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,23 @@
|
||||
attn_tp: null
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
extra_num_tokens: null
|
||||
ffn_tp: null
|
||||
include_target_embedded_mtp: false
|
||||
is_moe: true
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
ray_enable_datasets_serializers: false
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,27 @@
|
||||
block_shape: null
|
||||
device: h20
|
||||
disable_ray: true
|
||||
disable_replicated: false
|
||||
enable_load_imbalance: true
|
||||
expert_parallel_sizes:
|
||||
- 1
|
||||
extra_num_tokens: null
|
||||
gating_runtime_context: prefill_hot
|
||||
load_distributions:
|
||||
- uniform
|
||||
max_tokens: 16
|
||||
models:
|
||||
- Qwen3-235B-A22B-FP8
|
||||
num_gpus: 1
|
||||
num_samples_per_distribution: 1
|
||||
num_tensor_parallel_workers:
|
||||
- 4
|
||||
num_tokens_list:
|
||||
- 16
|
||||
output_dir: /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles
|
||||
per_channel_quant: false
|
||||
precision: null
|
||||
profile_method: cuda_event
|
||||
routing_runtime_path: standard_fused_topk
|
||||
skip_confirmation: true
|
||||
use_fp8: null
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"flashinfer_metadata_version": "0.3.1.post1",
|
||||
"flashinfer_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/flashinfer/__init__.py",
|
||||
"frontier_metadata_version": "0.1.0",
|
||||
"frontier_path": "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6/frontier",
|
||||
"python": "3.12.3",
|
||||
"torch": "2.8.0+cu128",
|
||||
"torch_cuda": "12.8",
|
||||
"vllm_import_version": "0.10.2",
|
||||
"vllm_metadata_version": "0.10.2",
|
||||
"vllm_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/vllm/__init__.py"
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"flashinfer-python": "0.3.1.post1",
|
||||
"frontier-simulator": "0.1.0",
|
||||
"python": "3.12.3",
|
||||
"tokenizers": "0.21.4",
|
||||
"torch": "2.8.0",
|
||||
"transformers": "4.55.2",
|
||||
"vllm": "0.10.2"
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
5184cb3f458685616364a09db28eee1603af26d58894073428a67ab677d655f5 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/environment.post_transformers_pin.json
|
||||
2d17f91ca22ed56dbffc4549972814be911901f321963f8f07a2b940df191340 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/requirements.freeze.txt
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"flashinfer_metadata_version": "0.3.1.post1",
|
||||
"flashinfer_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/flashinfer/__init__.py",
|
||||
"frontier_metadata_version": "0.1.0",
|
||||
"frontier_path": "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6/frontier",
|
||||
"python": "3.12.3",
|
||||
"torch": "2.8.0+cu128",
|
||||
"torch_cuda": "12.8",
|
||||
"vllm_import_version": "0.10.2",
|
||||
"vllm_metadata_version": "0.10.2",
|
||||
"vllm_path": "/tmp/wjh-frontier-vllm0102-smoke/.venv/lib/python3.12/site-packages/vllm/__init__.py"
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
8564395327d10aff62c48e0e4fb74bdd050d17faed3d8d3948ff2e96e767d03a /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/environment.json
|
||||
a931219d3038697371775d2e5d797321adefc4ca48da2ab5999e1206e17a3881 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/requirements.freeze.txt
|
||||
@@ -0,0 +1,332 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3MoeForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"decoder_sparse_step": 1,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 12288,
|
||||
"max_position_embeddings": 40960,
|
||||
"max_window_layers": 94,
|
||||
"mlp_only_layers": [],
|
||||
"model_type": "qwen3_moe",
|
||||
"moe_intermediate_size": 1536,
|
||||
"norm_topk_prob": true,
|
||||
"num_attention_heads": 64,
|
||||
"num_experts": 128,
|
||||
"num_experts_per_tok": 8,
|
||||
"num_hidden_layers": 94,
|
||||
"num_key_value_heads": 4,
|
||||
"output_router_logits": false,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000.0,
|
||||
"router_aux_loss_coef": 0.001,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.0",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936,
|
||||
"quantization_config": {
|
||||
"activation_scheme": "dynamic",
|
||||
"modules_to_not_convert": [
|
||||
"lm_head",
|
||||
"model.layers.0.input_layernorm",
|
||||
"model.layers.0.mlp.gate",
|
||||
"model.layers.0.post_attention_layernorm",
|
||||
"model.layers.1.input_layernorm",
|
||||
"model.layers.1.mlp.gate",
|
||||
"model.layers.1.post_attention_layernorm",
|
||||
"model.layers.2.input_layernorm",
|
||||
"model.layers.2.mlp.gate",
|
||||
"model.layers.2.post_attention_layernorm",
|
||||
"model.layers.3.input_layernorm",
|
||||
"model.layers.3.mlp.gate",
|
||||
"model.layers.3.post_attention_layernorm",
|
||||
"model.layers.4.input_layernorm",
|
||||
"model.layers.4.mlp.gate",
|
||||
"model.layers.4.post_attention_layernorm",
|
||||
"model.layers.5.input_layernorm",
|
||||
"model.layers.5.mlp.gate",
|
||||
"model.layers.5.post_attention_layernorm",
|
||||
"model.layers.6.input_layernorm",
|
||||
"model.layers.6.mlp.gate",
|
||||
"model.layers.6.post_attention_layernorm",
|
||||
"model.layers.7.input_layernorm",
|
||||
"model.layers.7.mlp.gate",
|
||||
"model.layers.7.post_attention_layernorm",
|
||||
"model.layers.8.input_layernorm",
|
||||
"model.layers.8.mlp.gate",
|
||||
"model.layers.8.post_attention_layernorm",
|
||||
"model.layers.9.input_layernorm",
|
||||
"model.layers.9.mlp.gate",
|
||||
"model.layers.9.post_attention_layernorm",
|
||||
"model.layers.10.input_layernorm",
|
||||
"model.layers.10.mlp.gate",
|
||||
"model.layers.10.post_attention_layernorm",
|
||||
"model.layers.11.input_layernorm",
|
||||
"model.layers.11.mlp.gate",
|
||||
"model.layers.11.post_attention_layernorm",
|
||||
"model.layers.12.input_layernorm",
|
||||
"model.layers.12.mlp.gate",
|
||||
"model.layers.12.post_attention_layernorm",
|
||||
"model.layers.13.input_layernorm",
|
||||
"model.layers.13.mlp.gate",
|
||||
"model.layers.13.post_attention_layernorm",
|
||||
"model.layers.14.input_layernorm",
|
||||
"model.layers.14.mlp.gate",
|
||||
"model.layers.14.post_attention_layernorm",
|
||||
"model.layers.15.input_layernorm",
|
||||
"model.layers.15.mlp.gate",
|
||||
"model.layers.15.post_attention_layernorm",
|
||||
"model.layers.16.input_layernorm",
|
||||
"model.layers.16.mlp.gate",
|
||||
"model.layers.16.post_attention_layernorm",
|
||||
"model.layers.17.input_layernorm",
|
||||
"model.layers.17.mlp.gate",
|
||||
"model.layers.17.post_attention_layernorm",
|
||||
"model.layers.18.input_layernorm",
|
||||
"model.layers.18.mlp.gate",
|
||||
"model.layers.18.post_attention_layernorm",
|
||||
"model.layers.19.input_layernorm",
|
||||
"model.layers.19.mlp.gate",
|
||||
"model.layers.19.post_attention_layernorm",
|
||||
"model.layers.20.input_layernorm",
|
||||
"model.layers.20.mlp.gate",
|
||||
"model.layers.20.post_attention_layernorm",
|
||||
"model.layers.21.input_layernorm",
|
||||
"model.layers.21.mlp.gate",
|
||||
"model.layers.21.post_attention_layernorm",
|
||||
"model.layers.22.input_layernorm",
|
||||
"model.layers.22.mlp.gate",
|
||||
"model.layers.22.post_attention_layernorm",
|
||||
"model.layers.23.input_layernorm",
|
||||
"model.layers.23.mlp.gate",
|
||||
"model.layers.23.post_attention_layernorm",
|
||||
"model.layers.24.input_layernorm",
|
||||
"model.layers.24.mlp.gate",
|
||||
"model.layers.24.post_attention_layernorm",
|
||||
"model.layers.25.input_layernorm",
|
||||
"model.layers.25.mlp.gate",
|
||||
"model.layers.25.post_attention_layernorm",
|
||||
"model.layers.26.input_layernorm",
|
||||
"model.layers.26.mlp.gate",
|
||||
"model.layers.26.post_attention_layernorm",
|
||||
"model.layers.27.input_layernorm",
|
||||
"model.layers.27.mlp.gate",
|
||||
"model.layers.27.post_attention_layernorm",
|
||||
"model.layers.28.input_layernorm",
|
||||
"model.layers.28.mlp.gate",
|
||||
"model.layers.28.post_attention_layernorm",
|
||||
"model.layers.29.input_layernorm",
|
||||
"model.layers.29.mlp.gate",
|
||||
"model.layers.29.post_attention_layernorm",
|
||||
"model.layers.30.input_layernorm",
|
||||
"model.layers.30.mlp.gate",
|
||||
"model.layers.30.post_attention_layernorm",
|
||||
"model.layers.31.input_layernorm",
|
||||
"model.layers.31.mlp.gate",
|
||||
"model.layers.31.post_attention_layernorm",
|
||||
"model.layers.32.input_layernorm",
|
||||
"model.layers.32.mlp.gate",
|
||||
"model.layers.32.post_attention_layernorm",
|
||||
"model.layers.33.input_layernorm",
|
||||
"model.layers.33.mlp.gate",
|
||||
"model.layers.33.post_attention_layernorm",
|
||||
"model.layers.34.input_layernorm",
|
||||
"model.layers.34.mlp.gate",
|
||||
"model.layers.34.post_attention_layernorm",
|
||||
"model.layers.35.input_layernorm",
|
||||
"model.layers.35.mlp.gate",
|
||||
"model.layers.35.post_attention_layernorm",
|
||||
"model.layers.36.input_layernorm",
|
||||
"model.layers.36.mlp.gate",
|
||||
"model.layers.36.post_attention_layernorm",
|
||||
"model.layers.37.input_layernorm",
|
||||
"model.layers.37.mlp.gate",
|
||||
"model.layers.37.post_attention_layernorm",
|
||||
"model.layers.38.input_layernorm",
|
||||
"model.layers.38.mlp.gate",
|
||||
"model.layers.38.post_attention_layernorm",
|
||||
"model.layers.39.input_layernorm",
|
||||
"model.layers.39.mlp.gate",
|
||||
"model.layers.39.post_attention_layernorm",
|
||||
"model.layers.40.input_layernorm",
|
||||
"model.layers.40.mlp.gate",
|
||||
"model.layers.40.post_attention_layernorm",
|
||||
"model.layers.41.input_layernorm",
|
||||
"model.layers.41.mlp.gate",
|
||||
"model.layers.41.post_attention_layernorm",
|
||||
"model.layers.42.input_layernorm",
|
||||
"model.layers.42.mlp.gate",
|
||||
"model.layers.42.post_attention_layernorm",
|
||||
"model.layers.43.input_layernorm",
|
||||
"model.layers.43.mlp.gate",
|
||||
"model.layers.43.post_attention_layernorm",
|
||||
"model.layers.44.input_layernorm",
|
||||
"model.layers.44.mlp.gate",
|
||||
"model.layers.44.post_attention_layernorm",
|
||||
"model.layers.45.input_layernorm",
|
||||
"model.layers.45.mlp.gate",
|
||||
"model.layers.45.post_attention_layernorm",
|
||||
"model.layers.46.input_layernorm",
|
||||
"model.layers.46.mlp.gate",
|
||||
"model.layers.46.post_attention_layernorm",
|
||||
"model.layers.47.input_layernorm",
|
||||
"model.layers.47.mlp.gate",
|
||||
"model.layers.47.post_attention_layernorm",
|
||||
"model.layers.48.input_layernorm",
|
||||
"model.layers.48.mlp.gate",
|
||||
"model.layers.48.post_attention_layernorm",
|
||||
"model.layers.49.input_layernorm",
|
||||
"model.layers.49.mlp.gate",
|
||||
"model.layers.49.post_attention_layernorm",
|
||||
"model.layers.50.input_layernorm",
|
||||
"model.layers.50.mlp.gate",
|
||||
"model.layers.50.post_attention_layernorm",
|
||||
"model.layers.51.input_layernorm",
|
||||
"model.layers.51.mlp.gate",
|
||||
"model.layers.51.post_attention_layernorm",
|
||||
"model.layers.52.input_layernorm",
|
||||
"model.layers.52.mlp.gate",
|
||||
"model.layers.52.post_attention_layernorm",
|
||||
"model.layers.53.input_layernorm",
|
||||
"model.layers.53.mlp.gate",
|
||||
"model.layers.53.post_attention_layernorm",
|
||||
"model.layers.54.input_layernorm",
|
||||
"model.layers.54.mlp.gate",
|
||||
"model.layers.54.post_attention_layernorm",
|
||||
"model.layers.55.input_layernorm",
|
||||
"model.layers.55.mlp.gate",
|
||||
"model.layers.55.post_attention_layernorm",
|
||||
"model.layers.56.input_layernorm",
|
||||
"model.layers.56.mlp.gate",
|
||||
"model.layers.56.post_attention_layernorm",
|
||||
"model.layers.57.input_layernorm",
|
||||
"model.layers.57.mlp.gate",
|
||||
"model.layers.57.post_attention_layernorm",
|
||||
"model.layers.58.input_layernorm",
|
||||
"model.layers.58.mlp.gate",
|
||||
"model.layers.58.post_attention_layernorm",
|
||||
"model.layers.59.input_layernorm",
|
||||
"model.layers.59.mlp.gate",
|
||||
"model.layers.59.post_attention_layernorm",
|
||||
"model.layers.60.input_layernorm",
|
||||
"model.layers.60.mlp.gate",
|
||||
"model.layers.60.post_attention_layernorm",
|
||||
"model.layers.61.input_layernorm",
|
||||
"model.layers.61.mlp.gate",
|
||||
"model.layers.61.post_attention_layernorm",
|
||||
"model.layers.62.input_layernorm",
|
||||
"model.layers.62.mlp.gate",
|
||||
"model.layers.62.post_attention_layernorm",
|
||||
"model.layers.63.input_layernorm",
|
||||
"model.layers.63.mlp.gate",
|
||||
"model.layers.63.post_attention_layernorm",
|
||||
"model.layers.64.input_layernorm",
|
||||
"model.layers.64.mlp.gate",
|
||||
"model.layers.64.post_attention_layernorm",
|
||||
"model.layers.65.input_layernorm",
|
||||
"model.layers.65.mlp.gate",
|
||||
"model.layers.65.post_attention_layernorm",
|
||||
"model.layers.66.input_layernorm",
|
||||
"model.layers.66.mlp.gate",
|
||||
"model.layers.66.post_attention_layernorm",
|
||||
"model.layers.67.input_layernorm",
|
||||
"model.layers.67.mlp.gate",
|
||||
"model.layers.67.post_attention_layernorm",
|
||||
"model.layers.68.input_layernorm",
|
||||
"model.layers.68.mlp.gate",
|
||||
"model.layers.68.post_attention_layernorm",
|
||||
"model.layers.69.input_layernorm",
|
||||
"model.layers.69.mlp.gate",
|
||||
"model.layers.69.post_attention_layernorm",
|
||||
"model.layers.70.input_layernorm",
|
||||
"model.layers.70.mlp.gate",
|
||||
"model.layers.70.post_attention_layernorm",
|
||||
"model.layers.71.input_layernorm",
|
||||
"model.layers.71.mlp.gate",
|
||||
"model.layers.71.post_attention_layernorm",
|
||||
"model.layers.72.input_layernorm",
|
||||
"model.layers.72.mlp.gate",
|
||||
"model.layers.72.post_attention_layernorm",
|
||||
"model.layers.73.input_layernorm",
|
||||
"model.layers.73.mlp.gate",
|
||||
"model.layers.73.post_attention_layernorm",
|
||||
"model.layers.74.input_layernorm",
|
||||
"model.layers.74.mlp.gate",
|
||||
"model.layers.74.post_attention_layernorm",
|
||||
"model.layers.75.input_layernorm",
|
||||
"model.layers.75.mlp.gate",
|
||||
"model.layers.75.post_attention_layernorm",
|
||||
"model.layers.76.input_layernorm",
|
||||
"model.layers.76.mlp.gate",
|
||||
"model.layers.76.post_attention_layernorm",
|
||||
"model.layers.77.input_layernorm",
|
||||
"model.layers.77.mlp.gate",
|
||||
"model.layers.77.post_attention_layernorm",
|
||||
"model.layers.78.input_layernorm",
|
||||
"model.layers.78.mlp.gate",
|
||||
"model.layers.78.post_attention_layernorm",
|
||||
"model.layers.79.input_layernorm",
|
||||
"model.layers.79.mlp.gate",
|
||||
"model.layers.79.post_attention_layernorm",
|
||||
"model.layers.80.input_layernorm",
|
||||
"model.layers.80.mlp.gate",
|
||||
"model.layers.80.post_attention_layernorm",
|
||||
"model.layers.81.input_layernorm",
|
||||
"model.layers.81.mlp.gate",
|
||||
"model.layers.81.post_attention_layernorm",
|
||||
"model.layers.82.input_layernorm",
|
||||
"model.layers.82.mlp.gate",
|
||||
"model.layers.82.post_attention_layernorm",
|
||||
"model.layers.83.input_layernorm",
|
||||
"model.layers.83.mlp.gate",
|
||||
"model.layers.83.post_attention_layernorm",
|
||||
"model.layers.84.input_layernorm",
|
||||
"model.layers.84.mlp.gate",
|
||||
"model.layers.84.post_attention_layernorm",
|
||||
"model.layers.85.input_layernorm",
|
||||
"model.layers.85.mlp.gate",
|
||||
"model.layers.85.post_attention_layernorm",
|
||||
"model.layers.86.input_layernorm",
|
||||
"model.layers.86.mlp.gate",
|
||||
"model.layers.86.post_attention_layernorm",
|
||||
"model.layers.87.input_layernorm",
|
||||
"model.layers.87.mlp.gate",
|
||||
"model.layers.87.post_attention_layernorm",
|
||||
"model.layers.88.input_layernorm",
|
||||
"model.layers.88.mlp.gate",
|
||||
"model.layers.88.post_attention_layernorm",
|
||||
"model.layers.89.input_layernorm",
|
||||
"model.layers.89.mlp.gate",
|
||||
"model.layers.89.post_attention_layernorm",
|
||||
"model.layers.90.input_layernorm",
|
||||
"model.layers.90.mlp.gate",
|
||||
"model.layers.90.post_attention_layernorm",
|
||||
"model.layers.91.input_layernorm",
|
||||
"model.layers.91.mlp.gate",
|
||||
"model.layers.91.post_attention_layernorm",
|
||||
"model.layers.92.input_layernorm",
|
||||
"model.layers.92.mlp.gate",
|
||||
"model.layers.92.post_attention_layernorm",
|
||||
"model.layers.93.input_layernorm",
|
||||
"model.layers.93.mlp.gate",
|
||||
"model.layers.93.post_attention_layernorm"
|
||||
],
|
||||
"fmt": "e4m3",
|
||||
"quant_method": "fp8",
|
||||
"weight_block_size": [
|
||||
128,
|
||||
128
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/model_config.original.json
|
||||
1d7389f77563bb26bdb8cad077f9ae81f94dbe0b7f2e51cc72f7f4ff0f550acb /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6/data/config/models/Qwen3-235B-A22B-FP8.json
|
||||
@@ -0,0 +1,2 @@
|
||||
{'name': 'Qwen3-235B-A22B-FP8', 'num_layers': 94, 'num_q_heads': 64, 'num_kv_heads': 4, 'embedding_dim': 4096, 'mlp_hidden_dim': 1536, 'max_position_embeddings': 40960, 'use_gated_mlp': True, 'use_bias': False, 'use_qkv_bias': False, 'activation': 'silu', 'norm': 'rms_norm', 'post_attn_norm': True, 'vocab_size': 151936, 'is_neox_style': True, 'rope_theta': 1000000.0, 'rope_scaling': None, 'partial_rotary_factor': 1.0, 'no_tensor_parallel': False, 'is_moe': True, 'num_experts': 128, 'num_experts_per_tok': 8, 'moe_layers_enum': None, 'use_qk_norm': True, 'attn_output_gate': False, 'rms_norm_eps': 1e-06, 'dtype': 'BF16', 'model_type': 'qwen3_moe', 'fused_add_norm_capability': True, 'model_arch': 'generic', 'share_expert_dim': None, 'share_q_dim': None, 'head_dim': 128, 'quantization_config': {'quant_method': 'fp8', 'activation_scheme': 'dynamic', 'is_checkpoint_fp8_serialized': True, 'weight_block_size': (128, 128), 'ignored_layers': []}, 'tie_word_embeddings': False}
|
||||
{'tp_size': 4, 'attn_enabled': True, 'ffn_enabled': True, 'attn_sharded_enabled': True, 'ffn_sharded_enabled': True, 'replicated_enabled': True, 'disable_replicated': False, 'enabled_ops': ['input_layernorm', 'post_attention_layernorm', 'add', 'emb', 'attn_pre_proj', 'attn_rope', 'attn_post_proj'], 'disabled_ops': [], 'replicated_ops': ['input_layernorm', 'post_attention_layernorm', 'add', 'emb'], 'padded_n_embd': 4096, 'padded_n_expanded_embd': 1536, 'skip_reasons': []}
|
||||
@@ -0,0 +1 @@
|
||||
4459cca6fe7f01c44477460ccca85ec1f2a0eacdc764170c68fa02a102f4104e /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/platform.txt
|
||||
@@ -0,0 +1,99 @@
|
||||
captured_utc=2026-07-15T09:34:36Z
|
||||
ds-07429c65-1-6c5fd97778-9vhkr
|
||||
Linux ds-07429c65-1-6c5fd97778-9vhkr 5.10.134-013.8.2.kangaroo.al8.x86_64 #1 SMP Thu Mar 12 10:20:37 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
|
||||
PRETTY_NAME="Ubuntu 24.04.2 LTS"
|
||||
NAME="Ubuntu"
|
||||
VERSION_ID="24.04"
|
||||
VERSION="24.04.2 LTS (Noble Numbat)"
|
||||
VERSION_CODENAME=noble
|
||||
ID=ubuntu
|
||||
ID_LIKE=debian
|
||||
HOME_URL="https://www.ubuntu.com/"
|
||||
SUPPORT_URL="https://help.ubuntu.com/"
|
||||
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
|
||||
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
|
||||
UBUNTU_CODENAME=noble
|
||||
LOGO=ubuntu-logo
|
||||
Architecture: x86_64
|
||||
CPU op-mode(s): 32-bit, 64-bit
|
||||
Address sizes: 52 bits physical, 57 bits virtual
|
||||
Byte Order: Little Endian
|
||||
CPU(s): 160
|
||||
On-line CPU(s) list: 0-159
|
||||
Vendor ID: GenuineIntel
|
||||
Model name: Intel(R) Xeon(R) Processor
|
||||
CPU family: 6
|
||||
Model: 143
|
||||
Thread(s) per core: 1
|
||||
Core(s) per socket: 80
|
||||
Socket(s): 2
|
||||
Stepping: 8
|
||||
BogoMIPS: 5200.00
|
||||
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd avx512vbmi umip pku waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities
|
||||
Hypervisor vendor: KVM
|
||||
Virtualization type: full
|
||||
L1d cache: 3.8 MiB (80 instances)
|
||||
L1i cache: 2.5 MiB (80 instances)
|
||||
L2 cache: 160 MiB (80 instances)
|
||||
L3 cache: 195 MiB (2 instances)
|
||||
NUMA node(s): 2
|
||||
NUMA node0 CPU(s): 0-79
|
||||
NUMA node1 CPU(s): 80-159
|
||||
Vulnerability Itlb multihit: Not affected
|
||||
Vulnerability L1tf: Not affected
|
||||
Vulnerability Mds: Not affected
|
||||
Vulnerability Meltdown: Not affected
|
||||
Vulnerability Mmio stale data: Not affected
|
||||
Vulnerability Retbleed: Not affected
|
||||
Vulnerability Spec rstack overflow: Not affected
|
||||
Vulnerability Spec store bypass: Vulnerable
|
||||
Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
|
||||
Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Vulnerable
|
||||
Vulnerability Srbds: Not affected
|
||||
Vulnerability Tsx async abort: Not affected
|
||||
0, NVIDIA H20, GPU-ad3e049a-5bf0-44b7-e7f1-9af297b172af, 580.95.05, 97871 MiB, 9.0
|
||||
1, NVIDIA H20, GPU-8c088079-d0f5-ba23-8650-5e6b1436691f, 580.95.05, 97871 MiB, 9.0
|
||||
2, NVIDIA H20, GPU-a9f6fe67-324b-8bb2-19b3-c7f1b9cce96a, 580.95.05, 97871 MiB, 9.0
|
||||
3, NVIDIA H20, GPU-6bcd68b7-ffa7-26b5-df6a-b7eb3f65c901, 580.95.05, 97871 MiB, 9.0
|
||||
4, NVIDIA H20, GPU-b409f9c9-05b6-55ef-3f3b-12eaa7c6ebfe, 580.95.05, 97871 MiB, 9.0
|
||||
5, NVIDIA H20, GPU-56932433-efce-8215-6418-98166d8ab798, 580.95.05, 97871 MiB, 9.0
|
||||
6, NVIDIA H20, GPU-ddcd1b77-e38d-75f5-ac24-788c76e56c89, 580.95.05, 97871 MiB, 9.0
|
||||
7, NVIDIA H20, GPU-9b4a31bd-5e78-b5a7-55aa-d786ae5a3f21, 580.95.05, 97871 MiB, 9.0
|
||||
[4mGPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 NIC0 NIC1 NIC2 NIC3 CPU Affinity NUMA Affinity GPU NUMA ID[0m
|
||||
GPU0 X NV18 NV18 NV18 NV18 NV18 NV18 NV18 PIX PHB SYS SYS 0-79 0 N/A
|
||||
GPU1 NV18 X NV18 NV18 NV18 NV18 NV18 NV18 PXB PHB SYS SYS 0-79 0 N/A
|
||||
GPU2 NV18 NV18 X NV18 NV18 NV18 NV18 NV18 PHB PIX SYS SYS 0-79 0 N/A
|
||||
GPU3 NV18 NV18 NV18 X NV18 NV18 NV18 NV18 PHB PXB SYS SYS 0-79 0 N/A
|
||||
GPU4 NV18 NV18 NV18 NV18 X NV18 NV18 NV18 SYS SYS PIX PHB 80-159 1 N/A
|
||||
GPU5 NV18 NV18 NV18 NV18 NV18 X NV18 NV18 SYS SYS PXB PHB 80-159 1 N/A
|
||||
GPU6 NV18 NV18 NV18 NV18 NV18 NV18 X NV18 SYS SYS PHB PIX 80-159 1 N/A
|
||||
GPU7 NV18 NV18 NV18 NV18 NV18 NV18 NV18 X SYS SYS PHB PXB 80-159 1 N/A
|
||||
NIC0 PIX PXB PHB PHB SYS SYS SYS SYS X PHB SYS SYS
|
||||
NIC1 PHB PHB PIX PXB SYS SYS SYS SYS PHB X SYS SYS
|
||||
NIC2 SYS SYS SYS SYS PIX PXB PHB PHB SYS SYS X PHB
|
||||
NIC3 SYS SYS SYS SYS PHB PHB PIX PXB SYS SYS PHB X
|
||||
|
||||
Legend:
|
||||
|
||||
X = Self
|
||||
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
|
||||
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
|
||||
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
|
||||
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
|
||||
PIX = Connection traversing at most a single PCIe bridge
|
||||
NV# = Connection traversing a bonded set of # NVLinks
|
||||
|
||||
NIC Legend:
|
||||
|
||||
NIC0: mlx5_0
|
||||
NIC1: mlx5_1
|
||||
NIC2: mlx5_2
|
||||
NIC3: mlx5_3
|
||||
|
||||
nvcc: NVIDIA (R) Cuda compiler driver
|
||||
Copyright (c) 2005-2025 NVIDIA Corporation
|
||||
Built on Tue_May_27_02:21:03_PDT_2025
|
||||
Cuda compilation tools, release 12.9, V12.9.86
|
||||
Build cuda_12.9.r12.9/compiler.36037853_0
|
||||
gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
|
||||
ldd (Ubuntu GLIBC 2.39-0ubuntu8.4) 2.39
|
||||
@@ -0,0 +1 @@
|
||||
2d17f91ca22ed56dbffc4549972814be911901f321963f8f07a2b940df191340 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/provenance/requirements.freeze.txt
|
||||
@@ -0,0 +1,163 @@
|
||||
aiohappyeyeballs==2.7.1
|
||||
aiohttp==3.14.1
|
||||
aiosignal==1.4.0
|
||||
annotated-doc==0.0.4
|
||||
annotated-types==0.7.0
|
||||
anyio==4.14.2
|
||||
astor==0.8.1
|
||||
attrs==26.1.0
|
||||
blake3==1.0.9
|
||||
cachetools==7.1.4
|
||||
cbor2==6.1.3
|
||||
certifi==2026.6.17
|
||||
cffi==2.1.0
|
||||
charset-normalizer==3.4.9
|
||||
click==8.4.2
|
||||
cloudpickle==3.1.2
|
||||
compressed-tensors==0.11.0
|
||||
cuda-pathfinder==1.5.6
|
||||
cupy-cuda12x==14.1.1
|
||||
ddsketch==3.0.1
|
||||
depyf==0.19.0
|
||||
detect-installer==0.1.0
|
||||
dill==0.4.1
|
||||
diskcache==5.6.3
|
||||
distro==1.9.0
|
||||
dnspython==2.8.0
|
||||
einops==0.8.2
|
||||
email-validator==2.3.0
|
||||
fastapi==0.139.0
|
||||
fastapi-cli==0.0.29
|
||||
fastapi-cloud-cli==0.22.2
|
||||
fastar==0.11.0
|
||||
fasteners==0.20
|
||||
filelock==3.29.7
|
||||
flashinfer-python==0.3.1.post1
|
||||
-e file:///home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6
|
||||
frozendict==2.4.7
|
||||
frozenlist==1.8.0
|
||||
fsspec==2026.6.0
|
||||
gguf==0.19.0
|
||||
h11==0.16.0
|
||||
hf-xet==1.5.1
|
||||
httpcore==1.0.9
|
||||
httptools==0.8.0
|
||||
httpx==0.28.1
|
||||
huggingface-hub==0.36.2
|
||||
idna==3.18
|
||||
iniconfig==2.3.0
|
||||
interegular==0.3.3
|
||||
jinja2==3.1.6
|
||||
jiter==0.16.0
|
||||
joblib==1.5.3
|
||||
jsonschema==4.26.0
|
||||
jsonschema-specifications==2025.9.1
|
||||
lark==1.2.2
|
||||
llguidance==0.7.30
|
||||
llvmlite==0.44.0
|
||||
lm-format-enforcer==0.11.3
|
||||
markdown-it-py==4.2.0
|
||||
markupsafe==3.0.3
|
||||
mdurl==0.1.2
|
||||
mistral-common==1.11.5
|
||||
mpmath==1.3.0
|
||||
msgpack==1.2.1
|
||||
msgspec==0.21.1
|
||||
multidict==6.7.1
|
||||
narwhals==2.24.0
|
||||
networkx==3.6.1
|
||||
ninja==1.13.0
|
||||
numba==0.61.2
|
||||
numpy==2.2.6
|
||||
nvidia-cublas-cu12==12.8.4.1
|
||||
nvidia-cuda-cupti-cu12==12.8.90
|
||||
nvidia-cuda-nvrtc-cu12==12.8.93
|
||||
nvidia-cuda-runtime-cu12==12.8.90
|
||||
nvidia-cudnn-cu12==9.10.2.21
|
||||
nvidia-cudnn-frontend==1.26.0
|
||||
nvidia-cufft-cu12==11.3.3.83
|
||||
nvidia-cufile-cu12==1.13.1.3
|
||||
nvidia-curand-cu12==10.3.9.90
|
||||
nvidia-cusolver-cu12==11.7.3.90
|
||||
nvidia-cusparse-cu12==12.5.8.93
|
||||
nvidia-cusparselt-cu12==0.7.1
|
||||
nvidia-ml-py==13.610.43
|
||||
nvidia-nccl-cu12==2.27.3
|
||||
nvidia-nvjitlink-cu12==12.8.93
|
||||
nvidia-nvtx-cu12==12.8.90
|
||||
openai==2.45.0
|
||||
openai-harmony==0.0.8
|
||||
opencv-python-headless==5.0.0.93
|
||||
outlines-core==0.2.11
|
||||
packaging==26.2
|
||||
pandas==3.0.3
|
||||
partial-json-parser==0.2.1.1.post7
|
||||
pillow==12.3.0
|
||||
plotly==6.9.0
|
||||
pluggy==1.6.0
|
||||
prometheus-client==0.25.0
|
||||
prometheus-fastapi-instrumentator==8.0.2
|
||||
propcache==0.5.2
|
||||
protobuf==7.35.1
|
||||
psutil==7.2.2
|
||||
py-cpuinfo==9.0.0
|
||||
pybase64==1.4.3
|
||||
pycountry==26.2.16
|
||||
pycparser==3.0
|
||||
pydantic==2.13.4
|
||||
pydantic-core==2.46.4
|
||||
pydantic-extra-types==2.11.1
|
||||
pydantic-settings==2.14.2
|
||||
pygments==2.20.0
|
||||
pynvml==13.0.1
|
||||
pytest==9.1.1
|
||||
python-dateutil==2.9.0.post0
|
||||
python-dotenv==1.2.2
|
||||
python-json-logger==4.1.0
|
||||
python-multipart==0.0.32
|
||||
pyyaml==6.0.3
|
||||
pyzmq==27.1.0
|
||||
ray==2.56.0
|
||||
referencing==0.37.0
|
||||
regex==2026.7.10
|
||||
requests==2.34.2
|
||||
rich==15.0.0
|
||||
rich-toolkit==0.20.3
|
||||
rignore==0.7.6
|
||||
rpds-py==2026.6.3
|
||||
safetensors==0.8.0
|
||||
scikit-learn==1.9.0
|
||||
scipy==1.18.0
|
||||
sentencepiece==0.2.2
|
||||
sentry-sdk==2.65.0
|
||||
setproctitle==1.3.7
|
||||
setuptools==79.0.1
|
||||
shellingham==1.5.4
|
||||
six==1.17.0
|
||||
sniffio==1.3.1
|
||||
soundfile==0.14.0
|
||||
soxr==1.1.0
|
||||
starlette==1.3.1
|
||||
sympy==1.14.0
|
||||
tabulate==0.10.0
|
||||
threadpoolctl==3.6.0
|
||||
tiktoken==0.13.0
|
||||
tokenizers==0.21.4
|
||||
torch==2.8.0
|
||||
torchaudio==2.8.0
|
||||
torchvision==0.23.0
|
||||
tqdm==4.68.4
|
||||
transformers==4.55.2
|
||||
triton==3.4.0
|
||||
typer==0.26.8
|
||||
typing-extensions==4.16.0
|
||||
typing-inspection==0.4.2
|
||||
urllib3==2.7.0
|
||||
uvicorn==0.51.0
|
||||
uvloop==0.22.1
|
||||
vllm==0.10.2
|
||||
watchfiles==1.2.0
|
||||
websockets==16.1
|
||||
xformers==0.0.32.post1
|
||||
xgrammar==0.1.23
|
||||
yarl==1.24.2
|
||||
@@ -0,0 +1,163 @@
|
||||
aiohappyeyeballs==2.7.1
|
||||
aiohttp==3.14.1
|
||||
aiosignal==1.4.0
|
||||
annotated-doc==0.0.4
|
||||
annotated-types==0.7.0
|
||||
anyio==4.14.2
|
||||
astor==0.8.1
|
||||
attrs==26.1.0
|
||||
blake3==1.0.9
|
||||
cachetools==7.1.4
|
||||
cbor2==6.1.3
|
||||
certifi==2026.6.17
|
||||
cffi==2.1.0
|
||||
charset-normalizer==3.4.9
|
||||
click==8.4.2
|
||||
cloudpickle==3.1.2
|
||||
compressed-tensors==0.11.0
|
||||
cuda-pathfinder==1.5.6
|
||||
cupy-cuda12x==14.1.1
|
||||
ddsketch==3.0.1
|
||||
depyf==0.19.0
|
||||
detect-installer==0.1.0
|
||||
dill==0.4.1
|
||||
diskcache==5.6.3
|
||||
distro==1.9.0
|
||||
dnspython==2.8.0
|
||||
einops==0.8.2
|
||||
email-validator==2.3.0
|
||||
fastapi==0.139.0
|
||||
fastapi-cli==0.0.29
|
||||
fastapi-cloud-cli==0.22.2
|
||||
fastar==0.11.0
|
||||
fasteners==0.20
|
||||
filelock==3.29.7
|
||||
flashinfer-python==0.3.1.post1
|
||||
-e file:///home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6
|
||||
frozendict==2.4.7
|
||||
frozenlist==1.8.0
|
||||
fsspec==2026.6.0
|
||||
gguf==0.19.0
|
||||
h11==0.16.0
|
||||
hf-xet==1.5.1
|
||||
httpcore==1.0.9
|
||||
httptools==0.8.0
|
||||
httpx==0.28.1
|
||||
huggingface-hub==1.23.0
|
||||
idna==3.18
|
||||
iniconfig==2.3.0
|
||||
interegular==0.3.3
|
||||
jinja2==3.1.6
|
||||
jiter==0.16.0
|
||||
joblib==1.5.3
|
||||
jsonschema==4.26.0
|
||||
jsonschema-specifications==2025.9.1
|
||||
lark==1.2.2
|
||||
llguidance==0.7.30
|
||||
llvmlite==0.44.0
|
||||
lm-format-enforcer==0.11.3
|
||||
markdown-it-py==4.2.0
|
||||
markupsafe==3.0.3
|
||||
mdurl==0.1.2
|
||||
mistral-common==1.11.5
|
||||
mpmath==1.3.0
|
||||
msgpack==1.2.1
|
||||
msgspec==0.21.1
|
||||
multidict==6.7.1
|
||||
narwhals==2.24.0
|
||||
networkx==3.6.1
|
||||
ninja==1.13.0
|
||||
numba==0.61.2
|
||||
numpy==2.2.6
|
||||
nvidia-cublas-cu12==12.8.4.1
|
||||
nvidia-cuda-cupti-cu12==12.8.90
|
||||
nvidia-cuda-nvrtc-cu12==12.8.93
|
||||
nvidia-cuda-runtime-cu12==12.8.90
|
||||
nvidia-cudnn-cu12==9.10.2.21
|
||||
nvidia-cudnn-frontend==1.26.0
|
||||
nvidia-cufft-cu12==11.3.3.83
|
||||
nvidia-cufile-cu12==1.13.1.3
|
||||
nvidia-curand-cu12==10.3.9.90
|
||||
nvidia-cusolver-cu12==11.7.3.90
|
||||
nvidia-cusparse-cu12==12.5.8.93
|
||||
nvidia-cusparselt-cu12==0.7.1
|
||||
nvidia-ml-py==13.610.43
|
||||
nvidia-nccl-cu12==2.27.3
|
||||
nvidia-nvjitlink-cu12==12.8.93
|
||||
nvidia-nvtx-cu12==12.8.90
|
||||
openai==2.45.0
|
||||
openai-harmony==0.0.8
|
||||
opencv-python-headless==5.0.0.93
|
||||
outlines-core==0.2.11
|
||||
packaging==26.2
|
||||
pandas==3.0.3
|
||||
partial-json-parser==0.2.1.1.post7
|
||||
pillow==12.3.0
|
||||
plotly==6.9.0
|
||||
pluggy==1.6.0
|
||||
prometheus-client==0.25.0
|
||||
prometheus-fastapi-instrumentator==8.0.2
|
||||
propcache==0.5.2
|
||||
protobuf==7.35.1
|
||||
psutil==7.2.2
|
||||
py-cpuinfo==9.0.0
|
||||
pybase64==1.4.3
|
||||
pycountry==26.2.16
|
||||
pycparser==3.0
|
||||
pydantic==2.13.4
|
||||
pydantic-core==2.46.4
|
||||
pydantic-extra-types==2.11.1
|
||||
pydantic-settings==2.14.2
|
||||
pygments==2.20.0
|
||||
pynvml==13.0.1
|
||||
pytest==9.1.1
|
||||
python-dateutil==2.9.0.post0
|
||||
python-dotenv==1.2.2
|
||||
python-json-logger==4.1.0
|
||||
python-multipart==0.0.32
|
||||
pyyaml==6.0.3
|
||||
pyzmq==27.1.0
|
||||
ray==2.56.0
|
||||
referencing==0.37.0
|
||||
regex==2026.7.10
|
||||
requests==2.34.2
|
||||
rich==15.0.0
|
||||
rich-toolkit==0.20.3
|
||||
rignore==0.7.6
|
||||
rpds-py==2026.6.3
|
||||
safetensors==0.8.0
|
||||
scikit-learn==1.9.0
|
||||
scipy==1.18.0
|
||||
sentencepiece==0.2.2
|
||||
sentry-sdk==2.65.0
|
||||
setproctitle==1.3.7
|
||||
setuptools==79.0.1
|
||||
shellingham==1.5.4
|
||||
six==1.17.0
|
||||
sniffio==1.3.1
|
||||
soundfile==0.14.0
|
||||
soxr==1.1.0
|
||||
starlette==1.3.1
|
||||
sympy==1.14.0
|
||||
tabulate==0.10.0
|
||||
threadpoolctl==3.6.0
|
||||
tiktoken==0.13.0
|
||||
tokenizers==0.22.2
|
||||
torch==2.8.0
|
||||
torchaudio==2.8.0
|
||||
torchvision==0.23.0
|
||||
tqdm==4.68.4
|
||||
transformers==5.13.1
|
||||
triton==3.4.0
|
||||
typer==0.26.8
|
||||
typing-extensions==4.16.0
|
||||
typing-inspection==0.4.2
|
||||
urllib3==2.7.0
|
||||
uvicorn==0.51.0
|
||||
uvloop==0.22.1
|
||||
vllm==0.10.2
|
||||
watchfiles==1.2.0
|
||||
websockets==16.1
|
||||
xformers==0.0.32.post1
|
||||
xgrammar==0.1.23
|
||||
yarl==1.24.2
|
||||
@@ -0,0 +1,209 @@
|
||||
# Community Qwen235B / Frontier smoke report
|
||||
|
||||
## Bottom line
|
||||
|
||||
The community-vLLM TP4 feasibility smoke passed, but the unmodified Frontier
|
||||
profiles are not valid inputs for a formal simulator-versus-runtime ranking
|
||||
test yet. The smoke exposed an execution-semantic mismatch inside the MoE
|
||||
profiler: serving selects vLLM's tuned H20 block-FP8 Triton config, while the
|
||||
standalone Frontier profiler omits the FP8 dtype key and silently falls back to
|
||||
a default config. At the tested TP4/EP1/16-token point, fixing this lookup
|
||||
reduces measured grouped-GEMM time by about 19%, with the direction consistent
|
||||
across five paired routing seeds.
|
||||
|
||||
This is precisely why merely installing the same vLLM release on both sides is
|
||||
not a sufficient alignment contract.
|
||||
|
||||
## Change
|
||||
|
||||
- Built an isolated Python 3.12 environment around community vLLM 0.10.2 and
|
||||
Frontier commit `d9cfeb6d8791fbf2f295dd9744c56a666171776e`.
|
||||
- Pinned Transformers 4.55.2 and tokenizers 0.21.4 after reproducing an
|
||||
incompatibility with the unconstrained Transformers 5.13.1 resolver result.
|
||||
- Added Frontier metadata for the local block-FP8 checkpoint without changing
|
||||
the original Hugging Face config.
|
||||
- Collected representative FP8 linear, FlashInfer prefill-attention, FP8 MoE,
|
||||
and TP4 NCCL all-reduce measurements.
|
||||
- Loaded the full Qwen3-235B-A22B-FP8 checkpoint on four H20s through community
|
||||
vLLM and completed one real request.
|
||||
- Ran a 2x2 MoE diagnostic that independently toggles the runtime FP8 tuning
|
||||
key and BF16 compute type.
|
||||
|
||||
## Expected effect
|
||||
|
||||
The smoke was intended to determine whether community vLLM can provide a
|
||||
shared, reproducible profiler/serving substrate before paying for the complete
|
||||
response surface. It was not intended to establish Frontier ranking accuracy.
|
||||
|
||||
Success required all representative operator and collective paths to execute,
|
||||
the TP4 server to become ready, one request to return successfully, and all GPU
|
||||
processes to be released. A discovered profiler/runtime semantic mismatch was
|
||||
treated as a gate on the later simulator comparison rather than calibrated
|
||||
away with end-to-end serving data.
|
||||
|
||||
## Frozen environment
|
||||
|
||||
- Host: `dash0`; 8 NVIDIA H20 GPUs.
|
||||
- Model: `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8`.
|
||||
- Original model-config SHA256:
|
||||
`702c46d431bb984db9035a1225186bbfdb52c0d19c82104df4a37cd005e0369e`.
|
||||
- Python 3.12.3, torch 2.8.0+cu128, vLLM 0.10.2,
|
||||
flashinfer-python 0.3.1.post1, Transformers 4.55.2, tokenizers 0.21.4.
|
||||
- Ubuntu 24.04.2, kernel 5.10.134, CUDA runtime 12.8, CUDA toolkit 12.9,
|
||||
NVIDIA driver 580.95.05, GCC 13.3.0, glibc 2.39.
|
||||
- Two 80-core Intel Xeon sockets (160 online CPUs), 8 H20 GPUs with
|
||||
97,871 MiB each and all-to-all NV18 links; GPU0--3 share NUMA node 0.
|
||||
- Serving: TP4, FlashInfer, eager execution, custom all-reduce disabled,
|
||||
FP8 weights, BF16 KV cache, MNS=64, MBT=8192, max model length 40,960,
|
||||
prefix caching and speculative decoding disabled.
|
||||
|
||||
The first environment resolved Transformers 5.13.1 because vLLM 0.10.2 only
|
||||
declares `transformers>=4.55.2`. That environment failed before model loading
|
||||
because vLLM accesses `all_special_tokens_extended`, an API absent from the
|
||||
new tokenizer implementation. Pinning the declared minimum restored the API;
|
||||
the original and corrected freezes are both retained.
|
||||
|
||||
## Verification
|
||||
|
||||
### End-to-end smoke
|
||||
|
||||
| Stage | Result |
|
||||
|---|---|
|
||||
| FP8 linear, TP4, 16 tokens | Passed; QKV projection mean 0.2232 ms and output projection mean 0.1105 ms |
|
||||
| FlashInfer prefill attention, TP4 | Passed for sequence lengths 32/64/96/128; attention mean 0.0571--0.0665 ms |
|
||||
| FP8 MoE, TP4/EP1, 16 tokens | Passed; unmodified grouped-GEMM mean 0.3075 ms in the initial point |
|
||||
| TP4 NCCL all-reduce, 128 KiB/rank | Passed; mean 0.03694 ms, p50 0.03138 ms, p95 0.05843 ms |
|
||||
| Community-vLLM TP4 model load | Passed; 55.1328 GiB of weights/rank, 627 seconds |
|
||||
| KV-cache sizing | 18.72 GiB/rank, 417,616 tokens; reported 10.20x concurrency at length 40,960 |
|
||||
| API readiness and one-token completion | Passed; `/v1/models` and `/v1/completions` returned HTTP 200 |
|
||||
| Cleanup | Passed; all eight GPUs returned to zero compute processes and zero MiB used |
|
||||
|
||||
The successful serving-only run started at 09:09:52 UTC and completed at
|
||||
09:22:29 UTC (12 minutes 37 seconds). The one-request response contains one
|
||||
completion choice and one completion token. The copied artifacts match the
|
||||
remote SHA256 manifests.
|
||||
|
||||
### FP8 path audit
|
||||
|
||||
The Frontier confirmation UI prints `FP8 Quantization: N/A` and labels the
|
||||
operations BF16 because it displays the unset CLI override rather than the
|
||||
model-config-derived operation precision. This is misleading, not evidence of
|
||||
a BF16-weight profile:
|
||||
|
||||
- the quantization manager configured `attn_pre_proj`, `attn_post_proj`, and
|
||||
`moe_grouped_gemm` as FP8 from the model config;
|
||||
- the linear implementation invokes `apply_w8a8_block_fp8_linear` with a
|
||||
128x128 weight block;
|
||||
- the MoE implementation quantizes weights and activations and invokes vLLM's
|
||||
fused kernel with `use_fp8_w8a8=True`;
|
||||
- `FRONTIER_FP8_GEMM_SURROGATE` was not enabled;
|
||||
- each CSV carries
|
||||
`method=fp8|act=dynamic|serialized=True|block=128x128` separately from its
|
||||
BF16 output/compute dtype metadata.
|
||||
|
||||
### MoE runtime-alignment factorial
|
||||
|
||||
Point: Qwen3-235B-A22B-FP8, TP4, EP1, 16 input tokens, uniform routing seeds
|
||||
0--4. Values are the mean of each row's CUDA-event samples, then averaged
|
||||
across the five paired routing seeds.
|
||||
|
||||
| Variant | Grouped GEMM mean (ms) | Paired delta vs original | Routing-seed 95% CI |
|
||||
|---|---:|---:|---:|
|
||||
| Original: default config + FP16 compute type | 0.3100 | 0.00% | -- |
|
||||
| FP8 config key only + FP16 compute type | 0.2508 | -19.12% | [-21.26%, -16.98%] |
|
||||
| Default config + BF16 compute type only | 0.3088 | -0.40% | [-1.31%, 0.51%] |
|
||||
| FP8 config key + BF16 compute type | 0.2512 | -18.99% | [-21.14%, -16.85%] |
|
||||
|
||||
These intervals describe variation across the five routing seeds, not
|
||||
independent process/server-run uncertainty. Even with that limitation, the
|
||||
factorial localizes the dominant error at this point to kernel tuning-config
|
||||
selection rather than FP16-versus-BF16 compute type.
|
||||
|
||||
The concrete lookup difference is:
|
||||
|
||||
- Frontier calls `get_config_dtype_str(base_dtype)`. For BF16 this returns
|
||||
`None`, so it searches for a config without an FP8 dtype component and uses
|
||||
the default when that file is absent.
|
||||
- vLLM serving calls the same helper with `use_fp8_w8a8=True`, obtains
|
||||
`fp8_w8a8`, and loads
|
||||
`E=128,N=384,device_name=NVIDIA_H20,dtype=fp8_w8a8,block_shape=[128,128].json`.
|
||||
|
||||
Frontier also hard-codes `tl.float16` as the FP8 kernel compute type, while the
|
||||
serving path uses BF16 for this model. That difference was not material at this
|
||||
single point, but it should still be corrected to make the counterfactual
|
||||
execution semantics identical.
|
||||
|
||||
## Validity audit
|
||||
|
||||
### Headline claims and benchmark surface
|
||||
|
||||
| Claim | Verdict | Scope |
|
||||
|---|---|---|
|
||||
| Community vLLM 0.10.2 can load and serve this model on TP4 H20 | PASS | One frozen TP4 configuration and one real request |
|
||||
| Unmodified Frontier chooses a different MoE tuning config, causing a material operator-time error at the tested point | PASS | TP4/EP1, 16 tokens, five paired routing seeds |
|
||||
| Frontier is or is not sufficient as a Qwen235B config ranker | NEEDS EVIDENCE | No aligned full profiles, frozen simulator surface, or real config surface yet |
|
||||
|
||||
The mechanism diagnostic uses the unmodified Frontier wrapper as baseline,
|
||||
grouped-GEMM CUDA-event latency as its direct metric, and the same model,
|
||||
hardware, vLLM binary, token count, TP/EP point, and routing seeds in all four
|
||||
cells. Absolute times, paired relative differences, across-seed standard
|
||||
deviations, and a paired routing-seed interval are retained. The data range is
|
||||
deliberately narrow and cannot support an end-to-end or cross-config claim.
|
||||
|
||||
| Benchmark issue | Verdict | Severity | Evidence / required action |
|
||||
|---|---|---|---|
|
||||
| Microbenchmark presented as end-to-end performance | PASS | -- | The 19% number is reported only as mechanism evidence; ranking remains blocked. |
|
||||
| Simplified profiler matches the real execution path | FAIL | Blocking | Tuning-key and compute-type semantics differ. Apply the alignment patch and test exact config dictionaries before simulation. |
|
||||
| Statistical significance / repeat protocol | NEEDS EVIDENCE | Major | Five paired routing seeds are not independent process repeats. Repeat in reverse/randomized variant order and at separated times. |
|
||||
| Selective parameter range | NEEDS EVIDENCE | Major | Only TP4/EP1/16 tokens is measured. Cover the full decision-relevant token and TP range, including boundary values. |
|
||||
| Calibration set equals evaluation set | PASS | -- | No serving result calibrated the profile; the protocol freezes simulator outputs before ground truth. |
|
||||
| Proper diagnostic baseline | PASS | -- | Each ablation is compared with the exact unmodified Frontier commit on the same point. |
|
||||
| Platform and absolute values missing | PASS | -- | OS, kernel, CPU, GPU topology, driver/toolkit/runtime versions, commands, absolute metrics, logs, and hashes are retained. |
|
||||
| Full ranking baseline/SOTA comparison | N/A | -- | This smoke makes no ranking or tuner-superiority claim; it becomes required in the formal evaluation. |
|
||||
|
||||
Overall audit decision: the feasibility smoke and localized mismatch result are
|
||||
reportable, but any Frontier-sufficiency or config-ranking claim is **Block**
|
||||
until the required reruns and end-to-end comparison are complete.
|
||||
|
||||
## Result
|
||||
|
||||
The shared community stack is feasible: profiles can be collected and the
|
||||
235B checkpoint can serve on TP4 within memory. The stronger result, however,
|
||||
is a falsification of the current alignment assumption. Same model, hardware,
|
||||
vLLM package, quantization signature, and nominal backend did not imply the
|
||||
same kernel configuration. A small wrapper-level omission produced a roughly
|
||||
19% operator-time error before any scheduling approximation was involved.
|
||||
|
||||
Therefore no Frontier config ranking should be generated from the original
|
||||
MoE CSV and presented as a fair simulator-versus-real comparison. The correct
|
||||
next boundary is: first make the profiler reproduce the serving kernel
|
||||
selection, then freeze profile-only simulator outputs, and only then collect
|
||||
the real response surface.
|
||||
|
||||
## Remaining risk and next gate
|
||||
|
||||
- The 19% result is one TP4/EP1/16-token point. It must be checked across the
|
||||
decision-relevant token counts and TP4/TP8 before claiming its effect on
|
||||
config ordering.
|
||||
- Frontier's shuffling helper still performs a separate BF16/full-expert-width
|
||||
config lookup and reports a missing `E=128,N=1536` H20 file. At 16 tokens its
|
||||
default `BLOCK_SIZE_M` matches the tuned path, but this must be validated over
|
||||
the full token grid.
|
||||
- The derived Frontier model config adds
|
||||
`is_checkpoint_fp8_serialized=true`, which Frontier requires to parse the
|
||||
checkpoint metadata. This adaptation must remain explicit and hash-tracked.
|
||||
- The representative attention and collective smokes do not provide the
|
||||
profile coverage needed by the prefill candidate grid. Decode-dominant EP8
|
||||
remains blocked on all-to-all measurement/consumption.
|
||||
- A TP4 cold start spends about 10.5 minutes reading weights. Using that time as
|
||||
a lower-bound proxy, eight TP4/TP8 cells already cost roughly 8.4
|
||||
H20-GPU-hours in weight loading alone; a real one-pass response surface will
|
||||
exceed 10 H20-GPU-hours after initialization and load probes. It requires a
|
||||
separate launch approval after simulator outputs are frozen.
|
||||
|
||||
Recommended next step: upstream the two semantic fixes with regression tests
|
||||
that compare the exact vLLM config dictionary selected by profiler and serving
|
||||
for each `(M, TP, EP, quantization)` point; align the shuffling block-size
|
||||
lookup; regenerate TP4/TP8 profile closure; then run and hash the Frontier
|
||||
prefill predictions. Do not start the eight-cell real serving sweep before
|
||||
that gate passes.
|
||||
@@ -0,0 +1 @@
|
||||
{"object":"list","data":[{"id":"qwen3-235b-community-smoke","object":"model","created":1784107344,"owned_by":"vllm","root":"/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8","parent":null,"max_model_len":40960,"permission":[{"id":"modelperm-1715f628151642daa743a9bc47bffc88","object":"model_permission","created":1784107344,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]}]}
|
||||
@@ -0,0 +1,6 @@
|
||||
b186a5da9809c3f2a3ac8e008837719af2f0dd65c583e022d44183b3bd7587d0 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/moe_factorial_r5_summary.json
|
||||
ef9e8136bc6fc6d97a675cab95cb5b0425ac727d6d1e77598beadbf3d9cf4331 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/moe_factorial_r5_cells.csv
|
||||
8d9a6aad4ee9fbb3ed054ef1798850042077df40f16c4cf3e490abe7d416bf6e /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/aligned/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
bc2bf6fdb6987dc73d07c17034037f539ab2ba79901ee3ff6d33094c6beb2ffd /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/compute_type/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
9769cd09cb35060ce6e3ed6610700485edf0ab00eb74c97199b200a0f90bf3bc /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/config_key/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
0607dcb87a952ded5679f74ce151ca1624aa8f0484a40924920f0fd2ccb85634 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/profiles-moe-factorial-r5/original/compute/h20/Qwen3-235B-A22B-FP8/moe.csv
|
||||
@@ -0,0 +1,21 @@
|
||||
variant,seed,grouped_gemm_mean_ms,paired_delta_vs_original_pct
|
||||
original,0,0.3051120042800903,0.0
|
||||
original,1,0.305161565542221,0.0
|
||||
original,2,0.3159376084804535,0.0
|
||||
original,3,0.3228943943977356,0.0
|
||||
original,4,0.301118403673172,0.0
|
||||
config_key,0,0.2425376176834106,-20.508660989699024
|
||||
config_key,1,0.2544096112251282,-16.631175104543427
|
||||
config_key,2,0.2587344050407409,-18.105854416901945
|
||||
config_key,3,0.2593088150024414,-19.692376361594743
|
||||
config_key,4,0.2388928234577179,-20.664821364752093
|
||||
compute_type,0,0.3045775890350342,-0.17515379190571334
|
||||
compute_type,1,0.3057951927185058,0.20763662526077642
|
||||
compute_type,2,0.3109855651855469,-1.567411780675354
|
||||
compute_type,3,0.3209056258201599,-0.6159192020924187
|
||||
compute_type,4,0.3015664219856262,0.14878476605517665
|
||||
aligned,0,0.2454447746276855,-19.55584467847773
|
||||
aligned,1,0.2543264031410217,-16.658441999690808
|
||||
aligned,2,0.2585648000240326,-18.159537489811207
|
||||
aligned,3,0.2606095969676971,-19.289525773964723
|
||||
aligned,4,0.2369616031646728,-21.306170504986376
|
||||
|
@@ -0,0 +1,55 @@
|
||||
{
|
||||
"ci_scope": "paired t interval across routing seeds; not independent server-run uncertainty",
|
||||
"metric": "time_stats.moe_grouped_gemm.mean",
|
||||
"point": {
|
||||
"ep": 1,
|
||||
"model": "Qwen3-235B-A22B-FP8",
|
||||
"routing_seeds": [
|
||||
0,
|
||||
1,
|
||||
2,
|
||||
3,
|
||||
4
|
||||
],
|
||||
"tokens": 16,
|
||||
"tp": 4
|
||||
},
|
||||
"variants": {
|
||||
"aligned": {
|
||||
"mean_ms": 0.25118143558502193,
|
||||
"paired_delta_pct_95ci": [
|
||||
-21.135576186756534,
|
||||
-16.8522319920158
|
||||
],
|
||||
"paired_delta_pct_mean": -18.993904089386167,
|
||||
"sd_across_seeds_ms": 0.009854035105320383
|
||||
},
|
||||
"compute_type": {
|
||||
"mean_ms": 0.3087660789489746,
|
||||
"paired_delta_pct_95ci": [
|
||||
-1.3065750560434517,
|
||||
0.5057497027004384
|
||||
],
|
||||
"paired_delta_pct_mean": -0.4004126766715066,
|
||||
"sd_across_seeds_ms": 0.007591103189515662
|
||||
},
|
||||
"config_key": {
|
||||
"mean_ms": 0.2507766544818878,
|
||||
"paired_delta_pct_95ci": [
|
||||
-21.259089864169376,
|
||||
-16.98206543082712
|
||||
],
|
||||
"paired_delta_pct_mean": -19.120577647498248,
|
||||
"sd_across_seeds_ms": 0.009466111912783425
|
||||
},
|
||||
"original": {
|
||||
"mean_ms": 0.3100447952747345,
|
||||
"paired_delta_pct_95ci": [
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"paired_delta_pct_mean": 0.0,
|
||||
"sd_across_seeds_ms": 0.009051191520196651
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"id":"cmpl-78d1852cd51e445dab3372e1e2025d75","object":"text_completion","created":1784107344,"model":"qwen3-235b-community-smoke","choices":[{"index":0,"text":" ","logprobs":null,"finish_reason":"length","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt_token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":1,"total_tokens":2,"completion_tokens":1,"prompt_tokens_details":null},"kv_transfer_params":null}
|
||||
@@ -0,0 +1,3 @@
|
||||
a44ff3d6a0998324b52df358baf778b375f98be3e03c24b07f903f9c51525beb /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/allreduce_tp4.jsonl
|
||||
b6edf2c29065cdd303ec642bac85ceafcec30494dcce7ddf3d401c36de5162fe /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/models.json
|
||||
921505259d86a9037f2241cf5d611942cdf2985352c7080d1af0d39750fab211 /home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/results/one_request.json
|
||||
@@ -0,0 +1,17 @@
|
||||
case,cell_id,tp,dp,ep,mns,mbt,gpu_count,score_req_s_per_gpu,capacity_lower_bound_req_s_per_gpu,capacity_upper_bound_req_s_per_gpu,capacity_bracket_width_req_s_per_gpu,best_request_rate_req_s,best_sampling_u,best_pass_rate,probe_count,best_source,completed_with_probe_failure,fully_valid,result_path,result_sha256
|
||||
qwen235b_prefill_only,tp4_mns64_mbt8192,4,1,1,64,8192,4,0.1175,0.1175,0.13333333333333333,0.015833333333333338,0.47,0.021484375,0.9609929078014184,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/store/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/trials/trial-0001/result.json,cc9d8e3611c9719b5d8f8a14d376c24f08b348162cc49022889d9435816da13b
|
||||
qwen235b_prefill_only,tp4_mns64_mbt16384,4,1,1,64,16384,4,0.10666666666666667,0.10666666666666667,0.1175,0.01083333333333332,0.4266666666666667,0.01953125,0.9609375,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/store/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/trials/trial-0002/result.json,768360741f64ab39b3e7af497757c4fd92c66d5a0eff0125cc3d13f2814a0186
|
||||
qwen235b_prefill_only,tp4_mns128_mbt8192,4,1,1,128,8192,4,0.1175,0.1175,0.13333333333333333,0.015833333333333338,0.47,0.021484375,0.9609929078014184,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/store/interaction-qwen235b-prefill-c2-tp-mns-mbt-dash1-d8899c5-20260702T025518Z/trials/trial-0003/result.json,6cb56fb8ad0f9dc6e68b30528ed8993207c3951ef152ce8705a04c7fcf45a02a
|
||||
qwen235b_prefill_only,tp4_mns128_mbt16384,4,1,1,128,16384,4,0.10666666666666667,0.10666666666666667,0.1175,0.01083333333333332,0.4266666666666667,0.01953125,0.9609375,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0001/result.json,3c485648244243d529df8d7f16b0fb3d782cef1beec695f67cdb1fd5ea02b440
|
||||
qwen235b_prefill_only,tp8_mns64_mbt8192,8,1,1,64,8192,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9638118214716526,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0002/result.json,95c38d14ddc88f7f08730b7486ebb11538da6cec9ede11f5d16862b683290850
|
||||
qwen235b_prefill_only,tp8_mns64_mbt16384,8,1,1,64,16384,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9565741857659831,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0003/result.json,97b96526c568a572f10de602075c59a0a14555114f65d7c8e928b51fcf1cb787
|
||||
qwen235b_prefill_only,tp8_mns128_mbt8192,8,1,1,128,8192,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9601930036188179,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0004/result.json,f2695efe64c9c728f381e0d39ee42e7553b771928547f7a74a2cb2dafdc892f4
|
||||
qwen235b_prefill_only,tp8_mns128_mbt16384,8,1,1,128,16384,8,0.17270833333333332,0.17270833333333332,0.17791666666666667,0.005208333333333343,1.3816666666666666,0.0546875,0.9589867310012062,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/store/interaction-qwen235b-prefill-c2-remainder-dash1-d8899c5-20260702T163624Z/trials/trial-0005/result.json,1bc4d53f5ecbf4fbbd7c749a29ecf0f62e8382b84d899cc366d5bb485d78e261
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns64_mbt256,4,2,8,64,256,8,0.05354166666666667,0.05354166666666667,0.058958333333333335,0.005416666666666667,0.42833333333333334,0.01953125,1.0,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0001/result.json,f5a1c9ff3e8f98c37f8d3623e0376f34f67990f2e71e3b012beb8ac8ed447367
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns64_mbt384,4,2,8,64,384,8,0.05354166666666667,0.05354166666666667,0.058958333333333335,0.005416666666666667,0.42833333333333334,0.01953125,0.9922178988326849,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0002/result.json,adaafb060dc3ba642da34af7b69ee09ee0d04d857b8d4b5121a4b124373e1c57
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns128_mbt256,4,2,8,128,256,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9929328621908127,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0003/result.json,3555b3ecdef6337050c4e132e2b6f36a3cb98ea002a96f3efe118d301ad9d258
|
||||
qwen235b_decode_only,tp4_dp2_ep8_mns128_mbt384,4,2,8,128,384,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9929328621908127,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0004/result.json,3a010b17bc2cb785a57ef80a08646972109ad2dae28eeb38c153b35a1e3b6be2
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns64_mbt256,2,4,8,64,256,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9752650176678446,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0005/result.json,79649819c5599c8b81a115281a8e73aab5bc1a8188a1f611ace2329f9ae2f418
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns64_mbt384,2,4,8,64,384,8,0.05354166666666667,0.05354166666666667,0.058958333333333335,0.005416666666666667,0.42833333333333334,0.01953125,0.9961089494163424,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0006/result.json,6fdfeccfdbeb92bad8669251b26546966f46b42d667ae70eacdc544b44ad231d
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns128_mbt256,2,4,8,128,256,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9787985865724381,6,primary_search,False,True,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0007/result.json,8bc1c359e338de1d62bc87c43b1fa270b15023d223450552bffe5d1b778458f3
|
||||
qwen235b_decode_only,tp2_dp4_ep8_mns128_mbt384,2,4,8,128,384,8,0.058958333333333335,0.058958333333333335,0.066875,0.007916666666666669,0.4716666666666667,0.021484375,0.9823321554770318,6,partial_probe_before_failure,True,False,recovered-stores/aituner-interaction-runs-dash1-20260710/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/store/interaction-qwen235b-decode-c3-topo-mns-mbt-fixed-dash1-d8899c5-20260703T022514Z/trials/trial-0008/result.json,c371c2dd38db875db827ab85d4f399d0219cad513dde3c3b4db8cc782ec4debb
|
||||
|
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"cases": [
|
||||
{
|
||||
"best_score_req_s_per_gpu": 0.17270833333333332,
|
||||
"case": "qwen235b_prefill_only",
|
||||
"cell_count": 8,
|
||||
"distinct_score_count": 3,
|
||||
"distinct_scores_req_s_per_gpu": [
|
||||
0.17270833333333332,
|
||||
0.1175,
|
||||
0.10666666666666667
|
||||
],
|
||||
"fully_valid_cell_count": 8,
|
||||
"informative_pair_count": 20,
|
||||
"informative_pair_fraction": 0.7142857142857143,
|
||||
"invalid_cells": [],
|
||||
"pair_count": 28,
|
||||
"possibly_optimal_set_from_search_brackets": [
|
||||
"tp8_mns128_mbt16384",
|
||||
"tp8_mns128_mbt8192",
|
||||
"tp8_mns64_mbt16384",
|
||||
"tp8_mns64_mbt8192"
|
||||
],
|
||||
"possibly_optimal_set_size": 4,
|
||||
"random_top_set_hit_rate": 0.5,
|
||||
"tied_pair_count": 8,
|
||||
"top_set": [
|
||||
"tp8_mns128_mbt16384",
|
||||
"tp8_mns128_mbt8192",
|
||||
"tp8_mns64_mbt16384",
|
||||
"tp8_mns64_mbt8192"
|
||||
],
|
||||
"top_set_size": 4
|
||||
},
|
||||
{
|
||||
"best_score_req_s_per_gpu": 0.058958333333333335,
|
||||
"case": "qwen235b_decode_only",
|
||||
"cell_count": 8,
|
||||
"distinct_score_count": 2,
|
||||
"distinct_scores_req_s_per_gpu": [
|
||||
0.058958333333333335,
|
||||
0.05354166666666667
|
||||
],
|
||||
"fully_valid_cell_count": 7,
|
||||
"informative_pair_count": 15,
|
||||
"informative_pair_fraction": 0.5357142857142857,
|
||||
"invalid_cells": [
|
||||
"tp2_dp4_ep8_mns128_mbt384"
|
||||
],
|
||||
"pair_count": 28,
|
||||
"possibly_optimal_set_from_search_brackets": [
|
||||
"tp2_dp4_ep8_mns128_mbt256",
|
||||
"tp2_dp4_ep8_mns128_mbt384",
|
||||
"tp2_dp4_ep8_mns64_mbt256",
|
||||
"tp2_dp4_ep8_mns64_mbt384",
|
||||
"tp4_dp2_ep8_mns128_mbt256",
|
||||
"tp4_dp2_ep8_mns128_mbt384",
|
||||
"tp4_dp2_ep8_mns64_mbt256",
|
||||
"tp4_dp2_ep8_mns64_mbt384"
|
||||
],
|
||||
"possibly_optimal_set_size": 8,
|
||||
"random_top_set_hit_rate": 0.625,
|
||||
"tied_pair_count": 13,
|
||||
"top_set": [
|
||||
"tp2_dp4_ep8_mns128_mbt256",
|
||||
"tp2_dp4_ep8_mns128_mbt384",
|
||||
"tp2_dp4_ep8_mns64_mbt256",
|
||||
"tp4_dp2_ep8_mns128_mbt256",
|
||||
"tp4_dp2_ep8_mns128_mbt384"
|
||||
],
|
||||
"top_set_size": 5
|
||||
}
|
||||
],
|
||||
"schema": "frontier-multicase-ground-truth-v0"
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
# Qwen235B ground-truth audit
|
||||
|
||||
Objective: maximum SLO-feasible offered request throughput per GPU.
|
||||
This report contains real-machine data only; it makes no Frontier match claim.
|
||||
|
||||
| case | valid cells | score levels | top-set size | random top-set hit | informative pairs |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| qwen235b_prefill_only | 8/8 | 3 | 4/8 | 50.0% | 20/28 (71.4%) |
|
||||
| qwen235b_decode_only | 7/8 | 2 | 5/8 | 62.5% | 15/28 (53.6%) |
|
||||
|
||||
## Cells
|
||||
|
||||
### qwen235b_prefill_only
|
||||
|
||||
| cell | capacity bracket (req/s/GPU) | valid | probes | source |
|
||||
|---|---:|---:|---:|---|
|
||||
| tp4_mns128_mbt16384 | [0.106666667, 0.117500000) | yes | 6 | primary_search |
|
||||
| tp4_mns128_mbt8192 | [0.117500000, 0.133333333) | yes | 6 | primary_search |
|
||||
| tp4_mns64_mbt16384 | [0.106666667, 0.117500000) | yes | 6 | primary_search |
|
||||
| tp4_mns64_mbt8192 | [0.117500000, 0.133333333) | yes | 6 | primary_search |
|
||||
| tp8_mns128_mbt16384 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
| tp8_mns128_mbt8192 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
| tp8_mns64_mbt16384 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
| tp8_mns64_mbt8192 | [0.172708333, 0.177916667) | yes | 6 | primary_search |
|
||||
|
||||
Top set: `tp8_mns128_mbt16384, tp8_mns128_mbt8192, tp8_mns64_mbt16384, tp8_mns64_mbt8192`.
|
||||
Possibly optimal under binary-search brackets: `tp8_mns128_mbt16384, tp8_mns128_mbt8192, tp8_mns64_mbt16384, tp8_mns64_mbt8192`.
|
||||
|
||||
### qwen235b_decode_only
|
||||
|
||||
| cell | capacity bracket (req/s/GPU) | valid | probes | source |
|
||||
|---|---:|---:|---:|---|
|
||||
| tp2_dp4_ep8_mns128_mbt256 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp2_dp4_ep8_mns128_mbt384 | [0.058958333, 0.066875000) | no | 6 | partial_probe_before_failure |
|
||||
| tp2_dp4_ep8_mns64_mbt256 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp2_dp4_ep8_mns64_mbt384 | [0.053541667, 0.058958333) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns128_mbt256 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns128_mbt384 | [0.058958333, 0.066875000) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns64_mbt256 | [0.053541667, 0.058958333) | yes | 6 | primary_search |
|
||||
| tp4_dp2_ep8_mns64_mbt384 | [0.053541667, 0.058958333) | yes | 6 | primary_search |
|
||||
|
||||
Top set: `tp2_dp4_ep8_mns128_mbt256, tp2_dp4_ep8_mns128_mbt384, tp2_dp4_ep8_mns64_mbt256, tp4_dp2_ep8_mns128_mbt256, tp4_dp2_ep8_mns128_mbt384`.
|
||||
Possibly optimal under binary-search brackets: `tp2_dp4_ep8_mns128_mbt256, tp2_dp4_ep8_mns128_mbt384, tp2_dp4_ep8_mns64_mbt256, tp2_dp4_ep8_mns64_mbt384, tp4_dp2_ep8_mns128_mbt256, tp4_dp2_ep8_mns128_mbt384, tp4_dp2_ep8_mns64_mbt256, tp4_dp2_ep8_mns64_mbt384`.
|
||||
|
||||
## Interpretation guardrail
|
||||
|
||||
A Frontier top-set hit is insufficient by itself because the surfaces contain large ties. The later comparison must report selected-config regret and tie-aware pairwise ranking, and must keep invalid real cells visible.
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"calibration": {
|
||||
"fit_fixture": "coder_200_ts2",
|
||||
"fitted_a_tp": {
|
||||
"1": 0.7234810457606639,
|
||||
"2": 0.4680889959260082,
|
||||
"4": 0.3521372005220769
|
||||
},
|
||||
"holdout_fixture": "coder_200_ts3",
|
||||
"loss": "[log(G_raw_rerun(tp,scale2;a)/F_raw(tp,scale2))]^2 per TP",
|
||||
"refit_on_holdout": false
|
||||
},
|
||||
"rows": [
|
||||
{
|
||||
"agreement": 37,
|
||||
"false_feasible": 0,
|
||||
"false_infeasible": 55,
|
||||
"kendall_tau_b": 0.0,
|
||||
"mode": "uncalibrated/SLO-gated",
|
||||
"optimistic_real_regret": 0.25634517766497456,
|
||||
"pairwise_exact_sign_accuracy": 0.3787878787878788,
|
||||
"selected_cells": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64"
|
||||
],
|
||||
"worst_case_real_regret": 0.25634517766497456
|
||||
},
|
||||
{
|
||||
"agreement": 64,
|
||||
"false_feasible": 21,
|
||||
"false_infeasible": 7,
|
||||
"kendall_tau_b": 0.9668009539030813,
|
||||
"mode": "frozen-calibrated/SLO-gated",
|
||||
"optimistic_real_regret": 0.0,
|
||||
"pairwise_exact_sign_accuracy": 0.9393939393939394,
|
||||
"selected_cells": [
|
||||
"tp2_mns32",
|
||||
"tp2_mns64"
|
||||
],
|
||||
"worst_case_real_regret": 0.0076142131979695165
|
||||
}
|
||||
],
|
||||
"schema": "frontier-qwen30-calibration-audit-v0"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30B Frontier baseline audit
|
||||
|
||||
| mode | selected cells | worst real regret | Kendall tau-b | pair sign accuracy | feasibility (agree/FP/FN) |
|
||||
|---|---|---:|---:|---:|---:|
|
||||
| uncalibrated/SLO-gated | tp4_mns32, tp4_mns64 | 25.63% | 0.0000 | 37.88% | 37/0/55 |
|
||||
| frozen-calibrated/SLO-gated | tp2_mns32, tp2_mns64 | 0.76% | 0.9668 | 93.94% | 64/21/7 |
|
||||
|
||||
The calibrated mode applies a distinct end-to-end execution-time scale per TP: TP1=0.723481, TP2=0.468089, TP4=0.352137.
|
||||
|
||||
Those scales were fitted against real total throughput on `coder_200_ts2` and checked without refitting on `coder_200_ts3`. This validates within-workload transfer of the calibration, not zero-shot Frontier prediction across TP.
|
||||
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import json
|
||||
import os
|
||||
import statistics
|
||||
|
||||
import torch
|
||||
import torch.distributed as dist
|
||||
|
||||
|
||||
def main() -> None:
|
||||
local_rank = int(os.environ["LOCAL_RANK"])
|
||||
torch.cuda.set_device(local_rank)
|
||||
dist.init_process_group(backend="nccl")
|
||||
|
||||
# 16 tokens x 4096 hidden values in BF16: 128 KiB per rank.
|
||||
tensor = torch.ones((16, 4096), dtype=torch.bfloat16, device="cuda")
|
||||
for _ in range(10):
|
||||
dist.all_reduce(tensor)
|
||||
torch.cuda.synchronize()
|
||||
|
||||
samples_ms = []
|
||||
for _ in range(50):
|
||||
start = torch.cuda.Event(enable_timing=True)
|
||||
end = torch.cuda.Event(enable_timing=True)
|
||||
start.record()
|
||||
dist.all_reduce(tensor)
|
||||
end.record()
|
||||
end.synchronize()
|
||||
samples_ms.append(float(start.elapsed_time(end)))
|
||||
|
||||
if dist.get_rank() == 0:
|
||||
ordered = sorted(samples_ms)
|
||||
result = {
|
||||
"backend": "nccl",
|
||||
"collective": "all_reduce",
|
||||
"dtype": "bfloat16",
|
||||
"elements_per_rank": tensor.numel(),
|
||||
"bytes_per_rank": tensor.numel() * tensor.element_size(),
|
||||
"world_size": dist.get_world_size(),
|
||||
"warmup_iterations": 10,
|
||||
"measured_iterations": len(samples_ms),
|
||||
"mean_ms": statistics.fmean(samples_ms),
|
||||
"p50_ms": statistics.median(samples_ms),
|
||||
"p95_ms": ordered[int(0.95 * (len(ordered) - 1))],
|
||||
"min_ms": min(samples_ms),
|
||||
"max_ms": max(samples_ms),
|
||||
}
|
||||
print(json.dumps(result, sort_keys=True))
|
||||
|
||||
dist.destroy_process_group()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
diff --git a/frontier/profiling/moe/moe_vllm_kernel.py b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
--- a/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
+++ b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
@@ -232 +232 @@ def _invoke_kernel(
|
||||
- compute_type = tl.float16 # FP8 accumulates in FP16
|
||||
+ compute_type = tl.bfloat16
|
||||
@@ -0,0 +1,9 @@
|
||||
diff --git a/frontier/profiling/moe/moe_vllm_kernel.py b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
--- a/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
+++ b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
@@ -510 +510,4 @@ def profile_vllm_fused_moe(
|
||||
- config_dtype = get_config_dtype_str(base_dtype)
|
||||
+ config_dtype = get_config_dtype_str(
|
||||
+ base_dtype,
|
||||
+ use_fp8_w8a8=use_fp8,
|
||||
+ )
|
||||
@@ -0,0 +1,12 @@
|
||||
diff --git a/frontier/profiling/moe/moe_vllm_kernel.py b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
--- a/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
+++ b/frontier/profiling/moe/moe_vllm_kernel.py
|
||||
@@ -232 +232 @@ def _invoke_kernel(
|
||||
- compute_type = tl.float16 # FP8 accumulates in FP16
|
||||
+ compute_type = tl.bfloat16
|
||||
@@ -510 +510,4 @@ def profile_vllm_fused_moe(
|
||||
- config_dtype = get_config_dtype_str(base_dtype)
|
||||
+ config_dtype = get_config_dtype_str(
|
||||
+ base_dtype,
|
||||
+ use_fp8_w8a8=use_fp8,
|
||||
+ )
|
||||
189
runs/frontier-multicase-sufficiency-v0/smoke/run_gpu_smoke.sh
Normal file
189
runs/frontier-multicase-sufficiency-v0/smoke/run_gpu_smoke.sh
Normal file
@@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-${OUTPUT_ROOT}/Frontier-d9cfeb6}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8}"
|
||||
PROFILE_ROOT="${OUTPUT_ROOT}/profiles"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
RESULT_DIR="${OUTPUT_ROOT}/results"
|
||||
SERVER_PORT="${SERVER_PORT:-18900}"
|
||||
SKIP_LINEAR="${SKIP_LINEAR:-0}"
|
||||
SERVING_ONLY="${SERVING_ONLY:-0}"
|
||||
SERVED_MODEL="qwen3-235b-community-smoke"
|
||||
SERVER_PID=""
|
||||
|
||||
mkdir -p "${PROFILE_ROOT}" "${LOG_DIR}" "${RESULT_DIR}"
|
||||
exec > >(tee -a "${LOG_DIR}/gpu_smoke.log") 2>&1
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
for _ in $(seq 1 30); do
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
if [[ -z "${CUDA_VISIBLE_DEVICES:-}" ]]; then
|
||||
echo "ERROR: CUDA_VISIBLE_DEVICES must name exactly four allocated GPUs" >&2
|
||||
exit 1
|
||||
fi
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne 4 ]]; then
|
||||
echo "ERROR: expected four allocated GPUs, got CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "LAUNCH_ECHO host=$(hostname) gpus=${CUDA_VISIBLE_DEVICES} model=${MODEL_ROOT} frontier=d9cfeb6 vllm=community-0.10.2 transformers=4.55.2 backend=FLASHINFER execution=eager kv=BF16 spec=off tasks=representative-FP8-linear/attention/MoE+TP4-allreduce+TP4-model-load+one-request skip_linear=${SKIP_LINEAR} serving_only=${SERVING_ONLY} hard_wall_cap=1800s hard_gpu_cap=2_H20h"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
|
||||
test -x "${VENV_ROOT}/bin/python"
|
||||
test -f "${FRONTIER_ROOT}/pyproject.toml"
|
||||
test -f "${MODEL_ROOT}/config.json"
|
||||
|
||||
export PYTHONPATH="${FRONTIER_ROOT}${PYTHONPATH:+:${PYTHONPATH}}"
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export VLLM_USE_V1=1
|
||||
export VLLM_ATTENTION_BACKEND=FLASHINFER
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
|
||||
cd "${FRONTIER_ROOT}"
|
||||
|
||||
if [[ "${SERVING_ONLY}" -eq 0 && "${SKIP_LINEAR}" -eq 0 ]]; then
|
||||
echo "STAGE linear_op"
|
||||
timeout 300 "${VENV_ROOT}/bin/python" -m frontier.profiling.linear_op.main \
|
||||
--disable_ray \
|
||||
--models Qwen3-235B-A22B-FP8 \
|
||||
--num_gpus 1 \
|
||||
--max_tokens 16 \
|
||||
--num_tokens_list 16 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--profile_method cuda_event \
|
||||
--device h20 \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--is_moe \
|
||||
--yes
|
||||
elif [[ "${SERVING_ONLY}" -eq 0 ]]; then
|
||||
echo "STAGE linear_op SKIPPED (existing artifact retained)"
|
||||
fi
|
||||
|
||||
if [[ "${SERVING_ONLY}" -eq 0 ]]; then
|
||||
echo "STAGE attention"
|
||||
timeout 300 "${VENV_ROOT}/bin/python" -m frontier.profiling.attention.main \
|
||||
--disable_ray \
|
||||
--models Qwen3-235B-A22B-FP8 \
|
||||
--num_gpus 1 \
|
||||
--max_model_len 40960 \
|
||||
--max_seq_len 128 \
|
||||
--min_batch_size 1 \
|
||||
--max_batch_size 1 \
|
||||
--batch_size_list 1 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--max_pipeline_parallel_size 1 \
|
||||
--attention_backend FLASHINFER \
|
||||
--block_size 16 \
|
||||
--profile_only_prefill \
|
||||
--fixed_chunked_prefill_size 128 \
|
||||
--device h20 \
|
||||
--profile_method cuda_event \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--yes
|
||||
|
||||
echo "STAGE moe"
|
||||
timeout 300 "${VENV_ROOT}/bin/python" -m frontier.profiling.moe.main \
|
||||
--disable_ray \
|
||||
--models Qwen3-235B-A22B-FP8 \
|
||||
--device h20 \
|
||||
--num_gpus 1 \
|
||||
--max_tokens 16 \
|
||||
--num_tokens_list 16 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--expert_parallel_sizes 1 \
|
||||
--load_distributions uniform \
|
||||
--num_samples_per_distribution 1 \
|
||||
--routing_runtime_path standard_fused_topk \
|
||||
--gating_runtime_context prefill_hot \
|
||||
--profile_method cuda_event \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--yes
|
||||
|
||||
echo "STAGE allreduce"
|
||||
timeout 180 "${VENV_ROOT}/bin/torchrun" \
|
||||
--standalone \
|
||||
--nnodes=1 \
|
||||
--nproc-per-node=4 \
|
||||
"${OUTPUT_ROOT}/scripts/allreduce_smoke.py" \
|
||||
| tee "${RESULT_DIR}/allreduce_tp4.jsonl"
|
||||
else
|
||||
echo "STAGES Frontier profiles and allreduce SKIPPED (existing artifacts retained)"
|
||||
fi
|
||||
|
||||
echo "STAGE serving"
|
||||
setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
|
||||
--host 127.0.0.1 \
|
||||
--port "${SERVER_PORT}" \
|
||||
--served-model-name "${SERVED_MODEL}" \
|
||||
--tensor-parallel-size 4 \
|
||||
--disable-custom-all-reduce \
|
||||
--quantization fp8 \
|
||||
--gpu-memory-utilization 0.80 \
|
||||
--kv-cache-dtype auto \
|
||||
--max-model-len 40960 \
|
||||
--max-num-batched-tokens 8192 \
|
||||
--max-num-seqs 64 \
|
||||
--no-enable-prefix-caching \
|
||||
--enable-chunked-prefill \
|
||||
--enforce-eager \
|
||||
--disable-log-requests \
|
||||
> "${LOG_DIR}/server.log" 2>&1 &
|
||||
SERVER_PID=$!
|
||||
|
||||
READY=0
|
||||
for _ in $(seq 1 180); do
|
||||
if curl -fsS --max-time 2 "http://127.0.0.1:${SERVER_PORT}/v1/models" \
|
||||
> "${RESULT_DIR}/models.json" 2>/dev/null; then
|
||||
READY=1
|
||||
break
|
||||
fi
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
echo "ERROR: vLLM server exited before readiness" >&2
|
||||
tail -200 "${LOG_DIR}/server.log" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
if [[ "${READY}" -ne 1 ]]; then
|
||||
echo "ERROR: vLLM server did not become ready within 900 seconds" >&2
|
||||
tail -200 "${LOG_DIR}/server.log" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
curl -fsS --max-time 120 \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"qwen3-235b-community-smoke","prompt":"Hello","max_tokens":1,"temperature":0}' \
|
||||
"http://127.0.0.1:${SERVER_PORT}/v1/completions" \
|
||||
| tee "${RESULT_DIR}/one_request.json"
|
||||
echo
|
||||
jq -e '.choices | length == 1' "${RESULT_DIR}/one_request.json" >/dev/null
|
||||
|
||||
cleanup
|
||||
SERVER_PID=""
|
||||
|
||||
find "${PROFILE_ROOT}" -type f -maxdepth 5 -print -exec sha256sum {} \;
|
||||
sha256sum \
|
||||
"${RESULT_DIR}/allreduce_tp4.jsonl" \
|
||||
"${RESULT_DIR}/models.json" \
|
||||
"${RESULT_DIR}/one_request.json" \
|
||||
> "${RESULT_DIR}/results.sha256"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "GPU_SMOKE_COMPLETE"
|
||||
66
runs/frontier-multicase-sufficiency-v0/smoke/setup_env.sh
Executable file
66
runs/frontier-multicase-sufficiency-v0/smoke/setup_env.sh
Executable file
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-${OUTPUT_ROOT}/Frontier-d9cfeb6}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
|
||||
export UV_HTTP_TIMEOUT="${UV_HTTP_TIMEOUT:-300}"
|
||||
|
||||
mkdir -p "${LOG_DIR}" "${OUTPUT_ROOT}/provenance"
|
||||
exec > >(tee -a "${LOG_DIR}/setup_env.log") 2>&1
|
||||
|
||||
echo "SETUP_ENV output=${OUTPUT_ROOT} frontier=${FRONTIER_ROOT} venv=${VENV_ROOT} python=/usr/local/bin/python3.12 vllm=0.10.2 transformers=4.55.2"
|
||||
|
||||
if [[ ! -f "${FRONTIER_ROOT}/pyproject.toml" ]]; then
|
||||
echo "ERROR: Frontier source is missing at ${FRONTIER_ROOT}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
uv venv --clear --python /usr/local/bin/python3.12 "${VENV_ROOT}"
|
||||
uv pip install \
|
||||
--python "${VENV_ROOT}/bin/python" \
|
||||
"vllm==0.10.2" \
|
||||
"transformers==4.55.2" \
|
||||
"flashinfer-python>=0.3,<0.4" \
|
||||
-e "${FRONTIER_ROOT}[test]"
|
||||
|
||||
"${VENV_ROOT}/bin/python" - <<'PY' | tee "${OUTPUT_ROOT}/provenance/environment.json"
|
||||
import importlib.metadata as metadata
|
||||
import json
|
||||
import platform
|
||||
|
||||
import flashinfer
|
||||
import frontier
|
||||
import torch
|
||||
import vllm
|
||||
|
||||
record = {
|
||||
"python": platform.python_version(),
|
||||
"torch": torch.__version__,
|
||||
"torch_cuda": torch.version.cuda,
|
||||
"vllm_import_version": vllm.__version__,
|
||||
"vllm_metadata_version": metadata.version("vllm"),
|
||||
"vllm_path": vllm.__file__,
|
||||
"transformers_metadata_version": metadata.version("transformers"),
|
||||
"tokenizers_metadata_version": metadata.version("tokenizers"),
|
||||
"flashinfer_metadata_version": metadata.version("flashinfer-python"),
|
||||
"flashinfer_path": flashinfer.__file__,
|
||||
"frontier_metadata_version": metadata.version("frontier-simulator"),
|
||||
"frontier_path": frontier.__path__[0],
|
||||
}
|
||||
print(json.dumps(record, indent=2, sort_keys=True))
|
||||
assert record["vllm_import_version"] == "0.10.2", record
|
||||
assert record["vllm_metadata_version"] == "0.10.2", record
|
||||
assert record["transformers_metadata_version"] == "4.55.2", record
|
||||
PY
|
||||
|
||||
uv pip freeze --python "${VENV_ROOT}/bin/python" > "${OUTPUT_ROOT}/provenance/requirements.freeze.txt"
|
||||
sha256sum \
|
||||
"${OUTPUT_ROOT}/provenance/environment.json" \
|
||||
"${OUTPUT_ROOT}/provenance/requirements.freeze.txt" \
|
||||
> "${OUTPUT_ROOT}/provenance/environment.sha256"
|
||||
|
||||
echo "SETUP_ENV_COMPLETE"
|
||||
@@ -0,0 +1,71 @@
|
||||
import importlib.util
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
MODULE_PATH = Path(__file__).with_name("audit_ground_truth.py")
|
||||
SPEC = importlib.util.spec_from_file_location("audit_ground_truth", MODULE_PATH)
|
||||
audit = importlib.util.module_from_spec(SPEC)
|
||||
assert SPEC.loader is not None
|
||||
SPEC.loader.exec_module(audit)
|
||||
|
||||
|
||||
class AuditGroundTruthTest(unittest.TestCase):
|
||||
def test_case_summary_exposes_tied_top_set(self):
|
||||
rows = [
|
||||
{
|
||||
"cell_id": "a",
|
||||
"score_req_s_per_gpu": 2.0,
|
||||
"capacity_lower_bound_req_s_per_gpu": 2.0,
|
||||
"capacity_upper_bound_req_s_per_gpu": 2.1,
|
||||
"fully_valid": True,
|
||||
},
|
||||
{
|
||||
"cell_id": "b",
|
||||
"score_req_s_per_gpu": 2.0,
|
||||
"capacity_lower_bound_req_s_per_gpu": 2.0,
|
||||
"capacity_upper_bound_req_s_per_gpu": 2.2,
|
||||
"fully_valid": True,
|
||||
},
|
||||
{
|
||||
"cell_id": "c",
|
||||
"score_req_s_per_gpu": 1.0,
|
||||
"capacity_lower_bound_req_s_per_gpu": 1.0,
|
||||
"capacity_upper_bound_req_s_per_gpu": 1.5,
|
||||
"fully_valid": False,
|
||||
},
|
||||
]
|
||||
|
||||
summary = audit.summarize_case("test", rows)
|
||||
|
||||
self.assertEqual(summary["top_set"], ["a", "b"])
|
||||
self.assertEqual(summary["distinct_score_count"], 2)
|
||||
self.assertEqual(summary["tied_pair_count"], 1)
|
||||
self.assertEqual(summary["informative_pair_count"], 2)
|
||||
self.assertAlmostEqual(summary["random_top_set_hit_rate"], 2 / 3)
|
||||
self.assertEqual(summary["invalid_cells"], ["c"])
|
||||
self.assertEqual(
|
||||
summary["possibly_optimal_set_from_search_brackets"], ["a", "b"]
|
||||
)
|
||||
|
||||
def test_config_gpu_count_includes_data_parallelism(self):
|
||||
result = {
|
||||
"config_patch": {
|
||||
"flag_patch": {
|
||||
"tensor-parallel-size": 2,
|
||||
"data-parallel-size": 4,
|
||||
"expert-parallel-size": 8,
|
||||
"max-num-seqs": 128,
|
||||
"max-num-batched-tokens": 384,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
config = audit.config_from_result(result)
|
||||
|
||||
self.assertEqual(config["gpu_count"], 8)
|
||||
self.assertEqual(audit.cell_id(config), "tp2_dp4_ep8_mns128_mbt384")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
321
runs/frontier-multicase-sufficiency-v1/analyze_t0_surface.py
Normal file
321
runs/frontier-multicase-sufficiency-v1/analyze_t0_surface.py
Normal file
@@ -0,0 +1,321 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate and compare frozen T0 Frontier and two-round real surfaces."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
SLOS = ("tpot_40ms", "tpot_120ms", "tpot_150ms", "tpot_180ms")
|
||||
RATE_LATTICE = (0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20)
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
|
||||
|
||||
def rate_key(rate: float) -> str:
|
||||
return f"r{rate:.2f}".replace(".", "p")
|
||||
|
||||
|
||||
def find_real_result(roots: list[Path], name: str, round_id: int, rate: float) -> Path:
|
||||
relative = Path(name) / f"round{round_id}/results" / f"{rate_key(rate)}.json"
|
||||
matches = [root / relative for root in roots if (root / relative).is_file()]
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f"expected one real result for {relative}, got {matches}")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def capacity(loads: list[dict[str, Any]], slo: str, field: str) -> float | None:
|
||||
values = [float(load["rate"]) for load in loads if bool(load[field][slo]["feasible"])]
|
||||
return max(values) if values else None
|
||||
|
||||
|
||||
def real_boundary_status(loads: list[dict[str, Any]], slo: str) -> str:
|
||||
labels = {
|
||||
float(load["rate"]): bool(load["real_conservative"][slo]["feasible"])
|
||||
for load in loads
|
||||
}
|
||||
ordered = [(rate, labels[rate]) for rate in RATE_LATTICE if rate in labels]
|
||||
if any(not left and right for (_, left), (_, right) in zip(ordered, ordered[1:])):
|
||||
return "non_monotonic_requires_full_lattice"
|
||||
if len(labels) == len(RATE_LATTICE):
|
||||
return "complete_lattice"
|
||||
for lower, upper in zip(RATE_LATTICE, RATE_LATTICE[1:]):
|
||||
if labels.get(lower) is True and labels.get(upper) is False:
|
||||
return "adjacent_transition_bracketed"
|
||||
if labels.get(RATE_LATTICE[-1]) is True:
|
||||
return "upper_lattice_reached"
|
||||
if labels.get(RATE_LATTICE[0]) is False and not any(labels.values()):
|
||||
return "lowest_anchor_infeasible"
|
||||
return "unbracketed_requires_expansion"
|
||||
|
||||
|
||||
def kendall_tau_b(left: list[float], right: list[float]) -> float | None:
|
||||
concordant = discordant = left_ties = right_ties = 0
|
||||
for i in range(len(left)):
|
||||
for j in range(i + 1, len(left)):
|
||||
x = (left[i] > left[j]) - (left[i] < left[j])
|
||||
y = (right[i] > right[j]) - (right[i] < right[j])
|
||||
if x == 0 and y == 0:
|
||||
continue
|
||||
if x == 0:
|
||||
left_ties += 1
|
||||
elif y == 0:
|
||||
right_ties += 1
|
||||
elif x == y:
|
||||
concordant += 1
|
||||
else:
|
||||
discordant += 1
|
||||
denominator = math.sqrt(
|
||||
(concordant + discordant + left_ties)
|
||||
* (concordant + discordant + right_ties)
|
||||
)
|
||||
return (concordant - discordant) / denominator if denominator else None
|
||||
|
||||
|
||||
def sign(left: float, right: float) -> int:
|
||||
return (left > right) - (left < right)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-freeze", type=Path, required=True)
|
||||
parser.add_argument("--real-plan", type=Path, required=True)
|
||||
parser.add_argument("--real-root", type=Path, action="append", required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
freeze = json.loads(args.frontier_freeze.read_text())
|
||||
plan = json.loads(args.real_plan.read_text())
|
||||
real_roots = [root.resolve() for root in args.real_root]
|
||||
if freeze.get("status") != "frozen_before_real_surface" or len(freeze.get("config_results") or []) != 8:
|
||||
raise ValueError("Frontier freeze is incomplete")
|
||||
if plan.get("frontier_freeze", {}).get("sha256") != sha256(args.frontier_freeze):
|
||||
raise ValueError("real plan does not point to this Frontier freeze")
|
||||
sim_by_name = {item["config"]["name"]: item for item in freeze["config_results"]}
|
||||
|
||||
cells = []
|
||||
for cell in plan["cells"]:
|
||||
config = cell["config"]
|
||||
name = config["name"]
|
||||
sim_loads = {float(load["offered_request_rate"]): load for load in sim_by_name[name]["loads"]}
|
||||
loads = []
|
||||
for rate in cell["rates"]:
|
||||
round_summaries = []
|
||||
files = []
|
||||
for round_id in (1, 2):
|
||||
path = find_real_result(real_roots, name, round_id, float(rate))
|
||||
payload = json.loads(path.read_text())
|
||||
if payload.get("schema") != "qwen235b-t0-rate-anchor-v1":
|
||||
raise ValueError(f"bad real result schema: {path}")
|
||||
if payload["summary"]["completed"] != 64 or payload["summary"]["failed"] != 0:
|
||||
raise ValueError(f"incomplete real anchor: {path}")
|
||||
if float(payload["workload"]["offered_request_rate"]) != float(rate):
|
||||
raise ValueError(f"offered-rate drift: {path}")
|
||||
round_summaries.append(payload["summary"]["slos"])
|
||||
files.append({"path": str(path), "sha256": sha256(path)})
|
||||
conservative = {
|
||||
slo: {
|
||||
"feasible": all(summary[slo]["feasible"] for summary in round_summaries),
|
||||
"round_pass_rates": [summary[slo]["pass_rate"] for summary in round_summaries],
|
||||
}
|
||||
for slo in SLOS
|
||||
}
|
||||
loads.append(
|
||||
{
|
||||
"rate": float(rate),
|
||||
"real_conservative": conservative,
|
||||
"sim": sim_loads[float(rate)]["slos"],
|
||||
"real_files": files,
|
||||
}
|
||||
)
|
||||
cells.append({"config": config, "loads": loads})
|
||||
|
||||
comparisons = {}
|
||||
for slo in SLOS:
|
||||
records = []
|
||||
for cell in cells:
|
||||
real = capacity(cell["loads"], slo, "real_conservative")
|
||||
sim_values = [
|
||||
float(load["offered_request_rate"])
|
||||
for load in sim_by_name[cell["config"]["name"]]["loads"]
|
||||
if bool(load["slos"][slo]["feasible"])
|
||||
]
|
||||
sim = max(sim_values) if sim_values else None
|
||||
tp = int(cell["config"]["tp"])
|
||||
boundary = real_boundary_status(cell["loads"], slo)
|
||||
records.append(
|
||||
{
|
||||
"config": cell["config"],
|
||||
"real_capacity": real,
|
||||
"sim_capacity": sim,
|
||||
"real_capacity_per_gpu": real / tp if real is not None else None,
|
||||
"sim_capacity_per_gpu": sim / tp if sim is not None else None,
|
||||
"real_boundary_status": boundary,
|
||||
"expansion_required": boundary in {
|
||||
"non_monotonic_requires_full_lattice",
|
||||
"unbracketed_requires_expansion",
|
||||
},
|
||||
}
|
||||
)
|
||||
rankable = [
|
||||
row
|
||||
for row in records
|
||||
if not row["expansion_required"]
|
||||
and row["real_capacity_per_gpu"] is not None
|
||||
and row["sim_capacity_per_gpu"] is not None
|
||||
]
|
||||
tau = kendall_tau_b(
|
||||
[row["real_capacity_per_gpu"] for row in rankable],
|
||||
[row["sim_capacity_per_gpu"] for row in rankable],
|
||||
)
|
||||
pairwise = {
|
||||
"all_pairs": 0,
|
||||
"exact_sign_matches": 0,
|
||||
"real_non_tie_pairs": 0,
|
||||
"real_non_tie_direction_matches": 0,
|
||||
}
|
||||
for i, left in enumerate(rankable):
|
||||
for right in rankable[i + 1 :]:
|
||||
real_sign = sign(left["real_capacity_per_gpu"], right["real_capacity_per_gpu"])
|
||||
sim_sign = sign(left["sim_capacity_per_gpu"], right["sim_capacity_per_gpu"])
|
||||
pairwise["all_pairs"] += 1
|
||||
pairwise["exact_sign_matches"] += real_sign == sim_sign
|
||||
if real_sign:
|
||||
pairwise["real_non_tie_pairs"] += 1
|
||||
pairwise["real_non_tie_direction_matches"] += real_sign == sim_sign
|
||||
real_best = max((row["real_capacity_per_gpu"] for row in records if row["real_capacity_per_gpu"] is not None), default=None)
|
||||
sim_best = max((row["sim_capacity_per_gpu"] for row in records if row["sim_capacity_per_gpu"] is not None), default=None)
|
||||
sim_top = [row for row in records if sim_best is not None and row["sim_capacity_per_gpu"] == sim_best]
|
||||
real_top = [row for row in records if real_best is not None and row["real_capacity_per_gpu"] == real_best]
|
||||
optimistic_regret = worst_regret = None
|
||||
if real_best is not None and sim_top and all(row["real_capacity_per_gpu"] is not None for row in sim_top):
|
||||
regrets = [(real_best - row["real_capacity_per_gpu"]) / real_best for row in sim_top]
|
||||
optimistic_regret = min(regrets)
|
||||
worst_regret = max(regrets)
|
||||
confusion = {"anchors": 0, "match": 0, "false_feasible": 0, "false_infeasible": 0}
|
||||
for cell in cells:
|
||||
for load in cell["loads"]:
|
||||
real_feasible = bool(load["real_conservative"][slo]["feasible"])
|
||||
sim_feasible = bool(load["sim"][slo]["feasible"])
|
||||
confusion["anchors"] += 1
|
||||
confusion["match"] += real_feasible == sim_feasible
|
||||
confusion["false_feasible"] += sim_feasible and not real_feasible
|
||||
confusion["false_infeasible"] += real_feasible and not sim_feasible
|
||||
comparisons[slo] = {
|
||||
"records": records,
|
||||
"kendall_tau_b": tau,
|
||||
"pairwise": pairwise,
|
||||
"anchor_confusion": confusion,
|
||||
"real_top_set": [row["config"]["name"] for row in real_top],
|
||||
"sim_top_set": [row["config"]["name"] for row in sim_top],
|
||||
"top_set_intersection": sorted(
|
||||
{row["config"]["name"] for row in real_top}
|
||||
& {row["config"]["name"] for row in sim_top}
|
||||
),
|
||||
"top_set_exact_match": {
|
||||
row["config"]["name"] for row in real_top
|
||||
} == {row["config"]["name"] for row in sim_top},
|
||||
"optimistic_regret": optimistic_regret,
|
||||
"worst_tie_break_regret": worst_regret,
|
||||
}
|
||||
|
||||
run_costs = []
|
||||
for root in real_roots:
|
||||
config_names = [child.name for child in root.iterdir() if child.is_dir() and child.name in sim_by_name]
|
||||
if len(config_names) != 1:
|
||||
raise ValueError(f"expected one config directory in real root {root}, got {config_names}")
|
||||
config = sim_by_name[config_names[0]]["config"]
|
||||
remote_run = root.parents[2] / "remote_run"
|
||||
started = datetime.fromisoformat((remote_run / "started_at").read_text().strip())
|
||||
finished = datetime.fromisoformat((remote_run / "finished_at").read_text().strip())
|
||||
wall_seconds = (finished - started).total_seconds()
|
||||
run_costs.append(
|
||||
{
|
||||
"run_id": root.parents[2].name,
|
||||
"config": config_names[0],
|
||||
"gpu_count": int(config["tp"]),
|
||||
"wall_seconds": wall_seconds,
|
||||
"h20_gpu_hours": wall_seconds * int(config["tp"]) / 3600,
|
||||
"started_at": started.isoformat(),
|
||||
"finished_at": finished.isoformat(),
|
||||
}
|
||||
)
|
||||
fresh_server_anchors = 2 * sum(len(cell["loads"]) for cell in cells)
|
||||
real_execution_cost = {
|
||||
"accepted_fleet_jobs": len(run_costs),
|
||||
"fresh_server_anchors": fresh_server_anchors,
|
||||
"measured_requests": fresh_server_anchors * 64,
|
||||
"warmup_requests": 2
|
||||
* sum(
|
||||
min(32, max(4, math.ceil(float(load["rate"]) * 20)))
|
||||
for cell in cells
|
||||
for load in cell["loads"]
|
||||
),
|
||||
"accepted_h20_gpu_hours": sum(run["h20_gpu_hours"] for run in run_costs),
|
||||
"campaign_wall_span_seconds": (
|
||||
max(datetime.fromisoformat(run["finished_at"]) for run in run_costs)
|
||||
- min(datetime.fromisoformat(run["started_at"]) for run in run_costs)
|
||||
).total_seconds(),
|
||||
"runs": run_costs,
|
||||
}
|
||||
|
||||
output = {
|
||||
"schema": "qwen235b-t0-simulator-real-comparison-v1",
|
||||
"frontier_freeze_sha256": sha256(args.frontier_freeze),
|
||||
"real_plan_sha256": sha256(args.real_plan),
|
||||
"real_execution_cost": real_execution_cost,
|
||||
"cells": cells,
|
||||
"comparisons": comparisons,
|
||||
}
|
||||
args.output_root.mkdir(parents=True, exist_ok=True)
|
||||
write_json(args.output_root / "comparison.json", output)
|
||||
with (args.output_root / "capacity.csv").open("w", newline="") as target:
|
||||
writer = csv.DictWriter(
|
||||
target,
|
||||
fieldnames=[
|
||||
"slo",
|
||||
"config",
|
||||
"tp",
|
||||
"mns",
|
||||
"mbt",
|
||||
"real_capacity_per_gpu",
|
||||
"sim_capacity_per_gpu",
|
||||
"real_boundary_status",
|
||||
"expansion_required",
|
||||
],
|
||||
)
|
||||
writer.writeheader()
|
||||
for slo, comparison in comparisons.items():
|
||||
for row in comparison["records"]:
|
||||
writer.writerow(
|
||||
{
|
||||
"slo": slo,
|
||||
"config": row["config"]["name"],
|
||||
"tp": row["config"]["tp"],
|
||||
"mns": row["config"]["mns"],
|
||||
"mbt": row["config"]["mbt"],
|
||||
"real_capacity_per_gpu": row["real_capacity_per_gpu"],
|
||||
"sim_capacity_per_gpu": row["sim_capacity_per_gpu"],
|
||||
"real_boundary_status": row["real_boundary_status"],
|
||||
"expansion_required": row["expansion_required"],
|
||||
}
|
||||
)
|
||||
print(args.output_root / "comparison.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,61 @@
|
||||
{
|
||||
"schema": "qwen235b-t0-real-exclusions-v1",
|
||||
"policy": "Exclude the complete output directory whenever another process shares any allocated GPU or benchmark endpoint during warmup or a measured anchor.",
|
||||
"excluded_attempts": [
|
||||
{
|
||||
"reason": "Fleet monitor re-probed during model load and oversubscribed both TP4 allocations; the later MNS128 attempts overlapped the MNS64 warmup and start of r0p10.",
|
||||
"remote_quarantine": "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/contaminated-queue-race-20260716T045500Z",
|
||||
"run_ids": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T045338281933Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T045339503446Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1-20260716T045458846499Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1-20260716T045500253803Z"
|
||||
]
|
||||
},
|
||||
{
|
||||
"reason": "A dispatch command still copying the source was mistakenly resubmitted. The duplicate controllers reused the first pair's open ports and issued concurrent warmup requests, so both the original and duplicate outputs were rejected.",
|
||||
"remote_quarantine": "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/contaminated-duplicate-dispatch-20260716T0501Z",
|
||||
"run_ids": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T050017477963Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T050018748070Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T050123485776Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T050124800483Z"
|
||||
]
|
||||
},
|
||||
{
|
||||
"reason": "Clean diagnostic run exposed cross-anchor execution-state leakage: TPOT at 1.60 req/s failed after a sustained 0.10 anchor but passed after a 2.40 anchor. The multi-rate-per-server contract does not match Frontier's independent rate runs, so the complete attempt is diagnostic-only.",
|
||||
"remote_quarantine": "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/diagnostic-cross-anchor-state-20260716T0504Z",
|
||||
"run_ids": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T050405703312Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T050407104293Z"
|
||||
]
|
||||
}
|
||||
],
|
||||
"accepted_attempts": [
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1-20260716T054640909739Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1-20260716T054642115787Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt8192-expansion-r3p20-20260716-v1-20260716T062904775741Z",
|
||||
"qwen235b-t0-real-tp4_mns64_mbt16384-expansion-r3p20-20260716-v1-20260716T062906244819Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1-20260716T063826749477Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1-20260716T063827805737Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt8192-expansion-r3p20-20260716-v1-20260716T072121119478Z",
|
||||
"qwen235b-t0-real-tp4_mns128_mbt16384-expansion-r3p20-20260716-v1-20260716T072122501851Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1-20260716T073151443129Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r2p40-20260716-v1-20260716T081536141326Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r3p20-20260716-v1-20260716T082603837807Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt16384-20260716-v1-20260716T083632620015Z",
|
||||
"qwen235b-t0-real-tp8_mns64_mbt16384-expansion-r2p40-20260716-v1-20260716T092040851202Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt8192-20260716-v1-20260716T093245831612Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt8192-expansion-r2p40-20260716-v1-20260716T101734683241Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt16384-20260716-v1-20260716T102855604708Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r2p40-20260716-v1-20260716T111437770615Z",
|
||||
"qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r3p20-20260716-v1-20260716T112608807037Z"
|
||||
],
|
||||
"accepted_warnings": [
|
||||
{
|
||||
"run_id": "qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1-20260716T073151443129Z",
|
||||
"anchor": "round1/r1p20",
|
||||
"reason": "One rank-7 TCPStore heartbeat warning occurred 1.1 s after the complete 64-request result was written, while all eight workers were terminating after the runner intentionally stopped the fresh server. The application then shut down normally; no request, engine, OOM, or in-window NCCL failure occurred."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,299 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"preregistered_rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6,
|
||||
2.4,
|
||||
3.2
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_protocol_amendment": {
|
||||
"contract": "fresh server and target-rate warmup for every config-rate-round",
|
||||
"reason": "observed cross-anchor GPU/kernel/batch warm-state leakage",
|
||||
"timing": "after excluded multi-rate-per-server diagnostic and before any accepted real surface cell"
|
||||
},
|
||||
"expansion_artifact_roots": [
|
||||
"artifacts/t0-real-expansion-r2p40-v1",
|
||||
"artifacts/t0-real-expansion-r3p20-v1"
|
||||
],
|
||||
"expected_total_h20_gpu_hours": 39.98888888888888,
|
||||
"frontier_freeze": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/frontier-t0-surface-v1/frontier_surface_frozen.json",
|
||||
"sha256": "801aa36451c8647f71cc87011144622d2203786e82f189ed1375d964399b106a"
|
||||
},
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
"post_pilot_sensitivities": [
|
||||
"tpot_120ms",
|
||||
"tpot_150ms",
|
||||
"tpot_180ms"
|
||||
],
|
||||
"preregistered_plan": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/t0-real-surface-v1/real-plan.json",
|
||||
"sha256": "f7acb45d1183a6026dba4bbf9bc2d7578452b815b8de0035ca732512f3d9d47a"
|
||||
},
|
||||
"schema": "qwen235b-t0-real-executed-plan-v1",
|
||||
"selection_slo": "tpot_150ms",
|
||||
"selection_timing": "rate anchors frozen after complete simulator surface and before any accepted real surface cell",
|
||||
"status": "complete",
|
||||
"strict_preregistered_slo": "tpot_40ms"
|
||||
}
|
||||
@@ -0,0 +1,187 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 2.2125,
|
||||
"expected_wall_seconds": 1991.25,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
},
|
||||
{
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 4.541666666666667,
|
||||
"expected_wall_seconds": 2043.75,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2
|
||||
}
|
||||
],
|
||||
"expected_total_h20_gpu_hours": 27.016666666666666,
|
||||
"frontier_freeze": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/frontier-t0-surface-v1/frontier_surface_frozen.json",
|
||||
"sha256": "801aa36451c8647f71cc87011144622d2203786e82f189ed1375d964399b106a"
|
||||
},
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
"post_pilot_sensitivities": [
|
||||
"tpot_120ms",
|
||||
"tpot_150ms",
|
||||
"tpot_180ms"
|
||||
],
|
||||
"schema": "qwen235b-t0-real-plan-v1",
|
||||
"selection_slo": "tpot_150ms",
|
||||
"selection_timing": "after_complete_simulator_freeze_before_any_real_surface_cell",
|
||||
"strict_preregistered_slo": "tpot_40ms"
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns64_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt8192",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 1,
|
||||
"moe_tp": 4,
|
||||
"name": "tp4_mns128_mbt16384",
|
||||
"num_gpu_blocks": 26101,
|
||||
"tp": 4
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 3.284259259259259,
|
||||
"expected_wall_seconds": 2955.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.6,
|
||||
2.4
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.60": 32,
|
||||
"2.40": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 64,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns64_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 8192,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt8192",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
},
|
||||
{
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"config": {
|
||||
"mbt": 16384,
|
||||
"mns": 128,
|
||||
"moe_ep": 8,
|
||||
"moe_tp": 1,
|
||||
"name": "tp8_mns128_mbt16384",
|
||||
"num_gpu_blocks": 62351,
|
||||
"tp": 8
|
||||
},
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
"expected_h20_gpu_hours": 6.712962962962962,
|
||||
"expected_wall_seconds": 3020.833333333333,
|
||||
"rates": [
|
||||
0.1,
|
||||
1.2,
|
||||
1.6
|
||||
],
|
||||
"requests_per_anchor": 64,
|
||||
"rounds": 2,
|
||||
"target_rate_warmup_requests": {
|
||||
"0.10": 4,
|
||||
"1.20": 24,
|
||||
"1.60": 32
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_protocol_amendment": {
|
||||
"contract": "fresh server and target-rate warmup for every config-rate-round",
|
||||
"reason": "observed cross-anchor GPU/kernel/batch warm-state leakage",
|
||||
"timing": "after excluded multi-rate-per-server diagnostic and before any accepted real surface cell"
|
||||
},
|
||||
"expected_total_h20_gpu_hours": 39.98888888888888,
|
||||
"frontier_freeze": {
|
||||
"path": "/home/gahow/phd/aituner/runs/frontier-multicase-sufficiency-v1/artifacts/frontier-t0-surface-v1/frontier_surface_frozen.json",
|
||||
"sha256": "801aa36451c8647f71cc87011144622d2203786e82f189ed1375d964399b106a"
|
||||
},
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
"post_pilot_sensitivities": [
|
||||
"tpot_120ms",
|
||||
"tpot_150ms",
|
||||
"tpot_180ms"
|
||||
],
|
||||
"schema": "qwen235b-t0-real-plan-v1",
|
||||
"selection_slo": "tpot_150ms",
|
||||
"selection_timing": "rate anchors frozen after complete simulator surface and before any accepted real surface cell",
|
||||
"strict_preregistered_slo": "tpot_40ms"
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Create an immutable Frontier profile root with measured decode rows."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODEL = "Qwen3-235B-A22B-FP8"
|
||||
ATTENTION_RELATIVE_PATH = Path("compute/h20") / MODEL / "attention.csv"
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--base-profile-root", type=Path, required=True)
|
||||
parser.add_argument("--decode-attention-csv", type=Path, required=True)
|
||||
parser.add_argument("--true-mixed-attention-csv", type=Path)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def read_csv(path: Path) -> tuple[list[str], list[dict[str, str]]]:
|
||||
with path.open(newline="") as source:
|
||||
reader = csv.DictReader(source)
|
||||
if reader.fieldnames is None:
|
||||
raise ValueError(f"missing CSV header: {path}")
|
||||
return reader.fieldnames, list(reader)
|
||||
|
||||
|
||||
def is_true(value: str) -> bool:
|
||||
return value.strip().lower() == "true"
|
||||
|
||||
|
||||
def profile_hashes(root: Path) -> dict[str, str]:
|
||||
return {
|
||||
str(path.relative_to(root)): sha256(path)
|
||||
for path in sorted(root.rglob("*"))
|
||||
if path.is_file() and path.name != "profile_closure_manifest.json"
|
||||
}
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
base_root = args.base_profile_root.resolve()
|
||||
decode_csv = args.decode_attention_csv.resolve()
|
||||
output_root = args.output_root.resolve()
|
||||
base_attention = base_root / ATTENTION_RELATIVE_PATH
|
||||
|
||||
if output_root.exists():
|
||||
raise FileExistsError(f"refusing to overwrite profile root: {output_root}")
|
||||
for path in (base_attention, decode_csv):
|
||||
if not path.is_file():
|
||||
raise FileNotFoundError(path)
|
||||
|
||||
base_fields, base_rows = read_csv(base_attention)
|
||||
decode_fields, decode_source_rows = read_csv(decode_csv)
|
||||
if base_fields != decode_fields:
|
||||
raise ValueError("base and decode attention CSV schemas differ")
|
||||
if not base_rows or any(not is_true(row["is_prefill"]) for row in base_rows):
|
||||
raise ValueError("base attention profile must contain only prefill rows")
|
||||
decode_rows = [row for row in decode_source_rows if not is_true(row["is_prefill"])]
|
||||
if not decode_rows:
|
||||
raise ValueError("decode attention profile contains no decode rows")
|
||||
if any(not row["time_stats.attn_decode.median"] for row in decode_rows):
|
||||
raise ValueError("decode attention profile has an empty median")
|
||||
|
||||
dimensions = {
|
||||
(
|
||||
int(row["num_tensor_parallel_workers"]),
|
||||
int(row["batch_size"]),
|
||||
int(row["kv_cache_size"]),
|
||||
row["attention_backend"],
|
||||
)
|
||||
for row in decode_rows
|
||||
}
|
||||
if len(dimensions) != len(decode_rows):
|
||||
raise ValueError("decode attention profile has duplicate coverage coordinates")
|
||||
if any(dimension[-1] != "FLASHINFER" for dimension in dimensions):
|
||||
raise ValueError("decode attention profile is not entirely FlashInfer")
|
||||
|
||||
true_mixed_csv = (
|
||||
args.true_mixed_attention_csv.resolve()
|
||||
if args.true_mixed_attention_csv is not None
|
||||
else None
|
||||
)
|
||||
true_mixed_rows: list[dict[str, str]] = []
|
||||
output_fields = list(base_fields)
|
||||
if true_mixed_csv is not None:
|
||||
if not true_mixed_csv.is_file():
|
||||
raise FileNotFoundError(true_mixed_csv)
|
||||
true_mixed_fields, true_mixed_rows = read_csv(true_mixed_csv)
|
||||
required = {
|
||||
"is_true_mixed_batch",
|
||||
"decode_batch_size",
|
||||
"decode_avg_kv_cache_size",
|
||||
"num_prefill_seqs",
|
||||
"time_stats.attn_decode.median",
|
||||
"time_stats.attn_prefill.median",
|
||||
}
|
||||
missing = required - set(true_mixed_fields)
|
||||
if missing:
|
||||
raise ValueError(f"true-mixed attention CSV lacks columns: {sorted(missing)}")
|
||||
if not true_mixed_rows or any(
|
||||
not is_true(row["is_true_mixed_batch"]) for row in true_mixed_rows
|
||||
):
|
||||
raise ValueError("true-mixed attention CSV has non-mixed rows")
|
||||
if any(
|
||||
not row["time_stats.attn_decode.median"]
|
||||
or not row["time_stats.attn_prefill.median"]
|
||||
for row in true_mixed_rows
|
||||
):
|
||||
raise ValueError("true-mixed attention profile has an empty median")
|
||||
if {
|
||||
int(row["num_tensor_parallel_workers"]) for row in true_mixed_rows
|
||||
} != {4, 8}:
|
||||
raise ValueError("true-mixed attention profile must cover TP4 and TP8")
|
||||
output_fields.extend(
|
||||
field for field in true_mixed_fields if field not in output_fields
|
||||
)
|
||||
|
||||
for row in [*base_rows, *decode_rows]:
|
||||
if "is_true_mixed_batch" in output_fields:
|
||||
row["is_true_mixed_batch"] = "False"
|
||||
|
||||
output_root.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix=f".{output_root.name}.", dir=output_root.parent
|
||||
) as temporary:
|
||||
temporary_root = Path(temporary) / output_root.name
|
||||
shutil.copytree(base_root, temporary_root)
|
||||
merged_attention = temporary_root / ATTENTION_RELATIVE_PATH
|
||||
with merged_attention.open("w", newline="") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=output_fields, lineterminator="\n")
|
||||
writer.writeheader()
|
||||
writer.writerows([*base_rows, *decode_rows, *true_mixed_rows])
|
||||
|
||||
payload = {
|
||||
"schema": "frontier-profile-closure-v1",
|
||||
"model": MODEL,
|
||||
"base_profile_root": str(base_root),
|
||||
"base_attention_sha256": sha256(base_attention),
|
||||
"decode_attention_csv": str(decode_csv),
|
||||
"decode_attention_sha256": sha256(decode_csv),
|
||||
"base_prefill_rows": len(base_rows),
|
||||
"decode_source_rows": len(decode_source_rows),
|
||||
"added_decode_rows": len(decode_rows),
|
||||
"true_mixed_attention_csv": (
|
||||
str(true_mixed_csv) if true_mixed_csv is not None else None
|
||||
),
|
||||
"true_mixed_attention_sha256": (
|
||||
sha256(true_mixed_csv) if true_mixed_csv is not None else None
|
||||
),
|
||||
"added_true_mixed_rows": len(true_mixed_rows),
|
||||
"merged_attention_rows": (
|
||||
len(base_rows) + len(decode_rows) + len(true_mixed_rows)
|
||||
),
|
||||
"decode_dimensions": [list(values) for values in sorted(dimensions)],
|
||||
"output_files_sha256": profile_hashes(temporary_root),
|
||||
}
|
||||
write_json(temporary_root / "profile_closure_manifest.json", payload)
|
||||
temporary_root.rename(output_root)
|
||||
|
||||
print(json.dumps(payload, indent=2, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
183
runs/frontier-multicase-sufficiency-v1/audit_trace_contract.py
Normal file
183
runs/frontier-multicase-sufficiency-v1/audit_trace_contract.py
Normal file
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audit Qwen235B trace token lengths and source prefix-hash identities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import platform
|
||||
import socket
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--trace", type=Path, required=True)
|
||||
parser.add_argument("--model", type=Path, required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--max-model-len", type=int, default=40960)
|
||||
parser.add_argument("--source-block-size", type=int, default=64)
|
||||
parser.add_argument("--batch-size", type=int, default=16)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
if args.max_model_len <= 0 or args.source_block_size <= 0 or args.batch_size <= 0:
|
||||
raise ValueError("length and batch-size arguments must be positive")
|
||||
|
||||
import transformers
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
rows = [json.loads(line) for line in args.trace.open() if line.strip()]
|
||||
context_exceeded = [
|
||||
row
|
||||
for row in rows
|
||||
if int(row["input_length"]) + int(row["output_length"]) > args.max_model_len
|
||||
]
|
||||
zero_output = [row for row in rows if int(row["output_length"]) == 0]
|
||||
eligible = [
|
||||
row
|
||||
for row in rows
|
||||
if int(row["input_length"]) + int(row["output_length"]) <= args.max_model_len
|
||||
and int(row["output_length"]) > 0
|
||||
]
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
|
||||
length_mismatch_count = 0
|
||||
hash_count_mismatch_count = 0
|
||||
hash_to_key: dict[str, bytes] = {}
|
||||
key_to_hash: dict[bytes, str] = {}
|
||||
hash_to_key_conflict_count = 0
|
||||
key_to_hash_conflict_count = 0
|
||||
total_tokens = 0
|
||||
full_blocks = 0
|
||||
partial_blocks = 0
|
||||
length_digest = hashlib.sha256()
|
||||
token_digest = hashlib.sha256()
|
||||
started = time.time()
|
||||
|
||||
for start in range(0, len(eligible), args.batch_size):
|
||||
batch = eligible[start : start + args.batch_size]
|
||||
encoded = tokenizer(
|
||||
[row["prompt"] for row in batch],
|
||||
add_special_tokens=False,
|
||||
padding=False,
|
||||
truncation=False,
|
||||
)["input_ids"]
|
||||
for offset, (row, token_ids) in enumerate(zip(batch, encoded, strict=True)):
|
||||
row_index = start + offset
|
||||
actual_length = len(token_ids)
|
||||
expected_length = int(row["input_length"])
|
||||
total_tokens += actual_length
|
||||
length_digest.update(f"{row_index}:{actual_length}\n".encode())
|
||||
if actual_length != expected_length:
|
||||
length_mismatch_count += 1
|
||||
|
||||
source_hashes = row["hash_ids"]
|
||||
expected_hashes = math.ceil(actual_length / args.source_block_size)
|
||||
if len(source_hashes) != expected_hashes:
|
||||
hash_count_mismatch_count += 1
|
||||
continue
|
||||
|
||||
request_token_digest = hashlib.sha256()
|
||||
parent = b"ROOT"
|
||||
for block_index, source_hash in enumerate(source_hashes):
|
||||
begin = block_index * args.source_block_size
|
||||
chunk = token_ids[begin : begin + args.source_block_size]
|
||||
token_payload = b"".join(
|
||||
int(token_id).to_bytes(4, "little", signed=False)
|
||||
for token_id in chunk
|
||||
)
|
||||
request_token_digest.update(token_payload)
|
||||
chunk_digest = hashlib.blake2b(token_payload, digest_size=16).digest()
|
||||
key_digest = hashlib.blake2b(
|
||||
parent + b"\0" + chunk_digest, digest_size=16
|
||||
).digest()
|
||||
if len(chunk) == args.source_block_size:
|
||||
full_blocks += 1
|
||||
else:
|
||||
partial_blocks += 1
|
||||
|
||||
hash_id = str(source_hash)
|
||||
previous_key = hash_to_key.setdefault(hash_id, key_digest)
|
||||
if previous_key != key_digest:
|
||||
hash_to_key_conflict_count += 1
|
||||
previous_hash = key_to_hash.setdefault(key_digest, hash_id)
|
||||
if previous_hash != hash_id:
|
||||
key_to_hash_conflict_count += 1
|
||||
parent = hash_id.encode()
|
||||
|
||||
token_digest.update(row_index.to_bytes(4, "little"))
|
||||
token_digest.update(request_token_digest.digest())
|
||||
|
||||
payload: dict[str, Any] = {
|
||||
"schema": "qwen235b-trace-contract-audit-v1",
|
||||
"status": "pass_offline_source_contract"
|
||||
if not any(
|
||||
(
|
||||
length_mismatch_count,
|
||||
hash_count_mismatch_count,
|
||||
hash_to_key_conflict_count,
|
||||
key_to_hash_conflict_count,
|
||||
)
|
||||
)
|
||||
else "fail",
|
||||
"execution": {
|
||||
"host": socket.gethostname(),
|
||||
"device": "cpu_only",
|
||||
"elapsed_seconds": round(time.time() - started, 3),
|
||||
"python_version": platform.python_version(),
|
||||
"tokenizer_class": type(tokenizer).__name__,
|
||||
"transformers_version": transformers.__version__,
|
||||
"model_path": str(args.model.resolve()),
|
||||
},
|
||||
"trace": {
|
||||
"path": str(args.trace.resolve()),
|
||||
"sha256": sha256_file(args.trace),
|
||||
"source_request_count": len(rows),
|
||||
"context_exceeded_count": len(context_exceeded),
|
||||
"zero_output_count": len(zero_output),
|
||||
"exclusion_overlap_count": sum(row in zero_output for row in context_exceeded),
|
||||
"eligible_request_count": len(eligible),
|
||||
},
|
||||
"tokenization": {
|
||||
"total_token_count": total_tokens,
|
||||
"input_length_mismatch_count": length_mismatch_count,
|
||||
"length_order_sha256": length_digest.hexdigest(),
|
||||
"per_request_token_digest_sha256": token_digest.hexdigest(),
|
||||
},
|
||||
"source_hash_contract": {
|
||||
"source_block_size_tokens": args.source_block_size,
|
||||
"hash_count_mismatch_count": hash_count_mismatch_count,
|
||||
"full_block_count": full_blocks,
|
||||
"partial_block_count": partial_blocks,
|
||||
"unique_hash_id_count": len(hash_to_key),
|
||||
"unique_parent_chunk_key_count": len(key_to_hash),
|
||||
"hash_id_to_parent_chunk_conflict_count": hash_to_key_conflict_count,
|
||||
"parent_chunk_to_hash_id_conflict_count": key_to_hash_conflict_count,
|
||||
"key_definition": (
|
||||
"(parent source hash id, BLAKE2b-128 of the tokenizer token-id chunk)"
|
||||
),
|
||||
},
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
if payload["status"] != "pass_offline_source_contract":
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
24
runs/frontier-multicase-sufficiency-v1/fleet.toml
Normal file
24
runs/frontier-multicase-sufficiency-v1/fleet.toml
Normal file
@@ -0,0 +1,24 @@
|
||||
version = 1
|
||||
|
||||
[paths]
|
||||
state_dir = "runs/frontier-multicase-sufficiency-v1/fleet-state"
|
||||
artifacts_dir = "runs/frontier-multicase-sufficiency-v1/fleet-artifacts"
|
||||
|
||||
[ssh]
|
||||
connect_timeout_sec = 10
|
||||
|
||||
[scheduler]
|
||||
gpu_free_memory_mb = 1024
|
||||
gpu_free_utilization_pct = 10
|
||||
prefer_pack = true
|
||||
|
||||
[sync]
|
||||
mode = "scp"
|
||||
local_path = "runs/frontier-multicase-sufficiency-v1"
|
||||
|
||||
[[hosts]]
|
||||
name = "dash0"
|
||||
ssh_alias = "dash0"
|
||||
enabled = true
|
||||
sync_remote_path = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1"
|
||||
fleet_root = "/home/admin/cpfs/wjh/aituner/gpu-fleet-fidelity-v1"
|
||||
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Render the fleet queue for a frozen T0 real-surface plan."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def quoted(value: object) -> str:
|
||||
return json.dumps(str(value))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--plan", type=Path, required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--config", action="append", dest="configs")
|
||||
parser.add_argument("--rates", nargs="+", type=float)
|
||||
parser.add_argument("--artifact-root", default="artifacts/t0-real-surface-v1")
|
||||
parser.add_argument("--name-suffix", default="")
|
||||
args = parser.parse_args()
|
||||
plan = json.loads(args.plan.read_text())
|
||||
if plan.get("schema") != "qwen235b-t0-real-plan-v1" or len(plan.get("cells") or []) != 8:
|
||||
raise ValueError("invalid or incomplete T0 real plan")
|
||||
lattice = {0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20}
|
||||
if args.rates and any(rate not in lattice for rate in args.rates):
|
||||
raise ValueError("rate override must stay on the frozen T0 lattice")
|
||||
|
||||
indexed_cells = list(enumerate(plan["cells"]))
|
||||
if args.configs:
|
||||
requested = set(args.configs)
|
||||
known = {cell["config"]["name"] for _, cell in indexed_cells}
|
||||
if not requested <= known:
|
||||
raise ValueError(f"unknown configs: {sorted(requested - known)}")
|
||||
indexed_cells = [
|
||||
(index, cell)
|
||||
for index, cell in indexed_cells
|
||||
if cell["config"]["name"] in requested
|
||||
]
|
||||
|
||||
lines = [
|
||||
"# Generated from the frozen T0 real plan; do not edit rates in place.",
|
||||
"version = 1",
|
||||
"",
|
||||
]
|
||||
for index, cell in indexed_cells:
|
||||
config = cell["config"]
|
||||
suffix = f"-{args.name_suffix}" if args.name_suffix else ""
|
||||
name = f"qwen235b-t0-real-{config['name']}{suffix}-20260716-v1"
|
||||
artifact = f"{args.artifact_root.rstrip('/')}/{config['name']}"
|
||||
rates = args.rates or cell["rates"]
|
||||
lines.extend(
|
||||
[
|
||||
"[[jobs]]",
|
||||
f"name = {quoted(name)}",
|
||||
f"gpus = {int(config['tp'])}",
|
||||
'gpu_model = "H20"',
|
||||
'hosts = ["dash0"]',
|
||||
'command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"',
|
||||
f"artifacts = [{quoted(artifact)}]",
|
||||
"",
|
||||
"[jobs.env]",
|
||||
f"OUTPUT_ROOT = {quoted('/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/' + artifact)}",
|
||||
f"TP = {quoted(config['tp'])}",
|
||||
f"MNS = {quoted(config['mns'])}",
|
||||
f"MBT = {quoted(config['mbt'])}",
|
||||
f"RATES = {quoted(' '.join(f'{rate:.2f}' for rate in rates))}",
|
||||
f"SERVER_PORT = {quoted(18920 + index)}",
|
||||
'VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"',
|
||||
'MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"',
|
||||
"",
|
||||
]
|
||||
)
|
||||
args.output.write_text("\n".join(lines))
|
||||
print(args.output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
43
runs/frontier-multicase-sufficiency-v1/jobs.toml
Normal file
43
runs/frontier-multicase-sufficiency-v1/jobs.toml
Normal file
@@ -0,0 +1,43 @@
|
||||
# Append-only queue for the simulator-fidelity v1 campaign.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-tp4-smoke-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 1800 bash run_t0_smoke.sh"
|
||||
artifacts = ["artifacts/t0-smoke-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-smoke-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6"
|
||||
SERVER_PORT = "18910"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-decode-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_decode_attention_profile.sh"
|
||||
artifacts = ["artifacts/decode-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/decode-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-full-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_t0_full_attention_profile.sh"
|
||||
artifacts = ["artifacts/t0-full-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-full-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
@@ -0,0 +1,16 @@
|
||||
# One-shot queue view for the next authorized profile job. The canonical
|
||||
# append-only campaign queue remains jobs.toml.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-decode-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_decode_attention_profile.sh"
|
||||
artifacts = ["artifacts/decode-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/decode-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
@@ -0,0 +1,37 @@
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-anchor-isolation-smoke-tp4-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 1800 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "1.60"
|
||||
SERVER_PORT = "18930"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-anchor-isolation-smoke-tp4-mbt16k-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 1800 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-anchor-isolation-smoke-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "1.60"
|
||||
SERVER_PORT = "18931"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,15 @@
|
||||
# One-shot queue view for the T0 full-coverage attention profile.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-full-attention-profile-20260716-v1"
|
||||
gpus = 1
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=30s 900 bash run_t0_full_attention_profile.sh"
|
||||
artifacts = ["artifacts/t0-full-attention-profile-20260716"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-full-attention-profile-20260716"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
FRONTIER_ROOT = "/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18922"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18923"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18920"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18921"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18926"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18925"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt8192-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18922"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt16384-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns128_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18923"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,38 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt8192-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18920"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt16384-expansion-r3p20-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18921"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-expansion-r3p20-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt8192-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns128_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18926"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt16384-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18925"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r2p40-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r2p40-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "2.40"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
@@ -0,0 +1,20 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-expansion-r3p20-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-expansion-r3p20-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-expansion-r3p20-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "3.20"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
146
runs/frontier-multicase-sufficiency-v1/jobs_t0_real_surface.toml
Normal file
146
runs/frontier-multicase-sufficiency-v1/jobs_t0_real_surface.toml
Normal file
@@ -0,0 +1,146 @@
|
||||
# Generated from the frozen T0 real plan; do not edit rates in place.
|
||||
version = 1
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18920"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns64_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns64_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18921"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt8192-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt8192"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18922"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp4_mns128_mbt16384-20260716-v1"
|
||||
gpus = 4
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp4_mns128_mbt16384"
|
||||
TP = "4"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.60 2.40"
|
||||
SERVER_PORT = "18923"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18924"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns64_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns64_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "64"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18925"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt8192-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt8192"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "8192"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18926"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
|
||||
[[jobs]]
|
||||
name = "qwen235b-t0-real-tp8_mns128_mbt16384-20260716-v1"
|
||||
gpus = 8
|
||||
gpu_model = "H20"
|
||||
hosts = ["dash0"]
|
||||
command = "timeout --signal=TERM --kill-after=60s 7200 bash run_t0_real_config.sh"
|
||||
artifacts = ["artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"]
|
||||
|
||||
[jobs.env]
|
||||
OUTPUT_ROOT = "/home/admin/cpfs/wjh/aituner/aituner-fidelity-v1/artifacts/t0-real-surface-v1/tp8_mns128_mbt16384"
|
||||
TP = "8"
|
||||
MNS = "128"
|
||||
MBT = "16384"
|
||||
RATES = "0.10 1.20 1.60"
|
||||
SERVER_PORT = "18927"
|
||||
VENV_ROOT = "/tmp/wjh-frontier-vllm0102-smoke/.venv"
|
||||
MODEL_ROOT = "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
128
runs/frontier-multicase-sufficiency-v1/prepare_t0_real_plan.py
Normal file
128
runs/frontier-multicase-sufficiency-v1/prepare_t0_real_plan.py
Normal file
@@ -0,0 +1,128 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Select predeclared simulator-lattice anchors for blind real confirmation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
RATES = (0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20)
|
||||
CONFIG_NAMES = {
|
||||
f"tp{tp}_mns{mns}_mbt{mbt}"
|
||||
for tp in (4, 8)
|
||||
for mns in (64, 128)
|
||||
for mbt in (8192, 16384)
|
||||
}
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-freeze", type=Path, required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--selection-slo", default="tpot_150ms")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def select_rates(loads: list[dict[str, Any]], slo: str) -> list[float]:
|
||||
ordered = sorted(loads, key=lambda item: float(item["offered_request_rate"]))
|
||||
rates = [float(item["offered_request_rate"]) for item in ordered]
|
||||
feasible = [bool(item["slos"][slo]["feasible"]) for item in ordered]
|
||||
selected = {rates[0]}
|
||||
for index in range(len(rates) - 1):
|
||||
if feasible[index] != feasible[index + 1]:
|
||||
selected.update((rates[index], rates[index + 1]))
|
||||
if len(selected) == 1:
|
||||
if all(feasible):
|
||||
selected.update(rates[-2:])
|
||||
elif not any(feasible):
|
||||
selected.update(rates[:2])
|
||||
return sorted(selected)
|
||||
|
||||
|
||||
def warmup_requests(rate: float) -> int:
|
||||
return min(32, max(4, math.ceil(rate * 20.0)))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
freeze_path = args.frontier_freeze.resolve()
|
||||
freeze = json.loads(freeze_path.read_text())
|
||||
if freeze.get("schema") != "frontier-qwen235b-t0-surface-v1":
|
||||
raise ValueError("unexpected Frontier freeze schema")
|
||||
if freeze.get("status") != "frozen_before_real_surface":
|
||||
raise ValueError("Frontier surface is not frozen")
|
||||
results = freeze.get("config_results") or []
|
||||
if len(results) != 8 or any(len(item.get("loads") or []) != 8 for item in results):
|
||||
raise ValueError("Frontier surface is incomplete")
|
||||
names = {item.get("config", {}).get("name") for item in results}
|
||||
if names != CONFIG_NAMES:
|
||||
raise ValueError(f"Frontier config set mismatch: {names}")
|
||||
for item in results:
|
||||
rates = tuple(sorted(float(load["offered_request_rate"]) for load in item["loads"]))
|
||||
if rates != RATES:
|
||||
raise ValueError(f"Frontier rate lattice mismatch for {item['config']['name']}: {rates}")
|
||||
|
||||
cells = []
|
||||
total_expected_seconds = 0.0
|
||||
for item in results:
|
||||
config = item["config"]
|
||||
rates = select_rates(item["loads"], args.selection_slo)
|
||||
# Every anchor gets an independent server in both rounds. The estimate
|
||||
# includes server startup, target-rate warmup and conservative drain
|
||||
# allowances for both the discarded and measured request streams.
|
||||
expected_seconds = 2 * sum(
|
||||
120.0
|
||||
+ (warmup_requests(rate) - 1) / rate
|
||||
+ 60.0
|
||||
+ 63.0 / rate
|
||||
+ 60.0
|
||||
for rate in rates
|
||||
)
|
||||
total_expected_seconds += expected_seconds * int(config["tp"])
|
||||
cells.append(
|
||||
{
|
||||
"config": config,
|
||||
"rates": rates,
|
||||
"rounds": 2,
|
||||
"requests_per_anchor": 64,
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"target_rate_warmup_requests": {
|
||||
f"{rate:.2f}": warmup_requests(rate) for rate in rates
|
||||
},
|
||||
"expected_wall_seconds": expected_seconds,
|
||||
"expected_h20_gpu_hours": expected_seconds * int(config["tp"]) / 3600.0,
|
||||
"expansion_rule": "if real labels do not bracket a transition, expand to the next frozen lattice anchor and repeat both directions",
|
||||
}
|
||||
)
|
||||
payload = {
|
||||
"schema": "qwen235b-t0-real-plan-v1",
|
||||
"frontier_freeze": {"path": str(freeze_path), "sha256": sha256(freeze_path)},
|
||||
"selection_slo": args.selection_slo,
|
||||
"selection_timing": "rate anchors frozen after complete simulator surface and before any accepted real surface cell",
|
||||
"execution_protocol_amendment": {
|
||||
"timing": "after excluded multi-rate-per-server diagnostic and before any accepted real surface cell",
|
||||
"reason": "observed cross-anchor GPU/kernel/batch warm-state leakage",
|
||||
"contract": "fresh server and target-rate warmup for every config-rate-round",
|
||||
},
|
||||
"strict_preregistered_slo": "tpot_40ms",
|
||||
"post_pilot_sensitivities": ["tpot_120ms", "tpot_150ms", "tpot_180ms"],
|
||||
"cells": cells,
|
||||
"expected_total_h20_gpu_hours": total_expected_seconds / 3600.0,
|
||||
"hard_timeout_hours_per_cell": 2.0,
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
print(json.dumps({"cells": len(cells), "expected_total_h20_gpu_hours": payload["expected_total_h20_gpu_hours"]}, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
123
runs/frontier-multicase-sufficiency-v1/protocol.md
Normal file
123
runs/frontier-multicase-sufficiency-v1/protocol.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Qwen235B simulator fidelity 预注册协议 v1
|
||||
|
||||
状态：`IN PROGRESS`。更新日期：2026-07-16。T0 的 simulator 与 real 8-config surface 已完成；T1 和 T2 尚未运行。pilot 单独记录在 [t0-smoke-report.md](t0-smoke-report.md)，最终 T0 rank evaluation 记录在 [comparison.json](results/t0-final/comparison.json) 和根目录 [simulator-fidelity.md](../../simulator-fidelity.md)。
|
||||
|
||||
## Research question 与成功标准
|
||||
|
||||
对同一 workload、SLO 和候选 config surface，Frontier 是否能找到真机上低 regret 的 config，而不要求绝对 latency 或 capacity 完全一致？
|
||||
|
||||
主目标统一为：
|
||||
|
||||
```text
|
||||
capacity(c) = max { 被真实测试的 offered req/s | 至少 95% requests 满足全部 SLO }
|
||||
score(c) = capacity(c) / config 实际占用的 GPU 数
|
||||
```
|
||||
|
||||
每个 case 都报告 real/sim capacity、top set、worst tie-break regret、Kendall tau-b、informative-pair direction、anchor-level SLO confusion，以及为达到该 fidelity 使用的 profile、patch 和真机 calibration 成本。只有同时满足下列条件，才能说 Frontier 对该 compatibility envelope 是足够的 config ranker：
|
||||
|
||||
- worst selected-config regret 不超过 5%；
|
||||
- tie-aware rank correlation 不低于 0.8，且有足够的 non-tied pairs；
|
||||
- ground-truth capacity bracket 不足以反转最优决策；
|
||||
- 不使用被评测 workload/config 的端到端测量做 per-action calibration。
|
||||
|
||||
## 冻结的平台与软件边界
|
||||
|
||||
| 项目 | 设置 |
|
||||
|---|---|
|
||||
| machine | 仅 `dash0`，8×NVIDIA H20；不得调度到 `dash1` |
|
||||
| remote repo | `/home/admin/cpfs/wjh/aituner/aituner`；实验使用独立 clean worktree/clone，不修改当前 dirty checkout |
|
||||
| model | `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8` |
|
||||
| serving | community vLLM `0.10.2` isolated environment，eager，BF16 KV；包版本与二进制 hash 入 manifest |
|
||||
| simulator | Frontier upstream commit `d9cfeb6d8791fbf2f295dd9744c56a666171776e` + 明示、hash 后的 FP8/MoE/trace patches |
|
||||
| trace | `thinking_w20260327_1000.jsonl`，SHA256 `f878e9af18f94dcfaced94a8e1e6b20a2f7d97d64aa862448025660dbbd965b2` |
|
||||
| source window | 600 s，15,479 requests，natural offered rate 25.798 req/s |
|
||||
|
||||
每个 real trial 使用 fresh server。第一轮 config 顺序由固定 seed 随机化，第二轮反序；同一 config 的 offered-load anchors 从低到高和从高到低各跑一次。任何 OOM、engine crash、timeout 和右删失 cell 都作为结果保留，不能静默丢弃。
|
||||
|
||||
## Trace fidelity contract
|
||||
|
||||
source row 同时包含 exact prompt、arrival timestamp、`input_length`、`output_length`、session/turn/parent、`sampling_u`，以及 block-size=64 的 `hash_ids`。主 trace case 不设置 output override，也不按 input/output length 选择“好跑”的 cohort。
|
||||
|
||||
community model 的最大 context 是 40,960 tokens。预先冻结的 eligible universe 如下：
|
||||
|
||||
| universe | requests | input mean/p50/p95 | output mean/p50/p95 | total mean/p50/p95 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| source | 15,479 | 3,660.0 / 1,491 / 19,610.6 | 3,924.6 / 3,435 / 8,945.1 | 7,584.6 / 5,317 / 25,102.1 |
|
||||
| eligible | 15,401 | 3,575.0 / 1,490 / 18,887 | 3,823.3 / 3,417 / 8,768 | 7,398.2 / 5,305 / 23,697 |
|
||||
|
||||
唯一排除项是 72 个 `input_length + output_length > 40960` rows，以及 6 个 `output_length = 0` rows；二者无重叠。eligible 比例为 99.50%。
|
||||
|
||||
capacity search 使用 `sampling_u <= u` 调整 offered rate。`sampling_u` 在同一 session 内共享，因此这是与长度无关的 session-coherent thinning，而不是 workload filtering。对每个 anchor，real 与 sim 必须具有相同的 source row IDs、arrival/order hash、input/output vector hash、session vector hash 和 prefix vector hash。每个被选 request 仍使用原始 arrival、prompt、input、output 和 hash。
|
||||
|
||||
第一个 correctness gate 已完成：在 `dash0` 上使用 community model 的 `Qwen2TokenizerFast` 全量处理 15,401 个 prompts、55,057,919 tokens，实际 token length 与 trace `input_length` 的 mismatch 为 0；在 852,407 个完整和 15,131 个末尾 partial source blocks 上，`hash_id ↔ (parent_hash_id, 64-token chunk)` 均无冲突。结果和 digest 记录在 [trace-contract-audit.json](trace-contract-audit.json)。这说明 exact prompt/source hash 可以表达原 trace 的 prefix-equivalence relation。
|
||||
|
||||
在启动 GPU 前还需完成第二个 correctness gate：
|
||||
|
||||
1. real 与 Frontier 的 KV block size 都冻结为 16。source `hash_ids` 的 block size 是 64，因此应从 exact prompt token IDs 生成 block-16 content/parent identities，并验证每四个完整 block-16 对应的 64-token source equality/reuse relation 没有改变。两侧逐 request 比较 computed/hit/allocated block counts。
|
||||
|
||||
## T0：fixed-shape sanity case
|
||||
|
||||
T0 是最简单的机制隔离基线，不声称代表 production trace。
|
||||
|
||||
| 项目 | 设置 |
|
||||
|---|---|
|
||||
| requests | deterministic 生成；ISL=2,048、OSL=128 |
|
||||
| arrivals | uniform QPS；相同 request IDs/order；每个 anchor 至少 64 个 completed requests |
|
||||
| prefix | off；不同 token content，保证没有共享完整 KV blocks |
|
||||
| SLO | TTFT `<= 1000 ms + 1000×input_tokens/8000`（本 case 为 1,256 ms），TPOT `<= 40 ms`，joint pass rate ≥0.95 |
|
||||
| configs | `TP∈{4,8} × MNS∈{64,128} × MBT∈{8192,16384}`；沿用已闭合的 TP4/TP8 MoE mapping |
|
||||
|
||||
先在 simulator 中用宽 anchor lattice 找到每个 config 的边界，再冻结 simulator output 和 hash；真机只运行相同 anchors。T0 回答的是：在没有 length variance、prefix reuse 和 burstiness 时，profile composition 与 scheduler 是否已经能保持 config rank。它不能用于回答真实 trace fidelity。
|
||||
|
||||
首次真机 smoke 之后、完整 surface 之前预先记录如下 amendment：TP4 的 zero-queue TPOT 已经是 136.2 ms，因此 40 ms 主 SLO 对至少 TP4 没有非零 capacity。40 ms 仍保留为 preregistered primary，不因结果不可行而删除；同时固定报告 TPOT 120/150/180 ms sensitivity。150 ms 是 post-pilot 的 decision-bearing sensitivity，因为它略高于已观测 TP4 idle floor，又仍可能在 batch/queue 增大时产生 capacity knee；它不能冒充 blind primary result。
|
||||
|
||||
完整 simulator lattice 固定为 system offered rate `{0.10,0.20,0.40,0.80,1.20,1.60,2.40,3.20}` req/s，每个点 64 requests。所有 8×8 simulator cells 必须在任何 real surface cell 之前冻结。真机对每个 config 先测 simulator 在 150 ms sensitivity 下的相邻 transition anchors，并始终加入最低 0.10 req/s anchor；若没有 transition，则测边界方向的两个 anchors。若这些 real labels 没有 bracket，按冻结 lattice 向外扩展，且 simulator 结果不得重算。每个被测 anchor 做两个 fresh-server rounds，rate 顺序正向/反向，所有 strict/sensitivity SLO 均在同一 raw requests 上离线打分。真机 anchor 只有在两个 round 都达到 95% joint pass rate 时才保守地记为 feasible；两个 round 标签不一致时同时报告原始 pass rates，并把该点记为重复性不确定而不是择优取样。
|
||||
|
||||
第一次多-rate 真机 run 后、接受任何 real surface cell 之前增加一项 execution amendment。原 runner 每个 round 只启动一次 server，正向 `0.10→1.60→2.40` 时 1.60 的前 24 个 requests 出现 114--234 ms 的递减 TPOT transient；反向先运行 2.40 后，同一 1.60 anchor 从第一个 request 起稳定在 113--128 ms。两个独立 TP4 config 同时复现，说明前一 rate 的 GPU/kernel/batch warm state 泄漏到后一 rate，而 Frontier 的各 rate 是独立 simulator run。该诊断 run 整体排除，不作为 ground truth。修正后的每个 `(config, rate, round)` 使用独立 fresh server，并在测量前按同一 target rate 丢弃 `min(32, max(4, ceil(rate×20)))` 个 exact-shape warmup requests；随后从空 scheduler queue 开始记录 64 requests。正/逆序只用于复测和控制机器时间漂移，不再共享 execution state。这个 amendment 是观察 diagnostic transient 后作出的，必须在结果中明示，不能写成原始 blind protocol。
|
||||
|
||||
steady-QPS profile closure 不能只含 pure decode。完整 profile 预先覆盖 TP `{4,8}`、decode batch `{1,2,4,8,16,32,64,96,128}`、KV 2,048--2,175，并加入 prefill-chunk=2,048 的 true mixed prefill+decode rows。最终 attention root 含 726 个既有 prefill rows、162 个 standard decode rows 和 216 个 true-mixed rows；没有使用 T0 端到端 latency 做 calibration。
|
||||
|
||||
## T1：trace-faithful mixed case
|
||||
|
||||
T1 直接使用上述 15,401-row eligible universe：不设置 completion override，不改变入选 request 的 input/output，不做 length-stratified sampling，prefix caching 在 real 与 sim 两侧同时打开。
|
||||
|
||||
| 项目 | 设置 |
|
||||
|---|---|
|
||||
| selection | source `sampling_u <= u`；同 session 一起入选；real/sim 共用 frozen anchor files |
|
||||
| arrival | 原 timestamp、600 s window、原 burst/order；`replay_time_scale=1.0` |
|
||||
| output | `min_tokens=max_tokens=output_length`；记录实际 usage，必须与 trace 相等 |
|
||||
| prefix | exact prompt token blocks；两侧相同 block size、capacity、cache policy 与 sticky session routing |
|
||||
| SLO | 与 T0 相同的 TTFT 规则和 TPOT 40 ms；另报告 TPOT 20/50 ms sensitivity，不用 sensitivity 改选主结论 |
|
||||
| configs | 与 T0 相同的 8 cells，先隔离 workload state，再扩展 DP/EP topology |
|
||||
|
||||
由于 natural QPS 远高于该模型 capacity，直接全量同时到达只会让所有 config 都 infeasible，不能产生排序信息。这里 `sampling_u` 是 trace 原生的负载抽样维度；它只改变入选 session 数量，不改变入选请求的 joint distribution 和字段。每个边界至少用两个预先冻结的 session-hash folds 或两个独立 trace windows 重复，避免一个低-u 小样本偶然决定 ranking；若暂时只有一个 window，结论标为 single-window evidence。
|
||||
|
||||
## T2：Qwen235B strict decode-only case
|
||||
|
||||
T2 不用“短 prefill + 长 output”冒充 decode-only。它要求一个可检查的 initial-KV contract：request 到达 scheduler 时，与 input token IDs 对应的 KV blocks 已经 resident，request 状态为 prefill complete，首次被调度的 token 是第一个 decode token。
|
||||
|
||||
两侧 contract 必须共同记录：initial KV token/block count、block identities、replica placement、填充/传输开始结束时间、首次 scheduler admission time。若真机 connector 在 GPU 上同步填充 dummy KV，这项带宽和同步成本会干扰并发 decode；Frontier 必须显式建模这段 admission cost，或真机把它移到计时区间之前。不能只忽略 TTFT，就假设 connector 没有影响 TPOT。
|
||||
|
||||
T2 分两步，避免把 execution mechanisms 混成一个不可解释的 gap：
|
||||
|
||||
| 层次 | Workload / mechanisms | Config surface | 目的 |
|
||||
|---|---|---|---|
|
||||
| T2a controlled decode | fixed ISL=2,048、OSL=512；BF16 KV；eager；prefix/speculation/CUDA graph off | `TP∈{4,8} × MNS∈{64,128} × MBT∈{256,384}`，DP=1 | 验证 initial-KV、decode attention/batching 与 TP rank |
|
||||
| T2b topology stress | trace exact input/output；EP8；仍先关闭 EAGLE3 与 decode graph | `(TP4,DP2,EP8)` 与 `(TP2,DP4,EP8)` × MNS `{64,128}` × MBT `{256,384}` | 激活 replica count、expert communication、KV residency 的耦合 |
|
||||
|
||||
只有 T2a/T2b 对齐后，才按单变量顺序加入 FP8 KV、EAGLE3、DeepEP/NVSHMEM 和 `FULL_DECODE_ONLY` CUDA graph；每加入一项都重新冻结 simulator ranking 并测真机。现有 internal-vLLM decode surface 同时打开了这些机制，而且 8 个 config 的 capacity brackets 全部可能包含最优值，只作为 historical real-only evidence，不作为本协议 ground truth。
|
||||
|
||||
## Run order 与停止条件
|
||||
|
||||
1. 完成 tokenizer、row vector、prefix-block correctness audit；不使用 GPU。tokenizer/source-hash 离线部分已完成，block-16 runtime-counter parity 待完成。
|
||||
2. 在 dash0 做一次 community vLLM TP4 model-load + T0 one-request smoke。已完成：server 与 exact 2,048/128-token requests 成功，但最低负载 TPOT 为 136.2 ms，40 ms primary SLO 不可行；对应 Frontier run 暴露 decode attention profile 未闭合。
|
||||
3. 完成 T0 的 simulator 8-cell surface并冻结 SHA256；随后运行 T0 real anchors。已完成：64/64 simulator cells 冻结，8/8 real config boundaries 在 150 ms decision-bearing SLO 下闭合。
|
||||
4. 实现并单测 trace block translation，完成 T1 simulator freeze，再运行 T1 real anchors。
|
||||
5. 实现 community-vLLM 0.10.2 与 Frontier 的 initial-KV contract；一请求和并发请求状态机 smoke 通过后，运行 T2a。
|
||||
6. 只有 profile closure 包含 EP8 all-to-all 且 T2a 可解释时，运行 T2b。
|
||||
|
||||
每层遇到 mismatch，先按 request/stage 分解 queue wait、prefill、decode step、collective、MoE 和 KV admission residual。只有证据定位到某一 composition invariant 失效，才修改 simulator；不通过端到端 scalar 把 ranking 调到正确。
|
||||
|
||||
## 首次 GPU launch gate
|
||||
|
||||
首次 GPU smoke、full attention closure 与完整 T0 sweep 均已于 2026-07-16 在 `dash0` 完成，GPU 已释放。接受的 T0 ground truth 为 68 个 fresh-server anchors，消耗 36.26 H20-GPU-hours；污染和 cross-anchor-state diagnostic attempts 已隔离。任何 T1/T2 GPU launch 都必须重新 echo resolved workload、artifact paths、GPU 预算与预计时长。
|
||||
@@ -0,0 +1,33 @@
|
||||
slo,config,tp,mns,mbt,real_capacity_per_gpu,sim_capacity_per_gpu,real_boundary_status,expansion_required
|
||||
tpot_40ms,tp4_mns64_mbt8192,4,64,8192,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp4_mns64_mbt16384,4,64,16384,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp4_mns128_mbt8192,4,128,8192,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp4_mns128_mbt16384,4,128,16384,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp8_mns64_mbt8192,8,64,8192,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp8_mns64_mbt16384,8,64,16384,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp8_mns128_mbt8192,8,128,8192,,,lowest_anchor_infeasible,False
|
||||
tpot_40ms,tp8_mns128_mbt16384,8,128,16384,,,lowest_anchor_infeasible,False
|
||||
tpot_120ms,tp4_mns64_mbt8192,4,64,8192,0.025,0.2,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp4_mns64_mbt16384,4,64,16384,0.025,0.2,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp4_mns128_mbt8192,4,128,8192,0.025,0.2,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp4_mns128_mbt16384,4,128,16384,0.025,0.2,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp8_mns64_mbt8192,8,64,8192,0.0125,0.05,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp8_mns64_mbt16384,8,64,16384,0.0125,0.05,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp8_mns128_mbt8192,8,128,8192,0.0125,0.05,unbracketed_requires_expansion,True
|
||||
tpot_120ms,tp8_mns128_mbt16384,8,128,16384,0.0125,0.05,unbracketed_requires_expansion,True
|
||||
tpot_150ms,tp4_mns64_mbt8192,4,64,8192,0.6,0.4,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp4_mns64_mbt16384,4,64,16384,0.6,0.4,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp4_mns128_mbt8192,4,128,8192,0.6,0.4,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp4_mns128_mbt16384,4,128,16384,0.6,0.4,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp8_mns64_mbt8192,8,64,8192,0.3,0.15,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp8_mns64_mbt16384,8,64,16384,0.2,0.15,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp8_mns128_mbt8192,8,128,8192,0.2,0.15,adjacent_transition_bracketed,False
|
||||
tpot_150ms,tp8_mns128_mbt16384,8,128,16384,0.3,0.15,adjacent_transition_bracketed,False
|
||||
tpot_180ms,tp4_mns64_mbt8192,4,64,8192,0.8,0.6,upper_lattice_reached,False
|
||||
tpot_180ms,tp4_mns64_mbt16384,4,64,16384,0.8,0.6,upper_lattice_reached,False
|
||||
tpot_180ms,tp4_mns128_mbt8192,4,128,8192,0.8,0.6,upper_lattice_reached,False
|
||||
tpot_180ms,tp4_mns128_mbt16384,4,128,16384,0.8,0.6,upper_lattice_reached,False
|
||||
tpot_180ms,tp8_mns64_mbt8192,8,64,8192,0.4,0.2,upper_lattice_reached,False
|
||||
tpot_180ms,tp8_mns64_mbt16384,8,64,16384,0.3,0.2,unbracketed_requires_expansion,True
|
||||
tpot_180ms,tp8_mns128_mbt8192,8,128,8192,0.3,0.2,unbracketed_requires_expansion,True
|
||||
tpot_180ms,tp8_mns128_mbt16384,8,128,16384,0.4,0.2,upper_lattice_reached,False
|
||||
|
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-$(pwd)/artifacts/decode-attention-profile-20260716}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
PROFILE_ROOT="${OUTPUT_ROOT}/profiles"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
PROVENANCE_DIR="${OUTPUT_ROOT}/provenance"
|
||||
MODEL="Qwen3-235B-A22B-FP8"
|
||||
|
||||
mkdir -p "${PROFILE_ROOT}" "${LOG_DIR}" "${PROVENANCE_DIR}"
|
||||
exec > >(tee -a "${LOG_DIR}/profile.log") 2>&1
|
||||
|
||||
if [[ -z "${CUDA_VISIBLE_DEVICES:-}" ]]; then
|
||||
echo "ERROR: CUDA_VISIBLE_DEVICES must contain the fleet-allocated GPU" >&2
|
||||
exit 1
|
||||
fi
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne 1 ]]; then
|
||||
echo "ERROR: expected exactly one GPU, got ${CUDA_VISIBLE_DEVICES}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "PROFILE_LAUNCH_ECHO host=$(hostname) gpu=${CUDA_VISIBLE_DEVICES} model=${MODEL} operator=FlashInfer_attention phase=decode TP_workers=4 batch_sizes=1,2 kv_sizes=2048,2176 block=16 measurement=CUDA_EVENT output=${OUTPUT_ROOT} expected_wall=5-10m hard_wall=900s hard_gpu_cap=0.25_H20h"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
|
||||
test -x "${VENV_ROOT}/bin/python"
|
||||
test -f "${FRONTIER_ROOT}/pyproject.toml"
|
||||
test -f "${FRONTIER_ROOT}/data/config/models/${MODEL}.json"
|
||||
sha256sum run_decode_attention_profile.sh > "${PROVENANCE_DIR}/source.sha256"
|
||||
|
||||
export PYTHONPATH="${FRONTIER_ROOT}"
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
|
||||
cd "${FRONTIER_ROOT}"
|
||||
timeout --signal=TERM --kill-after=30s 600 \
|
||||
"${VENV_ROOT}/bin/python" -m frontier.profiling.attention.main \
|
||||
--disable_ray \
|
||||
--models "${MODEL}" \
|
||||
--num_gpus 1 \
|
||||
--max_model_len 40960 \
|
||||
--max_seq_len 2176 \
|
||||
--min_batch_size 1 \
|
||||
--max_batch_size 2 \
|
||||
--batch_size_list 1 2 \
|
||||
--decode_kv_cache_size_list 2048 2176 \
|
||||
--num_tensor_parallel_workers 4 \
|
||||
--max_pipeline_parallel_size 1 \
|
||||
--attention_backend FLASHINFER \
|
||||
--block_size 16 \
|
||||
--profile_only_decode \
|
||||
--device h20 \
|
||||
--profile_method cuda_event \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--yes
|
||||
|
||||
ATTENTION_CSV="${PROFILE_ROOT}/compute/h20/${MODEL}/attention.csv"
|
||||
test -s "${ATTENTION_CSV}"
|
||||
"${VENV_ROOT}/bin/python" - "${ATTENTION_CSV}" \
|
||||
> "${PROVENANCE_DIR}/coverage.json" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
path = sys.argv[1]
|
||||
frame = pd.read_csv(path)
|
||||
decode = frame[frame["is_prefill"] == False] # noqa: E712
|
||||
payload = {
|
||||
"path": path,
|
||||
"row_count": len(frame),
|
||||
"decode_row_count": len(decode),
|
||||
"batch_sizes": sorted(int(value) for value in decode["batch_size"].unique()),
|
||||
"kv_cache_sizes": sorted(int(value) for value in decode["kv_cache_size"].unique()),
|
||||
"attn_decode_median_non_null": int(
|
||||
decode["time_stats.attn_decode.median"].notna().sum()
|
||||
),
|
||||
}
|
||||
print(json.dumps(payload, indent=2, sort_keys=True))
|
||||
if payload["decode_row_count"] < 4 or payload["attn_decode_median_non_null"] < 4:
|
||||
raise SystemExit(1)
|
||||
PY
|
||||
|
||||
sha256sum \
|
||||
"${ATTENTION_CSV}" \
|
||||
"${PROVENANCE_DIR}/coverage.json" \
|
||||
"${PROVENANCE_DIR}/source.sha256" \
|
||||
> "${PROVENANCE_DIR}/artifacts.sha256"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "DECODE_ATTENTION_PROFILE_COMPLETE"
|
||||
298
runs/frontier-multicase-sufficiency-v1/run_frontier_t0_smoke.py
Normal file
298
runs/frontier-multicase-sufficiency-v1/run_frontier_t0_smoke.py
Normal file
@@ -0,0 +1,298 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run Frontier on the exact single/concurrency-2 T0 smoke workload."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODEL = "Qwen3-235B-A22B-FP8"
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-source", type=Path, required=True)
|
||||
parser.add_argument("--profile-root", type=Path, required=True)
|
||||
parser.add_argument("--python", type=Path, default=Path("/usr/bin/python3.12"))
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
|
||||
|
||||
def write_trace(path: Path, request_count: int) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", newline="") as output:
|
||||
writer = csv.DictWriter(
|
||||
output,
|
||||
fieldnames=[
|
||||
"arrived_at",
|
||||
"num_prefill_tokens",
|
||||
"num_decode_tokens",
|
||||
"slo_ttft_ms",
|
||||
],
|
||||
)
|
||||
writer.writeheader()
|
||||
for _ in range(request_count):
|
||||
writer.writerow(
|
||||
{
|
||||
"arrived_at": 0.0,
|
||||
"num_prefill_tokens": 2048,
|
||||
"num_decode_tokens": 128,
|
||||
"slo_ttft_ms": 1256.0,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def build_command(
|
||||
*,
|
||||
args: argparse.Namespace,
|
||||
trace: Path,
|
||||
run_dir: Path,
|
||||
run_id: str,
|
||||
) -> list[str]:
|
||||
compute_root = args.profile_root / "compute/h20" / MODEL
|
||||
network = args.profile_root / "network/h20_nccl/all_reduce.csv"
|
||||
return [
|
||||
str(args.python),
|
||||
"-m",
|
||||
"frontier.main",
|
||||
"--simulation_mode",
|
||||
"offline",
|
||||
"--offline_use_generated_request_arrivals",
|
||||
"--sys_arch",
|
||||
"co-location",
|
||||
"--cluster_config_num_replicas",
|
||||
"1",
|
||||
"--replica_config_model_name",
|
||||
MODEL,
|
||||
"--replica_config_attn_tensor_parallel_size",
|
||||
"4",
|
||||
"--replica_config_attn_data_parallel_size",
|
||||
"1",
|
||||
"--replica_config_moe_tensor_parallel_size",
|
||||
"4",
|
||||
"--replica_config_moe_expert_parallel_size",
|
||||
"1",
|
||||
"--replica_config_total_expert_num",
|
||||
"128",
|
||||
"--replica_config_router_topk",
|
||||
"8",
|
||||
"--replica_config_moe_routing_mode",
|
||||
"simulation",
|
||||
"--replica_config_moe_routing_seed",
|
||||
"42",
|
||||
"--replica_config_num_pipeline_stages",
|
||||
"1",
|
||||
"--replica_config_device",
|
||||
"h20",
|
||||
"--replica_config_network_device",
|
||||
"h20_dgx",
|
||||
"--cc_backend_config_type",
|
||||
"vidur",
|
||||
"--vidur_cc_backend_config_profiling_data_dir",
|
||||
str(args.profile_root),
|
||||
"--vidur_cc_backend_config_cache_dir",
|
||||
str(run_dir / "cache/collectives"),
|
||||
"--vidur_cc_backend_config_all_reduce_input_file",
|
||||
str(network),
|
||||
"--replica_scheduler_config_type",
|
||||
"vllm_v1",
|
||||
"--decode_cuda_graph_mode",
|
||||
"none",
|
||||
"--vllm_v1_scheduler_config_batch_size_cap",
|
||||
"64",
|
||||
"--vllm_v1_scheduler_config_block_size",
|
||||
"16",
|
||||
"--vllm_v1_scheduler_config_num_blocks",
|
||||
"26101",
|
||||
"--vllm_v1_scheduler_config_num_blocks_mode",
|
||||
"explicit",
|
||||
"--vllm_v1_scheduler_config_max_tokens_in_batch",
|
||||
"8192",
|
||||
"--vllm_v1_scheduler_config_enable_chunked_prefill",
|
||||
"--no-vllm_v1_scheduler_config_enable_prefix_caching",
|
||||
"--request_generator_config_type",
|
||||
"trace_replay",
|
||||
"--trace_request_generator_config_trace_file",
|
||||
str(trace),
|
||||
"--trace_request_generator_config_time_scale_factor",
|
||||
"1",
|
||||
"--trace_request_generator_config_prefill_scale_factor",
|
||||
"1",
|
||||
"--trace_request_generator_config_decode_scale_factor",
|
||||
"1",
|
||||
"--trace_request_generator_config_max_tokens",
|
||||
"40960",
|
||||
"--no-random_forrest_execution_time_predictor_config_enable_dummy_mode",
|
||||
"--random_forrest_execution_time_predictor_config_linear_op_input_file",
|
||||
str(compute_root / "linear_op.csv"),
|
||||
"--random_forrest_execution_time_predictor_config_atten_input_file",
|
||||
str(compute_root / "attention.csv"),
|
||||
"--random_forrest_execution_time_predictor_config_moe_input_file",
|
||||
str(compute_root / "moe.csv"),
|
||||
"--random_forrest_execution_time_predictor_config_all_reduce_input_file",
|
||||
str(network),
|
||||
"--random_forrest_execution_time_predictor_config_prediction_max_prefill_chunk_size",
|
||||
"16384",
|
||||
"--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request",
|
||||
"40960",
|
||||
"--random_forrest_execution_time_predictor_config_prediction_max_batch_size",
|
||||
"128",
|
||||
"--random_forrest_execution_time_predictor_config_skip_cpu_overhead_modeling",
|
||||
"--metrics_config_cache_dir",
|
||||
str(run_dir / "cache/execution"),
|
||||
"--metrics_config_output_dir",
|
||||
str(run_dir / "metrics"),
|
||||
"--metrics_config_run_id",
|
||||
run_id,
|
||||
"--metrics_config_write_metrics",
|
||||
"--metrics_config_store_request_metrics",
|
||||
"--no-metrics_config_store_plots",
|
||||
"--no-metrics_config_enable_chrome_trace",
|
||||
"--no-metrics_config_write_json_trace",
|
||||
]
|
||||
|
||||
|
||||
def find_request_metrics(run_dir: Path) -> Path:
|
||||
matches = list((run_dir / "metrics").rglob("request_metrics.csv"))
|
||||
if len(matches) != 1:
|
||||
raise RuntimeError(f"expected one request_metrics.csv, found {matches}")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def score(metrics_path: Path) -> dict[str, Any]:
|
||||
with metrics_path.open(newline="") as source:
|
||||
rows = list(csv.DictReader(source))
|
||||
requests = []
|
||||
for row in rows:
|
||||
ttft_ms = float(row["ttft"])
|
||||
e2e_ms = float(row["request_e2e_time"])
|
||||
decode_tokens = int(float(row["request_num_decode_tokens"]))
|
||||
aligned_tpot_ms = (
|
||||
(e2e_ms - ttft_ms) / (decode_tokens - 1) if decode_tokens > 1 else 0.0
|
||||
)
|
||||
requests.append(
|
||||
{
|
||||
"request_id": int(row["Request Id"]),
|
||||
"prompt_tokens": int(float(row["request_num_prefill_tokens"])),
|
||||
"completion_tokens": decode_tokens,
|
||||
"ttft_ms": ttft_ms,
|
||||
"e2e_ms": e2e_ms,
|
||||
"tpot_ms_aligned": aligned_tpot_ms,
|
||||
"frontier_decode_e2e_time_per_token_ms": float(
|
||||
row["decode_e2e_time_per_token"]
|
||||
),
|
||||
"joint_slo_pass": ttft_ms <= 1256.0 and aligned_tpot_ms <= 40.0,
|
||||
}
|
||||
)
|
||||
return {
|
||||
"request_count": len(requests),
|
||||
"joint_slo_pass_count": sum(row["joint_slo_pass"] for row in requests),
|
||||
"requests": requests,
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
source = args.frontier_source.resolve()
|
||||
profiles = args.profile_root.resolve()
|
||||
output = args.output_root.resolve()
|
||||
required = [
|
||||
source / "pyproject.toml",
|
||||
profiles / f"compute/h20/{MODEL}/linear_op.csv",
|
||||
profiles / f"compute/h20/{MODEL}/attention.csv",
|
||||
profiles / f"compute/h20/{MODEL}/moe.csv",
|
||||
profiles / "network/h20_nccl/all_reduce.csv",
|
||||
]
|
||||
missing = [str(path) for path in required if not path.is_file()]
|
||||
if missing:
|
||||
raise FileNotFoundError(missing)
|
||||
|
||||
results: dict[str, Any] = {}
|
||||
for label, count in (("single", 1), ("concurrency2", 2)):
|
||||
run_dir = output / label
|
||||
trace = run_dir / "trace.csv"
|
||||
write_trace(trace, count)
|
||||
command = build_command(
|
||||
args=args,
|
||||
trace=trace,
|
||||
run_dir=run_dir,
|
||||
run_id=f"t0_{label}",
|
||||
)
|
||||
write_json(run_dir / "command.json", command)
|
||||
env = dict(os.environ)
|
||||
env["PYTHONPATH"] = str(source)
|
||||
started = time.time()
|
||||
with (run_dir / "stdout.log").open("w") as stdout:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
cwd=source,
|
||||
env=env,
|
||||
stdout=stdout,
|
||||
stderr=subprocess.STDOUT,
|
||||
check=False,
|
||||
text=True,
|
||||
timeout=300,
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Frontier {label} failed with {completed.returncode}; "
|
||||
f"see {run_dir / 'stdout.log'}"
|
||||
)
|
||||
metrics = find_request_metrics(run_dir)
|
||||
results[label] = {
|
||||
"elapsed_seconds": time.time() - started,
|
||||
"trace_sha256": sha256(trace),
|
||||
"request_metrics_path": str(metrics),
|
||||
"request_metrics_sha256": sha256(metrics),
|
||||
**score(metrics),
|
||||
}
|
||||
|
||||
profile_files = required[1:]
|
||||
payload = {
|
||||
"schema": "frontier-qwen235b-t0-smoke-v1",
|
||||
"contract": {
|
||||
"topology": "TP4/DP1/MoE-TP4/EP1",
|
||||
"mns": 64,
|
||||
"mbt": 8192,
|
||||
"block_size": 16,
|
||||
"num_gpu_blocks": 26101,
|
||||
"prefix_caching": False,
|
||||
"input_tokens": 2048,
|
||||
"output_tokens": 128,
|
||||
"arrivals": "all at t=0",
|
||||
},
|
||||
"frontier": {
|
||||
"source": str(source),
|
||||
"declared_upstream_commit": "d9cfeb6d8791fbf2f295dd9744c56a666171776e",
|
||||
"python_and_config_tree_sha256": "172fc7ae19c40e67030208ff488d0d5d90764888ed76a7a543be6037ba62dc11",
|
||||
},
|
||||
"profiles": {str(path): sha256(path) for path in profile_files},
|
||||
"results": results,
|
||||
}
|
||||
write_json(output / "summary.json", payload)
|
||||
print(json.dumps(payload["results"], indent=2, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,387 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Freeze the full fixed-shape T0 Frontier response surface."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
MODEL = "Qwen3-235B-A22B-FP8"
|
||||
FRONTIER_DECLARED_UPSTREAM_COMMIT = "d9cfeb6d8791fbf2f295dd9744c56a666171776e"
|
||||
FRONTIER_PYTHON_CONFIG_TREE_SHA256 = "172fc7ae19c40e67030208ff488d0d5d90764888ed76a7a543be6037ba62dc11"
|
||||
RATES = (0.10, 0.20, 0.40, 0.80, 1.20, 1.60, 2.40, 3.20)
|
||||
TPOT_SLOS_MS = (40.0, 120.0, 150.0, 180.0)
|
||||
TTFT_SLO_MS = 1256.0
|
||||
TARGET_PASS_RATE = 0.95
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Config:
|
||||
tp: int
|
||||
mns: int
|
||||
mbt: int
|
||||
moe_tp: int
|
||||
moe_ep: int
|
||||
num_gpu_blocks: int
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return f"tp{self.tp}_mns{self.mns}_mbt{self.mbt}"
|
||||
|
||||
|
||||
GRID = tuple(
|
||||
Config(
|
||||
tp=tp,
|
||||
mns=mns,
|
||||
mbt=mbt,
|
||||
moe_tp=4 if tp == 4 else 1,
|
||||
moe_ep=1 if tp == 4 else 8,
|
||||
num_gpu_blocks=26101 if tp == 4 else 62351,
|
||||
)
|
||||
for tp in (4, 8)
|
||||
for mns in (64, 128)
|
||||
for mbt in (8192, 16384)
|
||||
)
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--frontier-source", type=Path, required=True)
|
||||
parser.add_argument("--profile-root", type=Path, required=True)
|
||||
parser.add_argument("--python", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
parser.add_argument("--requests", type=int, default=64)
|
||||
parser.add_argument("--rate", type=float, action="append")
|
||||
parser.add_argument("--config", action="append")
|
||||
parser.add_argument("--subprocess-timeout-seconds", type=int, default=1800)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def write_json(path: Path, payload: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_suffix(path.suffix + ".tmp")
|
||||
temporary.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
os.replace(temporary, path)
|
||||
|
||||
|
||||
def rate_key(rate: float) -> str:
|
||||
return f"r{rate:.2f}".replace(".", "p")
|
||||
|
||||
|
||||
def write_trace(path: Path, *, request_count: int, rate: float) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", newline="") as output:
|
||||
writer = csv.DictWriter(
|
||||
output,
|
||||
fieldnames=[
|
||||
"arrived_at",
|
||||
"num_prefill_tokens",
|
||||
"num_decode_tokens",
|
||||
"slo_ttft_ms",
|
||||
],
|
||||
)
|
||||
writer.writeheader()
|
||||
for request_id in range(request_count):
|
||||
writer.writerow(
|
||||
{
|
||||
"arrived_at": f"{request_id / rate:.12f}",
|
||||
"num_prefill_tokens": 2048,
|
||||
"num_decode_tokens": 128,
|
||||
"slo_ttft_ms": TTFT_SLO_MS,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def profile_paths(root: Path) -> dict[str, Path]:
|
||||
compute = root / "compute/h20" / MODEL
|
||||
paths = {
|
||||
"linear": compute / "linear_op.csv",
|
||||
"attention": compute / "attention.csv",
|
||||
"moe": compute / "moe.csv",
|
||||
"all_reduce": root / "network/h20_nccl/all_reduce.csv",
|
||||
"closure_manifest": root / "profile_closure_manifest.json",
|
||||
}
|
||||
missing = [str(path) for path in paths.values() if not path.is_file()]
|
||||
if missing:
|
||||
raise FileNotFoundError(missing)
|
||||
return paths
|
||||
|
||||
|
||||
def validate_attention_coverage(path: Path) -> dict[str, Any]:
|
||||
with path.open(newline="") as source:
|
||||
rows = list(csv.DictReader(source))
|
||||
standard_decode = [
|
||||
row
|
||||
for row in rows
|
||||
if row["is_prefill"].lower() == "false"
|
||||
and row.get("is_true_mixed_batch", "").lower() != "true"
|
||||
]
|
||||
true_mixed = [
|
||||
row for row in rows if row.get("is_true_mixed_batch", "").lower() == "true"
|
||||
]
|
||||
standard_by_tp = {
|
||||
str(tp): sum(int(row["num_tensor_parallel_workers"]) == tp for row in standard_decode)
|
||||
for tp in (4, 8)
|
||||
}
|
||||
mixed_by_tp = {
|
||||
str(tp): sum(int(row["num_tensor_parallel_workers"]) == tp for row in true_mixed)
|
||||
for tp in (4, 8)
|
||||
}
|
||||
if standard_by_tp != {"4": 81, "8": 81}:
|
||||
raise ValueError(f"standard decode coverage mismatch: {standard_by_tp}")
|
||||
if mixed_by_tp != {"4": 108, "8": 108}:
|
||||
raise ValueError(f"true-mixed coverage mismatch: {mixed_by_tp}")
|
||||
return {
|
||||
"rows": len(rows),
|
||||
"standard_decode_by_tp": standard_by_tp,
|
||||
"true_mixed_by_tp": mixed_by_tp,
|
||||
}
|
||||
|
||||
|
||||
def build_command(
|
||||
args: argparse.Namespace,
|
||||
paths: dict[str, Path],
|
||||
config: Config,
|
||||
trace: Path,
|
||||
run_dir: Path,
|
||||
) -> list[str]:
|
||||
cache = args.output_root / "cache" / f"tp{config.tp}"
|
||||
return [
|
||||
str(args.python), "-m", "frontier.main",
|
||||
"--simulation_mode", "offline",
|
||||
"--offline_use_generated_request_arrivals",
|
||||
"--sys_arch", "co-location",
|
||||
"--cluster_config_num_replicas", "1",
|
||||
"--replica_config_model_name", MODEL,
|
||||
"--replica_config_attn_tensor_parallel_size", str(config.tp),
|
||||
"--replica_config_attn_data_parallel_size", "1",
|
||||
"--replica_config_moe_tensor_parallel_size", str(config.moe_tp),
|
||||
"--replica_config_moe_expert_parallel_size", str(config.moe_ep),
|
||||
"--replica_config_total_expert_num", "128",
|
||||
"--replica_config_router_topk", "8",
|
||||
"--replica_config_moe_routing_mode", "simulation",
|
||||
"--replica_config_moe_routing_seed", "42",
|
||||
"--replica_config_num_pipeline_stages", "1",
|
||||
"--replica_config_device", "h20",
|
||||
"--replica_config_network_device", "h20_dgx",
|
||||
"--cc_backend_config_type", "vidur",
|
||||
"--vidur_cc_backend_config_profiling_data_dir", str(args.profile_root),
|
||||
"--vidur_cc_backend_config_cache_dir", str(cache / "collectives"),
|
||||
"--vidur_cc_backend_config_all_reduce_input_file", str(paths["all_reduce"]),
|
||||
"--replica_scheduler_config_type", "vllm_v1",
|
||||
"--decode_cuda_graph_mode", "none",
|
||||
"--vllm_v1_scheduler_config_batch_size_cap", str(config.mns),
|
||||
"--vllm_v1_scheduler_config_block_size", "16",
|
||||
"--vllm_v1_scheduler_config_num_blocks", str(config.num_gpu_blocks),
|
||||
"--vllm_v1_scheduler_config_num_blocks_mode", "explicit",
|
||||
"--vllm_v1_scheduler_config_max_tokens_in_batch", str(config.mbt),
|
||||
"--vllm_v1_scheduler_config_enable_chunked_prefill",
|
||||
"--no-vllm_v1_scheduler_config_enable_prefix_caching",
|
||||
"--request_generator_config_type", "trace_replay",
|
||||
"--trace_request_generator_config_trace_file", str(trace),
|
||||
"--trace_request_generator_config_time_scale_factor", "1",
|
||||
"--trace_request_generator_config_prefill_scale_factor", "1",
|
||||
"--trace_request_generator_config_decode_scale_factor", "1",
|
||||
"--trace_request_generator_config_max_tokens", "40960",
|
||||
"--no-random_forrest_execution_time_predictor_config_enable_dummy_mode",
|
||||
"--random_forrest_execution_time_predictor_config_linear_op_input_file", str(paths["linear"]),
|
||||
"--random_forrest_execution_time_predictor_config_atten_input_file", str(paths["attention"]),
|
||||
"--random_forrest_execution_time_predictor_config_moe_input_file", str(paths["moe"]),
|
||||
"--random_forrest_execution_time_predictor_config_all_reduce_input_file", str(paths["all_reduce"]),
|
||||
"--random_forrest_execution_time_predictor_config_prediction_max_prefill_chunk_size", "16384",
|
||||
"--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request", "40960",
|
||||
"--random_forrest_execution_time_predictor_config_prediction_max_batch_size", "128",
|
||||
"--random_forrest_execution_time_predictor_config_skip_cpu_overhead_modeling",
|
||||
"--metrics_config_cache_dir", str(cache / "execution"),
|
||||
"--metrics_config_output_dir", str(run_dir / "metrics"),
|
||||
"--metrics_config_run_id", f"{config.name}_{run_dir.name}",
|
||||
"--metrics_config_write_metrics",
|
||||
"--metrics_config_store_request_metrics",
|
||||
"--no-metrics_config_store_plots",
|
||||
"--no-metrics_config_enable_chrome_trace",
|
||||
"--no-metrics_config_write_json_trace",
|
||||
]
|
||||
|
||||
|
||||
def find_metrics(run_dir: Path) -> Path:
|
||||
matches = list((run_dir / "metrics").rglob("request_metrics.csv"))
|
||||
if len(matches) != 1:
|
||||
raise RuntimeError(f"expected one request_metrics.csv, got {matches}")
|
||||
return matches[0]
|
||||
|
||||
|
||||
def score(metrics: Path, expected_requests: int) -> dict[str, Any]:
|
||||
with metrics.open(newline="") as source:
|
||||
rows = list(csv.DictReader(source))
|
||||
if len(rows) != expected_requests:
|
||||
raise ValueError(f"request count mismatch: {len(rows)} != {expected_requests}")
|
||||
requests = []
|
||||
for row in rows:
|
||||
prompt = int(float(row["request_num_prefill_tokens"]))
|
||||
completion = int(float(row["request_num_decode_tokens"]))
|
||||
ttft = float(row["ttft"])
|
||||
e2e = float(row["request_e2e_time"])
|
||||
tpot = (e2e - ttft) / (completion - 1)
|
||||
if prompt != 2048 or completion != 128:
|
||||
raise ValueError("request shape drift")
|
||||
if not all(math.isfinite(value) and value >= 0 for value in (ttft, e2e, tpot)):
|
||||
raise ValueError("non-finite or negative latency")
|
||||
requests.append({"request_id": int(row["Request Id"]), "ttft_ms": ttft, "tpot_ms": tpot, "e2e_ms": e2e})
|
||||
slos = {}
|
||||
for limit in TPOT_SLOS_MS:
|
||||
passed = sum(row["ttft_ms"] <= TTFT_SLO_MS and row["tpot_ms"] <= limit for row in requests)
|
||||
pass_rate = passed / len(requests)
|
||||
slos[f"tpot_{int(limit)}ms"] = {
|
||||
"passed": passed,
|
||||
"pass_rate": pass_rate,
|
||||
"feasible": pass_rate >= TARGET_PASS_RATE,
|
||||
}
|
||||
return {"requests": requests, "slos": slos}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
args.frontier_source = args.frontier_source.resolve()
|
||||
args.profile_root = args.profile_root.resolve()
|
||||
# A venv interpreter is commonly a symlink to the system executable.
|
||||
# Keep the venv path so Python discovers that environment's site-packages.
|
||||
args.python = args.python.absolute()
|
||||
args.output_root = args.output_root.resolve()
|
||||
if args.requests < 2:
|
||||
raise ValueError("requests must be at least two")
|
||||
rates = tuple(args.rate or RATES)
|
||||
if any(rate <= 0 for rate in rates) or len(set(rates)) != len(rates):
|
||||
raise ValueError("rates must be positive and unique")
|
||||
selected = list(GRID)
|
||||
if args.config:
|
||||
names = set(args.config)
|
||||
selected = [config for config in GRID if config.name in names]
|
||||
if {config.name for config in selected} != names:
|
||||
raise ValueError(f"unknown configs: {names - {config.name for config in selected}}")
|
||||
|
||||
paths = profile_paths(args.profile_root)
|
||||
coverage = validate_attention_coverage(paths["attention"])
|
||||
args.output_root.mkdir(parents=True, exist_ok=True)
|
||||
traces = {}
|
||||
for rate in rates:
|
||||
path = args.output_root / "traces" / f"{rate_key(rate)}.csv"
|
||||
write_trace(path, request_count=args.requests, rate=rate)
|
||||
traces[rate] = path
|
||||
|
||||
config_results = []
|
||||
environment = dict(os.environ)
|
||||
environment.update(
|
||||
{
|
||||
"PYTHONPATH": str(args.frontier_source),
|
||||
"WANDB_DISABLED": "true",
|
||||
"VIDUR_DISABLE_WANDB": "1",
|
||||
# The best-effort source emits a per-layer OP-TRACE at INFO. It is
|
||||
# diagnostic only and can produce hundreds of MiB per T0 cell.
|
||||
"FRONTIER_LOG_LEVEL": "WARNING",
|
||||
}
|
||||
)
|
||||
for config in selected:
|
||||
loads = []
|
||||
for rate in rates:
|
||||
run_dir = args.output_root / "runs" / config.name / rate_key(rate)
|
||||
result_path = run_dir / "result.json"
|
||||
if result_path.is_file():
|
||||
result = json.loads(result_path.read_text())
|
||||
if result.get("status") == "completed":
|
||||
loads.append(result)
|
||||
continue
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
command = build_command(args, paths, config, traces[rate], run_dir)
|
||||
write_json(run_dir / "command.json", command)
|
||||
started = time.time()
|
||||
with (run_dir / "stdout.log").open("w") as output:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
cwd=args.frontier_source,
|
||||
env=environment,
|
||||
stdout=output,
|
||||
stderr=subprocess.STDOUT,
|
||||
timeout=args.subprocess_timeout_seconds,
|
||||
check=False,
|
||||
text=True,
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
raise RuntimeError(f"Frontier failed: {config.name} rate={rate}, rc={completed.returncode}")
|
||||
metrics = find_metrics(run_dir)
|
||||
result = {
|
||||
"status": "completed",
|
||||
"config": asdict(config) | {"name": config.name},
|
||||
"offered_request_rate": rate,
|
||||
"request_rate_per_gpu": rate / config.tp,
|
||||
"elapsed_seconds": time.time() - started,
|
||||
"trace_sha256": sha256(traces[rate]),
|
||||
"request_metrics_sha256": sha256(metrics),
|
||||
**score(metrics, args.requests),
|
||||
}
|
||||
write_json(result_path, result)
|
||||
loads.append(result)
|
||||
print(json.dumps({"config": config.name, "rate": rate, "elapsed_seconds": result["elapsed_seconds"], "slos": result["slos"]}, sort_keys=True), flush=True)
|
||||
config_results.append({"config": asdict(config) | {"name": config.name}, "loads": loads})
|
||||
|
||||
rankings = {}
|
||||
for slo in (f"tpot_{int(value)}ms" for value in TPOT_SLOS_MS):
|
||||
records = []
|
||||
for item in config_results:
|
||||
feasible = [load["offered_request_rate"] for load in item["loads"] if load["slos"][slo]["feasible"]]
|
||||
capacity = max(feasible) if feasible else None
|
||||
records.append({
|
||||
"config": item["config"],
|
||||
"maximum_tested_feasible_request_rate": capacity,
|
||||
"maximum_tested_feasible_request_rate_per_gpu": capacity / item["config"]["tp"] if capacity is not None else None,
|
||||
"lower_censored": capacity is None,
|
||||
"upper_censored": capacity == max(rates) if capacity is not None else False,
|
||||
})
|
||||
records.sort(key=lambda row: (-(row["maximum_tested_feasible_request_rate_per_gpu"] if row["maximum_tested_feasible_request_rate_per_gpu"] is not None else -1), row["config"]["name"]))
|
||||
rankings[slo] = records
|
||||
|
||||
is_complete_preregistered_surface = (
|
||||
selected == list(GRID) and rates == RATES and args.requests == 64
|
||||
)
|
||||
manifest = {
|
||||
"schema": "frontier-qwen235b-t0-surface-v1",
|
||||
"status": (
|
||||
"frozen_before_real_surface"
|
||||
if is_complete_preregistered_surface
|
||||
else "partial_surface_not_decision_bearing"
|
||||
),
|
||||
"contract": {"requests_per_anchor": args.requests, "rates": rates, "input_tokens": 2048, "output_tokens": 128, "ttft_slo_ms": TTFT_SLO_MS, "tpot_slos_ms": TPOT_SLOS_MS, "target_pass_rate": TARGET_PASS_RATE, "prefix_caching": False},
|
||||
"frontier": {
|
||||
"source": str(args.frontier_source),
|
||||
"declared_upstream_commit": FRONTIER_DECLARED_UPSTREAM_COMMIT,
|
||||
"python_and_config_tree_sha256": FRONTIER_PYTHON_CONFIG_TREE_SHA256,
|
||||
"fingerprint_source": "prefill-grid-v3 frozen run manifest for the same immutable source snapshot",
|
||||
},
|
||||
"runner": {"path": str(Path(__file__).resolve()), "sha256": sha256(Path(__file__).resolve())},
|
||||
"profiles": {"root": str(args.profile_root), "coverage": coverage, "files_sha256": {name: sha256(path) for name, path in paths.items()}},
|
||||
"config_results": config_results,
|
||||
"rankings": rankings,
|
||||
}
|
||||
write_json(args.output_root / "frontier_surface_frozen.json", manifest)
|
||||
print(args.output_root / "frontier_surface_frozen.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-$(pwd)/artifacts/t0-full-attention-profile-20260716}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6-best-effort-v6-batched-lanes-r3}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
PROFILE_ROOT="${OUTPUT_ROOT}/profiles"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
PROVENANCE_DIR="${OUTPUT_ROOT}/provenance"
|
||||
MODEL="Qwen3-235B-A22B-FP8"
|
||||
|
||||
mkdir -p "${PROFILE_ROOT}" "${LOG_DIR}" "${PROVENANCE_DIR}"
|
||||
exec > >(tee -a "${LOG_DIR}/profile.log") 2>&1
|
||||
|
||||
if [[ -z "${CUDA_VISIBLE_DEVICES:-}" ]]; then
|
||||
echo "ERROR: CUDA_VISIBLE_DEVICES must contain the fleet-allocated GPU" >&2
|
||||
exit 1
|
||||
fi
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne 1 ]]; then
|
||||
echo "ERROR: expected exactly one GPU, got ${CUDA_VISIBLE_DEVICES}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "FULL_PROFILE_LAUNCH_ECHO host=$(hostname) gpu=${CUDA_VISIBLE_DEVICES} model=${MODEL} operator=FlashInfer_attention phases=standard_decode,true_mixed TP_workers=4,8 batch_sizes=1,2,4,8,16,32,64,96,128 kv_sizes=2048:2175 true_mixed_prefill_chunk=2048 block=16 measurement=CUDA_EVENT output=${OUTPUT_ROOT} expected_wall=5-10m hard_wall=900s hard_gpu_cap=0.25_H20h"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
|
||||
test -x "${VENV_ROOT}/bin/python"
|
||||
test -f "${FRONTIER_ROOT}/pyproject.toml"
|
||||
test -f "${FRONTIER_ROOT}/data/config/models/${MODEL}.json"
|
||||
sha256sum run_t0_full_attention_profile.sh > "${PROVENANCE_DIR}/source.sha256"
|
||||
|
||||
export PYTHONPATH="${FRONTIER_ROOT}"
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
|
||||
cd "${FRONTIER_ROOT}"
|
||||
timeout --signal=TERM --kill-after=30s 780 \
|
||||
"${VENV_ROOT}/bin/python" -m frontier.profiling.attention.main \
|
||||
--disable_ray \
|
||||
--models "${MODEL}" \
|
||||
--num_gpus 1 \
|
||||
--max_model_len 40960 \
|
||||
--max_seq_len 2176 \
|
||||
--min_batch_size 1 \
|
||||
--max_batch_size 128 \
|
||||
--batch_size_list 1 2 4 8 16 32 64 96 128 \
|
||||
--decode_kv_cache_size_list 2048 2064 2080 2096 2112 2128 2144 2160 2175 \
|
||||
--num_tensor_parallel_workers 4 8 \
|
||||
--max_pipeline_parallel_size 1 \
|
||||
--attention_backend FLASHINFER \
|
||||
--block_size 16 \
|
||||
--enable_true_mixed \
|
||||
--true_mixed_prefill_batch_sizes 1 2 4 7 \
|
||||
--true_mixed_prefill_chunk_sizes 2048 \
|
||||
--true_mixed_decode_batch_sizes 1 2 4 8 16 32 64 96 124 127 \
|
||||
--true_mixed_decode_kv_cache_sizes 2048 2112 2175 \
|
||||
--true_mixed_prefill_kv_cache_size 0 \
|
||||
--device h20 \
|
||||
--profile_method cuda_event \
|
||||
--output_dir "${PROFILE_ROOT}" \
|
||||
--yes
|
||||
|
||||
MODEL_PROFILE_DIR="${PROFILE_ROOT}/compute/h20/${MODEL}"
|
||||
STANDARD_CSV="${MODEL_PROFILE_DIR}/attention.csv"
|
||||
TRUE_MIXED_CSV="${MODEL_PROFILE_DIR}/attention_true_mixed.csv"
|
||||
COMBINED_CSV="${MODEL_PROFILE_DIR}/attention_combined.csv"
|
||||
test -s "${STANDARD_CSV}"
|
||||
test -s "${TRUE_MIXED_CSV}"
|
||||
test -s "${COMBINED_CSV}"
|
||||
|
||||
"${VENV_ROOT}/bin/python" - "${STANDARD_CSV}" "${TRUE_MIXED_CSV}" \
|
||||
> "${PROVENANCE_DIR}/coverage.json" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
standard_path, true_mixed_path = sys.argv[1:]
|
||||
standard = pd.read_csv(standard_path)
|
||||
true_mixed = pd.read_csv(true_mixed_path)
|
||||
decode = standard[standard["is_prefill"] == False] # noqa: E712
|
||||
payload = {
|
||||
"standard_path": standard_path,
|
||||
"standard_rows": len(standard),
|
||||
"decode_rows": len(decode),
|
||||
"decode_rows_by_tp": {
|
||||
str(int(key)): int(value)
|
||||
for key, value in decode.groupby("num_tensor_parallel_workers").size().items()
|
||||
},
|
||||
"decode_batch_sizes": sorted(int(value) for value in decode["batch_size"].unique()),
|
||||
"decode_kv_cache_sizes": sorted(
|
||||
int(value) for value in decode["kv_cache_size"].unique()
|
||||
),
|
||||
"decode_median_non_null": int(
|
||||
decode["time_stats.attn_decode.median"].notna().sum()
|
||||
),
|
||||
"true_mixed_path": true_mixed_path,
|
||||
"true_mixed_rows": len(true_mixed),
|
||||
"true_mixed_rows_by_tp": {
|
||||
str(int(key)): int(value)
|
||||
for key, value in true_mixed.groupby("num_tensor_parallel_workers").size().items()
|
||||
},
|
||||
"true_mixed_decode_median_non_null": int(
|
||||
true_mixed["time_stats.attn_decode.median"].notna().sum()
|
||||
),
|
||||
"true_mixed_prefill_median_non_null": int(
|
||||
true_mixed["time_stats.attn_prefill.median"].notna().sum()
|
||||
),
|
||||
}
|
||||
print(json.dumps(payload, indent=2, sort_keys=True))
|
||||
expected_batch_sizes = [1, 2, 4, 8, 16, 32, 64, 96, 128]
|
||||
expected_kv_sizes = [2048, 2064, 2080, 2096, 2112, 2128, 2144, 2160, 2175]
|
||||
if payload["decode_rows_by_tp"] != {"4": 81, "8": 81}:
|
||||
raise SystemExit(1)
|
||||
if payload["decode_batch_sizes"] != expected_batch_sizes:
|
||||
raise SystemExit(1)
|
||||
if payload["decode_kv_cache_sizes"] != expected_kv_sizes:
|
||||
raise SystemExit(1)
|
||||
if payload["decode_median_non_null"] != 162:
|
||||
raise SystemExit(1)
|
||||
if set(payload["true_mixed_rows_by_tp"]) != {"4", "8"}:
|
||||
raise SystemExit(1)
|
||||
if payload["true_mixed_rows"] < 100:
|
||||
raise SystemExit(1)
|
||||
if payload["true_mixed_decode_median_non_null"] != payload["true_mixed_rows"]:
|
||||
raise SystemExit(1)
|
||||
if payload["true_mixed_prefill_median_non_null"] != payload["true_mixed_rows"]:
|
||||
raise SystemExit(1)
|
||||
PY
|
||||
|
||||
sha256sum \
|
||||
"${STANDARD_CSV}" \
|
||||
"${TRUE_MIXED_CSV}" \
|
||||
"${COMBINED_CSV}" \
|
||||
"${PROVENANCE_DIR}/coverage.json" \
|
||||
"${PROVENANCE_DIR}/source.sha256" \
|
||||
> "${PROVENANCE_DIR}/artifacts.sha256"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "T0_FULL_ATTENTION_PROFILE_COMPLETE"
|
||||
137
runs/frontier-multicase-sufficiency-v1/run_t0_real_config.sh
Normal file
137
runs/frontier-multicase-sufficiency-v1/run_t0_real_config.sh
Normal file
@@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:?OUTPUT_ROOT is required}"
|
||||
TP="${TP:?TP is required}"
|
||||
MNS="${MNS:?MNS is required}"
|
||||
MBT="${MBT:?MBT is required}"
|
||||
RATES="${RATES:?RATES is required}"
|
||||
SERVER_PORT="${SERVER_PORT:?SERVER_PORT is required}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8}"
|
||||
SERVED_MODEL="qwen3-235b-t0-surface"
|
||||
SERVER_PID=""
|
||||
|
||||
mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/provenance"
|
||||
exec > >(tee -a "${OUTPUT_ROOT}/logs/controller.log") 2>&1
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
for _ in $(seq 1 30); do
|
||||
kill -0 "${SERVER_PID}" 2>/dev/null || break
|
||||
sleep 1
|
||||
done
|
||||
kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
fi
|
||||
SERVER_PID=""
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES:-}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne "${TP}" ]]; then
|
||||
echo "ERROR: expected ${TP} allocated GPUs, got ${CUDA_VISIBLE_DEVICES:-unset}" >&2
|
||||
exit 1
|
||||
fi
|
||||
read -r -a RATE_ARRAY <<< "${RATES}"
|
||||
if [[ "${#RATE_ARRAY[@]}" -lt 1 ]]; then
|
||||
echo "ERROR: at least one frozen rate is required" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "REAL_CONFIG_LAUNCH_ECHO host=$(hostname) gpus=${CUDA_VISIBLE_DEVICES} model=${MODEL_ROOT} config=TP${TP}_MNS${MNS}_MBT${MBT} rates=${RATES// /,} repeats=2 requests_per_anchor=64 isolation=fresh_server_per_anchor target_warmup=min32_max4_ceil_rate_x20 trace=fixed_ISL2048_OSL128 prefix=off runtime=community_vllm_0.10.2 execution=eager kv=BF16 output=${OUTPUT_ROOT}"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
sha256sum run_t0_real_config.sh t0_rate_client.py > "${OUTPUT_ROOT}/provenance/source.sha256"
|
||||
"${VENV_ROOT}/bin/python" - "${TP}" "${MNS}" "${MBT}" "${RATES}" \
|
||||
> "${OUTPUT_ROOT}/provenance/contract.json" <<'PY'
|
||||
import importlib.metadata as metadata
|
||||
import json
|
||||
import platform
|
||||
import sys
|
||||
|
||||
tp, mns, mbt, rates = sys.argv[1:]
|
||||
print(json.dumps({
|
||||
"python": platform.python_version(),
|
||||
"torch": metadata.version("torch"),
|
||||
"transformers": metadata.version("transformers"),
|
||||
"vllm": metadata.version("vllm"),
|
||||
"config": {"tp": int(tp), "mns": int(mns), "mbt": int(mbt)},
|
||||
"rates": [float(value) for value in rates.split()],
|
||||
"rounds": 2,
|
||||
"requests_per_anchor": 64,
|
||||
"anchor_isolation": "fresh_server_per_rate_per_round",
|
||||
"target_rate_warmup_requests": "min(32, max(4, ceil(rate * 20)))",
|
||||
"ttft_slo_ms": 1256.0,
|
||||
"tpot_slos_ms": [40.0, 120.0, 150.0, 180.0],
|
||||
}, indent=2, sort_keys=True))
|
||||
PY
|
||||
nvidia-smi --query-gpu=index,name,uuid,driver_version --format=csv,noheader \
|
||||
> "${OUTPUT_ROOT}/provenance/gpus.csv"
|
||||
sha256sum "${MODEL_ROOT}/config.json" > "${OUTPUT_ROOT}/provenance/model.sha256"
|
||||
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export VLLM_USE_V1=1
|
||||
export VLLM_ATTENTION_BACKEND=FLASHINFER
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
|
||||
EXTRA_FLAGS=()
|
||||
NUM_BLOCKS=26101
|
||||
if [[ "${TP}" -eq 8 ]]; then
|
||||
EXTRA_FLAGS+=(--enable-expert-parallel)
|
||||
NUM_BLOCKS=62351
|
||||
fi
|
||||
|
||||
for ROUND in 1 2; do
|
||||
ROUND_ROOT="${OUTPUT_ROOT}/round${ROUND}"
|
||||
mkdir -p "${ROUND_ROOT}/logs" "${ROUND_ROOT}/results"
|
||||
ORDERED_RATES=("${RATE_ARRAY[@]}")
|
||||
if [[ "${ROUND}" -eq 2 ]]; then
|
||||
ORDERED_RATES=()
|
||||
for ((index=${#RATE_ARRAY[@]}-1; index>=0; index--)); do ORDERED_RATES+=("${RATE_ARRAY[index]}"); done
|
||||
fi
|
||||
for RATE in "${ORDERED_RATES[@]}"; do
|
||||
KEY="$(printf 'r%.2f' "${RATE}" | tr '.' 'p')"
|
||||
SERVER_LOG="${ROUND_ROOT}/logs/server_${KEY}.log"
|
||||
setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
|
||||
--host 127.0.0.1 --port "${SERVER_PORT}" --served-model-name "${SERVED_MODEL}" \
|
||||
--tensor-parallel-size "${TP}" --disable-custom-all-reduce --quantization fp8 \
|
||||
--gpu-memory-utilization 0.80 --num-gpu-blocks-override "${NUM_BLOCKS}" \
|
||||
--kv-cache-dtype auto --max-model-len 40960 --max-num-batched-tokens "${MBT}" \
|
||||
--max-num-seqs "${MNS}" --no-enable-prefix-caching --enable-chunked-prefill \
|
||||
--enforce-eager --disable-log-requests "${EXTRA_FLAGS[@]}" \
|
||||
> "${SERVER_LOG}" 2>&1 &
|
||||
SERVER_PID=$!
|
||||
READY=0
|
||||
for _ in $(seq 1 180); do
|
||||
if curl -fsS --max-time 2 "http://127.0.0.1:${SERVER_PORT}/v1/models" \
|
||||
> "${ROUND_ROOT}/results/models_${KEY}.json" 2>/dev/null; then
|
||||
READY=1
|
||||
break
|
||||
fi
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then tail -200 "${SERVER_LOG}"; exit 1; fi
|
||||
sleep 5
|
||||
done
|
||||
if [[ "${READY}" -ne 1 ]]; then tail -200 "${SERVER_LOG}"; exit 1; fi
|
||||
|
||||
WARMUP_REQUESTS="$("${VENV_ROOT}/bin/python" - "${RATE}" <<'PY'
|
||||
import math
|
||||
import sys
|
||||
print(min(32, max(4, math.ceil(float(sys.argv[1]) * 20.0))))
|
||||
PY
|
||||
)"
|
||||
"${VENV_ROOT}/bin/python" t0_rate_client.py --port "${SERVER_PORT}" \
|
||||
--served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" --rate "${RATE}" \
|
||||
--requests "${WARMUP_REQUESTS}" \
|
||||
--output "${ROUND_ROOT}/results/warmup_${KEY}.json"
|
||||
"${VENV_ROOT}/bin/python" t0_rate_client.py --port "${SERVER_PORT}" \
|
||||
--served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" --rate "${RATE}" \
|
||||
--requests 64 --output "${ROUND_ROOT}/results/${KEY}.json"
|
||||
cleanup
|
||||
done
|
||||
done
|
||||
|
||||
find "${OUTPUT_ROOT}" -type f ! -path '*/provenance/artifacts.sha256' -print0 \
|
||||
| sort -z | xargs -0 sha256sum > "${OUTPUT_ROOT}/provenance/artifacts.sha256"
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "T0_REAL_CONFIG_COMPLETE"
|
||||
146
runs/frontier-multicase-sufficiency-v1/run_t0_smoke.sh
Normal file
146
runs/frontier-multicase-sufficiency-v1/run_t0_smoke.sh
Normal file
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_ROOT="${OUTPUT_ROOT:-$(pwd)/artifacts/t0-smoke-20260716}"
|
||||
VENV_ROOT="${VENV_ROOT:-/tmp/wjh-frontier-vllm0102-smoke/.venv}"
|
||||
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8}"
|
||||
FRONTIER_ROOT="${FRONTIER_ROOT:-/home/admin/cpfs/wjh/frontier-community-qwen235-smoke-20260715/Frontier-d9cfeb6}"
|
||||
SERVER_PORT="${SERVER_PORT:-18910}"
|
||||
SERVED_MODEL="qwen3-235b-t0-smoke"
|
||||
LOG_DIR="${OUTPUT_ROOT}/logs"
|
||||
RESULT_DIR="${OUTPUT_ROOT}/results"
|
||||
PROVENANCE_DIR="${OUTPUT_ROOT}/provenance"
|
||||
SERVER_PID=""
|
||||
|
||||
mkdir -p "${LOG_DIR}" "${RESULT_DIR}" "${PROVENANCE_DIR}"
|
||||
exec > >(tee -a "${LOG_DIR}/smoke.log") 2>&1
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
for _ in $(seq 1 30); do
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
if [[ -z "${CUDA_VISIBLE_DEVICES:-}" ]]; then
|
||||
echo "ERROR: CUDA_VISIBLE_DEVICES must contain the four fleet-allocated GPUs" >&2
|
||||
exit 1
|
||||
fi
|
||||
IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES}"
|
||||
if [[ "${#GPU_IDS[@]}" -ne 4 ]]; then
|
||||
echo "ERROR: expected exactly four GPUs, got ${CUDA_VISIBLE_DEVICES}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "LAUNCH_ECHO host=$(hostname) gpus=${CUDA_VISIBLE_DEVICES} model=${MODEL_ROOT} trace=fixed_ISL2048_OSL128 prefix=off qps=single_then_concurrency2 runtime=community_vllm_0.10.2 topology=TP4_DP1 execution=eager kv=BF16 spec=off cuda_graph=off output=${OUTPUT_ROOT} expected_wall=20-30m hard_wall=1800s hard_gpu_cap=2_H20h"
|
||||
date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
|
||||
test -x "${VENV_ROOT}/bin/vllm"
|
||||
test -x "${VENV_ROOT}/bin/python"
|
||||
test -f "${MODEL_ROOT}/config.json"
|
||||
test -f "${FRONTIER_ROOT}/pyproject.toml"
|
||||
test -f "$(pwd)/t0_smoke_client.py"
|
||||
|
||||
sha256sum run_t0_smoke.sh t0_smoke_client.py > "${PROVENANCE_DIR}/source.sha256"
|
||||
"${VENV_ROOT}/bin/python" - <<'PY' > "${PROVENANCE_DIR}/environment.json"
|
||||
import importlib.metadata as metadata
|
||||
import json
|
||||
import platform
|
||||
|
||||
print(json.dumps({
|
||||
"python": platform.python_version(),
|
||||
"torch": metadata.version("torch"),
|
||||
"transformers": metadata.version("transformers"),
|
||||
"vllm": metadata.version("vllm"),
|
||||
}, indent=2, sort_keys=True))
|
||||
PY
|
||||
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
export VLLM_USE_V1=1
|
||||
export VLLM_ATTENTION_BACKEND=FLASHINFER
|
||||
export TORCH_CUDA_ARCH_LIST=9.0
|
||||
|
||||
echo "STAGE server_start"
|
||||
setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
|
||||
--host 127.0.0.1 \
|
||||
--port "${SERVER_PORT}" \
|
||||
--served-model-name "${SERVED_MODEL}" \
|
||||
--tensor-parallel-size 4 \
|
||||
--disable-custom-all-reduce \
|
||||
--quantization fp8 \
|
||||
--gpu-memory-utilization 0.80 \
|
||||
--kv-cache-dtype auto \
|
||||
--max-model-len 40960 \
|
||||
--max-num-batched-tokens 8192 \
|
||||
--max-num-seqs 64 \
|
||||
--no-enable-prefix-caching \
|
||||
--enable-chunked-prefill \
|
||||
--enforce-eager \
|
||||
--disable-log-requests \
|
||||
> "${LOG_DIR}/server.log" 2>&1 &
|
||||
SERVER_PID=$!
|
||||
|
||||
READY=0
|
||||
for _ in $(seq 1 180); do
|
||||
if curl -fsS --max-time 2 "http://127.0.0.1:${SERVER_PORT}/v1/models" \
|
||||
> "${RESULT_DIR}/models.json" 2>/dev/null; then
|
||||
READY=1
|
||||
break
|
||||
fi
|
||||
if ! kill -0 "${SERVER_PID}" 2>/dev/null; then
|
||||
echo "ERROR: vLLM server exited before readiness" >&2
|
||||
tail -200 "${LOG_DIR}/server.log" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
if [[ "${READY}" -ne 1 ]]; then
|
||||
echo "ERROR: vLLM server did not become ready within 900 seconds" >&2
|
||||
tail -200 "${LOG_DIR}/server.log" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "STAGE fixed_shape_single"
|
||||
"${VENV_ROOT}/bin/python" t0_smoke_client.py \
|
||||
--port "${SERVER_PORT}" \
|
||||
--served-model "${SERVED_MODEL}" \
|
||||
--model-path "${MODEL_ROOT}" \
|
||||
--input-tokens 2048 \
|
||||
--output-tokens 128 \
|
||||
--concurrency 1 \
|
||||
--requests 1 \
|
||||
--output "${RESULT_DIR}/single.json"
|
||||
|
||||
echo "STAGE fixed_shape_concurrency2"
|
||||
"${VENV_ROOT}/bin/python" t0_smoke_client.py \
|
||||
--port "${SERVER_PORT}" \
|
||||
--served-model "${SERVED_MODEL}" \
|
||||
--model-path "${MODEL_ROOT}" \
|
||||
--input-tokens 2048 \
|
||||
--output-tokens 128 \
|
||||
--concurrency 2 \
|
||||
--requests 2 \
|
||||
--output "${RESULT_DIR}/concurrency2.json"
|
||||
|
||||
cleanup
|
||||
SERVER_PID=""
|
||||
|
||||
sha256sum \
|
||||
"${PROVENANCE_DIR}/environment.json" \
|
||||
"${PROVENANCE_DIR}/source.sha256" \
|
||||
"${RESULT_DIR}/models.json" \
|
||||
"${RESULT_DIR}/single.json" \
|
||||
"${RESULT_DIR}/concurrency2.json" \
|
||||
> "${PROVENANCE_DIR}/artifacts.sha256"
|
||||
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
|
||||
date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
|
||||
echo "T0_SMOKE_COMPLETE"
|
||||
89
runs/frontier-multicase-sufficiency-v1/t0-smoke-report.md
Normal file
89
runs/frontier-multicase-sufficiency-v1/t0-smoke-report.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# Qwen235B T0 fixed-shape smoke report
|
||||
|
||||
日期：2026-07-16。状态：real 与 profile-closed Frontier smoke complete；不是 config-rank evaluation。
|
||||
|
||||
## Setup
|
||||
|
||||
| 项目 | 设置 |
|
||||
|---|---|
|
||||
| host | `dash0`，仅 GPU 0--3；GPU 4--7 始终空闲 |
|
||||
| model | `Qwen/Qwen3-235B-A22B-FP8` |
|
||||
| runtime | community vLLM 0.10.2、torch 2.8.0、Transformers 4.55.2 |
|
||||
| config | TP4/DP1、MNS64、MBT8192、block16、26,101 KV blocks/GPU |
|
||||
| execution | FlashInfer、eager、BF16 KV、prefix/spec/CUDA graph off |
|
||||
| workload | exact prompt token IDs；ISL=2,048、OSL=128；single 和两个同时到达 requests |
|
||||
| SLO | TTFT ≤1,256 ms、TPOT ≤40 ms，joint pass |
|
||||
| GPU cost | 02:33:48--02:36:01 UTC，133 s × 4 H20 = 0.148 H20-hours |
|
||||
|
||||
vLLM 报告每卡 model weights 约55.13 GiB、KV capacity 417,616 tokens，以及 `num_gpu_blocks=26101`。这与此前输入 Frontier 的 TP4 KV capacity 一致。
|
||||
|
||||
## Real result
|
||||
|
||||
| load | request | usage | TTFT | TPOT | E2E | joint pass |
|
||||
|---|---:|---|---:|---:|---:|---:|
|
||||
| single | 0 | 2,048+128 | 586.23 ms | 136.18 ms | 17,881.30 ms | no |
|
||||
| concurrency=2 | 0 | 2,048+128 | 186.09 ms | 124.87 ms | 16,045.24 ms | no |
|
||||
| concurrency=2 | 1 | 2,048+128 | 434.29 ms | 123.78 ms | 16,154.03 ms | no |
|
||||
|
||||
所有 response 的 usage completion tokens 和逐 chunk token IDs 都是128；不是 EOS 提前退出或 client length-hint 误差。artifact SHA256 全部通过 harvest 后复验。
|
||||
|
||||
单请求已经违反40 ms TPOT，说明该 SLO 对 community/eager controlled stack 在最低负载下不可行。concurrency=2 的 batching 将平均 TPOT 改善到124.33 ms，但仍不足以通过。这是 pilot finding，不应用于事后挑选一个恰好产生所需 ranking 的阈值。
|
||||
|
||||
## Frontier result
|
||||
|
||||
Frontier 使用冻结的 best-effort source：
|
||||
|
||||
```text
|
||||
upstream commit: d9cfeb6d8791fbf2f295dd9744c56a666171776e
|
||||
source/config tree SHA256: 172fc7ae19c40e67030208ff488d0d5d90764888ed76a7a543be6037ba62dc11
|
||||
profile root: profiles-best-effort-final-v2
|
||||
TP4/DP1/MoE-TP4/EP1, MNS64, MBT8192, block16, 26101 blocks
|
||||
```
|
||||
|
||||
simulator 正确加载1个请求，执行2,048-token prefill，并提交第一个 decode token。下一轮成为 pure-decode batch 时失败：
|
||||
|
||||
```text
|
||||
Skipping eager attn_decode training: no standard decode rows
|
||||
ValueError: attention decode prediction cache not found for cluster monolithic
|
||||
```
|
||||
|
||||
profile audit 确认 final attention CSV 有726 rows，但 `is_prefill` 只有 `True`。另外两个已有 Qwen235 attention files 也只有 prefill rows。由于没有产生完整 request metrics，本文不报告 simulator TTFT/TPOT，也不把 crash 当成 SLO failure。使用 dummy/fallback latency 会把缺失 measurement coverage 隐藏成虚假的 simulator prediction。
|
||||
|
||||
## Profile closure 与 rerun
|
||||
|
||||
在 `dash0` 单张 H20 上，用同一 vLLM 0.10.2/FlashInfer stack 测量 TP4、batch `{1,2}`、KV `{2048,2176}` 的4个 CUDA-event decode-attention points；4/4 rows 的 `attn_decode.median` 非空。它们与原726个 prefill rows 按完全相同的55-column schema 合并到新的 immutable profile root，未覆盖原 profile，也未加入 dummy 或 E2E calibration。
|
||||
|
||||
同一 Frontier smoke 随后精确完成1/2个请求：
|
||||
|
||||
| load | request | TTFT | TPOT | E2E |
|
||||
|---|---:|---:|---:|---:|
|
||||
| single | 0 | 267.67 ms | 88.27 ms | 11,478.45 ms |
|
||||
| concurrency=2 | 0 | 470.66 ms | 90.54 ms | 11,969.73 ms |
|
||||
| concurrency=2 | 1 | 470.66 ms | 90.54 ms | 11,969.73 ms |
|
||||
|
||||
请求数、ISL/OSL 与 real contract 完全一致，所有 latency 均有限且非负，representation gate 因而通过。absolute TPOT error 并非固定比例：single 低估约35%，concurrency=2 低估约27%。这组数据只能证明 simulator 现在能表达该 state path，不能证明 config rank 正确。
|
||||
|
||||
steady-QPS 下还会出现 prefill+decode true-mixed batches，因此 full surface 没有沿用4-row smoke profile。后续完整 closure 覆盖 TP4/TP8 各81个 standard-decode 与108个 true-mixed points；合并后的 attention root 共1,104 rows，并已确认 `attn_decode_in_mixed` 从每个 TP 的108个真实 samples 训练。
|
||||
|
||||
## Interpretation
|
||||
|
||||
Change：首次把原 prefill-only compatibility envelope用于有真实 decode tokens 的最简单 fixed-shape case。
|
||||
|
||||
Expected effect：如果 profiles 和 execution model 已闭合，Frontier 应至少产生同一 config 的 TTFT/TPOT，之后才能讨论绝对 gap 或 rank。
|
||||
|
||||
Verification：real exact-token streaming run；Frontier 同 config/blocks/profile run；profile CSV 与完整 traceback 审计。
|
||||
|
||||
Result：real serving 成功；原 Frontier compatibility envelope 在 pure-decode attention 处断裂；补齐最小 measurement coverage 后 smoke 通过，但出现27%--35%的 TPOT absolute error。因此此前的 prefill rank-match 仍不能支持“Frontier 已足够解决 Qwen235 mixed config selection”的外推，必须看完整 config response surface。
|
||||
|
||||
Remaining risk：完整 profile 已加入 mixed-attention coverage，但 MoE decode routing context、EP8 communication、batch-dependent TPOT 和 simulator/real action ranking 仍需由 full surface 检验；smoke 只关闭第一个必要缺口，并未证明它是唯一缺口。
|
||||
|
||||
## Evidence
|
||||
|
||||
- Real single request：[single.json](fleet-artifacts/qwen235b-t0-tp4-smoke-20260716-v1-20260716T023346844896Z/artifacts/artifacts/t0-smoke-20260716/results/single.json)
|
||||
- Real concurrency=2：[concurrency2.json](fleet-artifacts/qwen235b-t0-tp4-smoke-20260716-v1-20260716T023346844896Z/artifacts/artifacts/t0-smoke-20260716/results/concurrency2.json)
|
||||
- Server log：[server.log](fleet-artifacts/qwen235b-t0-tp4-smoke-20260716-v1-20260716T023346844896Z/artifacts/artifacts/t0-smoke-20260716/logs/server.log)
|
||||
- Frontier command：[command.json](frontier-smoke-failure/single/command.json)
|
||||
- Frontier traceback：[stdout.log](frontier-smoke-failure/single/stdout.log)
|
||||
- Frontier trace：[trace.csv](frontier-smoke-failure/single/trace.csv)
|
||||
- Minimal decode profile coverage：[coverage.json](fleet-artifacts/qwen235b-decode-attention-profile-20260716-v1-20260716T030929269383Z/artifacts/artifacts/decode-attention-profile-20260716/provenance/coverage.json)
|
||||
- Full attention profile coverage：[coverage.json](fleet-artifacts/qwen235b-t0-full-attention-profile-20260716-v1-20260716T032536186518Z/artifacts/artifacts/t0-full-attention-profile-20260716/provenance/coverage.json)
|
||||
223
runs/frontier-multicase-sufficiency-v1/t0_rate_client.py
Normal file
223
runs/frontier-multicase-sufficiency-v1/t0_rate_client.py
Normal file
@@ -0,0 +1,223 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Open-loop fixed-shape completion workload for one T0 offered-load anchor."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import hashlib
|
||||
import http.client
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
TPOT_SLOS_MS = (40.0, 120.0, 150.0, 180.0)
|
||||
TTFT_SLO_MS = 1256.0
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, required=True)
|
||||
parser.add_argument("--served-model", required=True)
|
||||
parser.add_argument("--model-path", type=Path, required=True)
|
||||
parser.add_argument("--rate", type=float, required=True)
|
||||
parser.add_argument("--requests", type=int, default=64)
|
||||
parser.add_argument("--input-tokens", type=int, default=2048)
|
||||
parser.add_argument("--output-tokens", type=int, default=128)
|
||||
parser.add_argument("--timeout-seconds", type=float, default=900.0)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def percentile(values: list[float], fraction: float) -> float | None:
|
||||
if not values:
|
||||
return None
|
||||
ordered = sorted(values)
|
||||
index = min(len(ordered) - 1, max(0, math.ceil(fraction * len(ordered)) - 1))
|
||||
return ordered[index]
|
||||
|
||||
|
||||
def run_request(
|
||||
*,
|
||||
request_index: int,
|
||||
scheduled_at: float,
|
||||
benchmark_start: float,
|
||||
args: argparse.Namespace,
|
||||
prompt_ids: list[int],
|
||||
) -> dict[str, Any]:
|
||||
delay = scheduled_at - time.perf_counter()
|
||||
if delay > 0:
|
||||
time.sleep(delay)
|
||||
admitted = time.perf_counter()
|
||||
record: dict[str, Any] = {
|
||||
"request_index": request_index,
|
||||
"scheduled_s": scheduled_at - benchmark_start,
|
||||
"admitted_s": admitted - benchmark_start,
|
||||
"admission_lag_ms": (admitted - scheduled_at) * 1000.0,
|
||||
"success": False,
|
||||
}
|
||||
connection = http.client.HTTPConnection(args.host, args.port, timeout=args.timeout_seconds)
|
||||
body = {
|
||||
"model": args.served_model,
|
||||
"prompt": prompt_ids,
|
||||
"min_tokens": args.output_tokens,
|
||||
"max_tokens": args.output_tokens,
|
||||
"ignore_eos": True,
|
||||
"temperature": 0,
|
||||
"stream": True,
|
||||
"stream_options": {"include_usage": True},
|
||||
"return_token_ids": True,
|
||||
}
|
||||
try:
|
||||
started = time.perf_counter()
|
||||
connection.request(
|
||||
"POST",
|
||||
"/v1/completions",
|
||||
body=json.dumps(body, separators=(",", ":")).encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
response = connection.getresponse()
|
||||
if response.status != 200:
|
||||
raise RuntimeError(f"HTTP {response.status}: {response.read().decode(errors='replace')}")
|
||||
first_token_at = None
|
||||
last_token_at = None
|
||||
streamed_tokens = 0
|
||||
usage = None
|
||||
while True:
|
||||
raw = response.readline()
|
||||
if not raw:
|
||||
break
|
||||
line = raw.decode(errors="replace").strip()
|
||||
if not line.startswith("data:"):
|
||||
continue
|
||||
data = line[5:].strip()
|
||||
if data == "[DONE]":
|
||||
break
|
||||
payload = json.loads(data)
|
||||
if payload.get("usage"):
|
||||
usage = payload["usage"]
|
||||
emitted = 0
|
||||
for choice in payload.get("choices") or []:
|
||||
token_ids = choice.get("token_ids") or []
|
||||
emitted += len(token_ids) if token_ids else int(bool(choice.get("text")))
|
||||
if emitted:
|
||||
now = time.perf_counter()
|
||||
first_token_at = first_token_at or now
|
||||
last_token_at = now
|
||||
streamed_tokens += emitted
|
||||
finished = time.perf_counter()
|
||||
if first_token_at is None or last_token_at is None or usage is None:
|
||||
raise RuntimeError("missing streaming tokens or usage")
|
||||
prompt_tokens = int(usage["prompt_tokens"])
|
||||
completion_tokens = int(usage["completion_tokens"])
|
||||
if prompt_tokens != args.input_tokens or completion_tokens != args.output_tokens:
|
||||
raise RuntimeError(f"usage mismatch: {prompt_tokens}+{completion_tokens}")
|
||||
ttft = (first_token_at - started) * 1000.0
|
||||
tpot = (last_token_at - first_token_at) * 1000.0 / (completion_tokens - 1)
|
||||
record.update(
|
||||
{
|
||||
"success": True,
|
||||
"prompt_tokens": prompt_tokens,
|
||||
"completion_tokens": completion_tokens,
|
||||
"streamed_token_count": streamed_tokens,
|
||||
"ttft_ms": ttft,
|
||||
"tpot_ms": tpot,
|
||||
"e2e_ms": (finished - started) * 1000.0,
|
||||
}
|
||||
)
|
||||
except Exception as error: # Preserve failed requests as SLO failures.
|
||||
record["error"] = f"{type(error).__name__}: {error}"
|
||||
finally:
|
||||
connection.close()
|
||||
return record
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
if args.rate <= 0 or args.requests <= 0:
|
||||
raise ValueError("rate and requests must be positive")
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(args.model_path, trust_remote_code=True)
|
||||
excluded = set(tokenizer.all_special_ids)
|
||||
candidates = [token_id for token_id in range(tokenizer.vocab_size) if token_id not in excluded]
|
||||
if len(candidates) < args.requests + 1:
|
||||
raise RuntimeError("tokenizer has too few non-special token IDs")
|
||||
base_id = candidates[0]
|
||||
prompts = [
|
||||
[candidates[index + 1], *([base_id] * (args.input_tokens - 1))]
|
||||
for index in range(args.requests)
|
||||
]
|
||||
prompt_hash = hashlib.sha256(
|
||||
"\n".join(",".join(map(str, prompt)) for prompt in prompts).encode()
|
||||
).hexdigest()
|
||||
|
||||
benchmark_start = time.perf_counter() + 2.0
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=args.requests) as pool:
|
||||
futures = [
|
||||
pool.submit(
|
||||
run_request,
|
||||
request_index=index,
|
||||
scheduled_at=benchmark_start + index / args.rate,
|
||||
benchmark_start=benchmark_start,
|
||||
args=args,
|
||||
prompt_ids=prompts[index],
|
||||
)
|
||||
for index in range(args.requests)
|
||||
]
|
||||
requests = [future.result() for future in futures]
|
||||
requests.sort(key=lambda row: int(row["request_index"]))
|
||||
completed = [row for row in requests if row["success"]]
|
||||
slos = {}
|
||||
for limit in TPOT_SLOS_MS:
|
||||
passed = sum(
|
||||
row["success"]
|
||||
and float(row["ttft_ms"]) <= TTFT_SLO_MS
|
||||
and float(row["tpot_ms"]) <= limit
|
||||
for row in requests
|
||||
)
|
||||
slos[f"tpot_{int(limit)}ms"] = {
|
||||
"passed": passed,
|
||||
"pass_rate": passed / len(requests),
|
||||
"feasible": passed / len(requests) >= 0.95,
|
||||
}
|
||||
ttfts = [float(row["ttft_ms"]) for row in completed]
|
||||
tpots = [float(row["tpot_ms"]) for row in completed]
|
||||
payload = {
|
||||
"schema": "qwen235b-t0-rate-anchor-v1",
|
||||
"workload": {
|
||||
"offered_request_rate": args.rate,
|
||||
"request_count": args.requests,
|
||||
"input_tokens": args.input_tokens,
|
||||
"output_tokens": args.output_tokens,
|
||||
"prefix_caching": False,
|
||||
"arrival": "open_loop_uniform",
|
||||
"last_scheduled_arrival_s": (args.requests - 1) / args.rate,
|
||||
"prompt_vector_sha256": prompt_hash,
|
||||
},
|
||||
"summary": {
|
||||
"completed": len(completed),
|
||||
"failed": len(requests) - len(completed),
|
||||
"ttft_p50_ms": percentile(ttfts, 0.50),
|
||||
"ttft_p95_ms": percentile(ttfts, 0.95),
|
||||
"tpot_p50_ms": percentile(tpots, 0.50),
|
||||
"tpot_p95_ms": percentile(tpots, 0.95),
|
||||
"admission_lag_max_ms": max(float(row["admission_lag_ms"]) for row in requests),
|
||||
"slos": slos,
|
||||
},
|
||||
"requests": requests,
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
print(json.dumps(payload["summary"], sort_keys=True), flush=True)
|
||||
if len(completed) != args.requests:
|
||||
raise SystemExit(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
200
runs/frontier-multicase-sufficiency-v1/t0_smoke_client.py
Normal file
200
runs/frontier-multicase-sufficiency-v1/t0_smoke_client.py
Normal file
@@ -0,0 +1,200 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Issue exact fixed-shape completion requests and record streaming latency."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import http.client
|
||||
import json
|
||||
import statistics
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, required=True)
|
||||
parser.add_argument("--served-model", required=True)
|
||||
parser.add_argument("--model-path", type=Path, required=True)
|
||||
parser.add_argument("--input-tokens", type=int, default=2048)
|
||||
parser.add_argument("--output-tokens", type=int, default=128)
|
||||
parser.add_argument("--concurrency", type=int, required=True)
|
||||
parser.add_argument("--requests", type=int, required=True)
|
||||
parser.add_argument("--timeout-seconds", type=float, default=600.0)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def run_request(
|
||||
*,
|
||||
request_index: int,
|
||||
args: argparse.Namespace,
|
||||
prompt_token_id: int,
|
||||
start_barrier: threading.Barrier,
|
||||
) -> dict[str, Any]:
|
||||
body = {
|
||||
"model": args.served_model,
|
||||
"prompt": [prompt_token_id] * args.input_tokens,
|
||||
"min_tokens": args.output_tokens,
|
||||
"max_tokens": args.output_tokens,
|
||||
"ignore_eos": True,
|
||||
"temperature": 0,
|
||||
"stream": True,
|
||||
"stream_options": {"include_usage": True},
|
||||
"return_token_ids": True,
|
||||
}
|
||||
encoded = json.dumps(body, separators=(",", ":")).encode()
|
||||
connection = http.client.HTTPConnection(
|
||||
args.host, args.port, timeout=args.timeout_seconds
|
||||
)
|
||||
start_barrier.wait()
|
||||
started = time.perf_counter()
|
||||
connection.request(
|
||||
"POST",
|
||||
"/v1/completions",
|
||||
body=encoded,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
response = connection.getresponse()
|
||||
if response.status != 200:
|
||||
detail = response.read().decode(errors="replace")
|
||||
raise RuntimeError(f"request {request_index} failed: HTTP {response.status}: {detail}")
|
||||
|
||||
first_token_at: float | None = None
|
||||
last_token_at: float | None = None
|
||||
streamed_token_count = 0
|
||||
usage: dict[str, Any] | None = None
|
||||
while True:
|
||||
raw = response.readline()
|
||||
if not raw:
|
||||
break
|
||||
line = raw.decode(errors="replace").strip()
|
||||
if not line.startswith("data:"):
|
||||
continue
|
||||
data = line[5:].strip()
|
||||
if data == "[DONE]":
|
||||
break
|
||||
payload = json.loads(data)
|
||||
if payload.get("usage"):
|
||||
usage = payload["usage"]
|
||||
emitted = 0
|
||||
for choice in payload.get("choices") or []:
|
||||
token_ids = choice.get("token_ids") or []
|
||||
if token_ids:
|
||||
emitted += len(token_ids)
|
||||
elif choice.get("text"):
|
||||
emitted += 1
|
||||
if emitted:
|
||||
now = time.perf_counter()
|
||||
if first_token_at is None:
|
||||
first_token_at = now
|
||||
last_token_at = now
|
||||
streamed_token_count += emitted
|
||||
finished = time.perf_counter()
|
||||
connection.close()
|
||||
|
||||
if first_token_at is None or last_token_at is None or usage is None:
|
||||
raise RuntimeError(
|
||||
f"request {request_index} missing streaming token or usage metadata"
|
||||
)
|
||||
prompt_tokens = int(usage["prompt_tokens"])
|
||||
completion_tokens = int(usage["completion_tokens"])
|
||||
if prompt_tokens != args.input_tokens or completion_tokens != args.output_tokens:
|
||||
raise RuntimeError(
|
||||
f"request {request_index} usage mismatch: prompt={prompt_tokens}, "
|
||||
f"completion={completion_tokens}"
|
||||
)
|
||||
|
||||
ttft_ms = (first_token_at - started) * 1000.0
|
||||
tpot_ms = (
|
||||
(last_token_at - first_token_at) * 1000.0 / (completion_tokens - 1)
|
||||
if completion_tokens > 1
|
||||
else 0.0
|
||||
)
|
||||
return {
|
||||
"request_index": request_index,
|
||||
"prompt_tokens": prompt_tokens,
|
||||
"completion_tokens": completion_tokens,
|
||||
"streamed_token_count": streamed_token_count,
|
||||
"ttft_ms": ttft_ms,
|
||||
"tpot_ms": tpot_ms,
|
||||
"e2e_ms": (finished - started) * 1000.0,
|
||||
"ttft_slo_ms": 1000.0 + args.input_tokens / 8.0,
|
||||
"tpot_slo_ms": 40.0,
|
||||
"joint_slo_pass": ttft_ms <= 1000.0 + args.input_tokens / 8.0
|
||||
and tpot_ms <= 40.0,
|
||||
}
|
||||
|
||||
|
||||
def percentile(values: list[float], fraction: float) -> float:
|
||||
ordered = sorted(values)
|
||||
index = min(len(ordered) - 1, max(0, round((len(ordered) - 1) * fraction)))
|
||||
return ordered[index]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
if args.concurrency <= 0 or args.requests < args.concurrency:
|
||||
raise ValueError("requests must be at least concurrency, and both must be positive")
|
||||
if args.input_tokens <= 0 or args.output_tokens <= 0:
|
||||
raise ValueError("token lengths must be positive")
|
||||
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(args.model_path, trust_remote_code=True)
|
||||
candidate_ids = tokenizer.encode(" hello", add_special_tokens=False)
|
||||
if not candidate_ids:
|
||||
raise RuntimeError("tokenizer returned no prompt token id")
|
||||
prompt_token_id = int(candidate_ids[0])
|
||||
|
||||
results: list[dict[str, Any]] = []
|
||||
for batch_start in range(0, args.requests, args.concurrency):
|
||||
batch_count = min(args.concurrency, args.requests - batch_start)
|
||||
barrier = threading.Barrier(batch_count)
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=batch_count) as pool:
|
||||
futures = [
|
||||
pool.submit(
|
||||
run_request,
|
||||
request_index=batch_start + offset,
|
||||
args=args,
|
||||
prompt_token_id=prompt_token_id,
|
||||
start_barrier=barrier,
|
||||
)
|
||||
for offset in range(batch_count)
|
||||
]
|
||||
results.extend(future.result() for future in futures)
|
||||
|
||||
ttfts = [float(row["ttft_ms"]) for row in results]
|
||||
tpots = [float(row["tpot_ms"]) for row in results]
|
||||
payload = {
|
||||
"schema": "qwen235b-t0-smoke-v1",
|
||||
"workload": {
|
||||
"input_tokens": args.input_tokens,
|
||||
"output_tokens": args.output_tokens,
|
||||
"uniform_qps": None,
|
||||
"prefix_caching": False,
|
||||
"concurrency": args.concurrency,
|
||||
"request_count": args.requests,
|
||||
"prompt_token_id": prompt_token_id,
|
||||
},
|
||||
"summary": {
|
||||
"completed_requests": len(results),
|
||||
"joint_slo_pass_count": sum(bool(row["joint_slo_pass"]) for row in results),
|
||||
"ttft_mean_ms": statistics.fmean(ttfts),
|
||||
"ttft_p95_ms": percentile(ttfts, 0.95),
|
||||
"tpot_mean_ms": statistics.fmean(tpots),
|
||||
"tpot_p95_ms": percentile(tpots, 0.95),
|
||||
},
|
||||
"requests": sorted(results, key=lambda row: int(row["request_index"])),
|
||||
}
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
|
||||
print(json.dumps(payload["summary"], sort_keys=True), flush=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"schema": "qwen235b-trace-contract-audit-v1",
|
||||
"status": "pass_offline_source_contract",
|
||||
"recorded_at": "2026-07-16",
|
||||
"execution": {
|
||||
"host": "dash0",
|
||||
"device": "cpu_only",
|
||||
"elapsed_seconds": 45.643,
|
||||
"tokenizer_class": "Qwen2TokenizerFast",
|
||||
"transformers_version": "4.55.2",
|
||||
"model_path": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8"
|
||||
},
|
||||
"trace": {
|
||||
"path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/traces/thinking_w20260327_1000.jsonl",
|
||||
"sha256": "f878e9af18f94dcfaced94a8e1e6b20a2f7d97d64aa862448025660dbbd965b2",
|
||||
"source_request_count": 15479,
|
||||
"eligible_request_count": 15401,
|
||||
"eligibility": "input_length + output_length <= 40960 and output_length > 0"
|
||||
},
|
||||
"tokenization": {
|
||||
"eligible_request_count": 15401,
|
||||
"total_token_count": 55057919,
|
||||
"input_length_mismatch_count": 0,
|
||||
"length_order_sha256": "8bb1c9b7278b261fe1be695ff16dd0c4735d63bd0b1830b612953961fefc370a",
|
||||
"per_request_token_digest_sha256": "b15b01c00042813c76701ffcc75fc0facde3c01806e167afdc69b5b5861e2381"
|
||||
},
|
||||
"source_hash_contract": {
|
||||
"source_block_size_tokens": 64,
|
||||
"full_block_count": 852407,
|
||||
"partial_block_count": 15131,
|
||||
"unique_hash_id_count": 509437,
|
||||
"unique_parent_chunk_key_count": 509437,
|
||||
"hash_id_to_parent_chunk_conflict_count": 0,
|
||||
"parent_chunk_to_hash_id_conflict_count": 0,
|
||||
"key_definition": "(parent source hash id, BLAKE2b-128 of the tokenizer token-id chunk)"
|
||||
},
|
||||
"interpretation": {
|
||||
"established": "The exact prompts reproduce input_length, and source hash ids preserve the tokenizer-visible parent/chunk prefix-equivalence relation over the eligible universe.",
|
||||
"not_yet_established": "Runtime parity of vLLM and Frontier block-size-16 computed, hit, allocated, eviction, and placement counters."
|
||||
}
|
||||
}
|
||||
653
runs/frontier-slo-alignment-v0/analyze.py
Normal file
653
runs/frontier-slo-alignment-v0/analyze.py
Normal file
@@ -0,0 +1,653 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Re-evaluate Frontier and real replay with one SLO-feasible throughput objective.
|
||||
|
||||
The analysis is deliberately paired: a cell's capacity is the largest offered
|
||||
load that satisfies the same request-level SLO among the anchors observed by
|
||||
both systems. It does not extrapolate beyond the common anchor grid.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import subprocess
|
||||
import sys
|
||||
from collections import Counter
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterable
|
||||
|
||||
|
||||
SCHEMA = "frontier-slo-alignment-v0"
|
||||
MODE = "frozen-calibrated"
|
||||
READING = "paired-grid-slo-feasible-max-offered-throughput"
|
||||
EXPECTED_CELLS = {
|
||||
f"tp{tp}_mns{mns}" for tp in (1, 2, 4) for mns in (8, 16, 32, 64)
|
||||
}
|
||||
|
||||
|
||||
def load_json(path: Path) -> dict[str, Any]:
|
||||
with path.open(encoding="utf-8") as source:
|
||||
value = json.load(source)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError(f"expected JSON object: {path}")
|
||||
return value
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
for chunk in iter(lambda: source.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def git_revision(path: Path) -> dict[str, str]:
|
||||
def run(*arguments: str) -> str:
|
||||
return subprocess.run(
|
||||
["git", *arguments],
|
||||
cwd=path,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
|
||||
return {"head": run("rev-parse", "HEAD"), "status_short": run("status", "--short")}
|
||||
|
||||
|
||||
def load_real_cells(ground_truth_path: Path) -> dict[str, dict[str, Any]]:
|
||||
ground_truth = load_json(ground_truth_path)
|
||||
cells = {}
|
||||
for cell in ground_truth["cells"]:
|
||||
cell_id = str(cell["cell_id"])
|
||||
if cell_id in cells:
|
||||
raise ValueError(f"duplicate real cell: {cell_id}")
|
||||
probes = {int(probe["probe_index"]): probe for probe in cell["probe_history"]}
|
||||
if len(probes) != len(cell["probe_history"]):
|
||||
raise ValueError(f"duplicate real probe index: {cell_id}")
|
||||
cells[cell_id] = {**cell, "probes": probes}
|
||||
if set(cells) != EXPECTED_CELLS:
|
||||
raise ValueError(f"unexpected real cells: {sorted(cells)}")
|
||||
return cells
|
||||
|
||||
|
||||
def load_frontier_runs(results_dir: Path, mode: str = MODE) -> dict[tuple[str, int], dict[str, Any]]:
|
||||
runs: dict[tuple[str, int], dict[str, Any]] = {}
|
||||
manifests = sorted((results_dir / "raw").glob("*/trial-*/run_manifest.json"))
|
||||
for manifest_path in manifests:
|
||||
manifest = load_json(manifest_path)
|
||||
run = manifest["run"]
|
||||
if run["mode"] != mode:
|
||||
continue
|
||||
status_path = manifest_path.with_name("trial_status.json")
|
||||
if not status_path.exists():
|
||||
raise ValueError(f"missing trial status: {status_path}")
|
||||
status = load_json(status_path)
|
||||
if status["status"] != "pass":
|
||||
continue
|
||||
scorer_path = manifest_path.with_name("scorer_output.json")
|
||||
if not scorer_path.exists():
|
||||
raise ValueError(f"missing scorer output: {scorer_path}")
|
||||
scorer = load_json(scorer_path)
|
||||
key = (str(run["cell_id"]), int(run["probe_index"]))
|
||||
if key in runs:
|
||||
raise ValueError(f"duplicate Frontier run: {key}")
|
||||
if int(scorer["total_requests"]) != int(run["request_count"]):
|
||||
raise ValueError(f"request count mismatch: {key}")
|
||||
runs[key] = {
|
||||
"manifest_path": str(manifest_path),
|
||||
"manifest": manifest,
|
||||
"scorer_path": str(scorer_path),
|
||||
"scorer": scorer,
|
||||
}
|
||||
return runs
|
||||
|
||||
|
||||
def build_anchor_records(
|
||||
real_cells: dict[str, dict[str, Any]],
|
||||
frontier_runs: dict[tuple[str, int], dict[str, Any]],
|
||||
) -> list[dict[str, Any]]:
|
||||
expected = {
|
||||
(cell_id, probe_index)
|
||||
for cell_id, cell in real_cells.items()
|
||||
for probe_index in cell["probes"]
|
||||
}
|
||||
if set(frontier_runs) != expected:
|
||||
missing = sorted(expected - set(frontier_runs))
|
||||
extra = sorted(set(frontier_runs) - expected)
|
||||
raise ValueError(f"Frontier/real anchor mismatch; missing={missing}, extra={extra}")
|
||||
|
||||
anchors = []
|
||||
for cell_id, probe_index in sorted(expected):
|
||||
cell = real_cells[cell_id]
|
||||
probe = cell["probes"][probe_index]
|
||||
frontier = frontier_runs[(cell_id, probe_index)]
|
||||
run = frontier["manifest"]["run"]
|
||||
scorer = frontier["scorer"]
|
||||
if int(run["request_count"]) != int(probe["request_count"]):
|
||||
raise ValueError(f"paired request count mismatch: {(cell_id, probe_index)}")
|
||||
if not math.isclose(
|
||||
float(run["sampling_u"]), float(probe["sampling_u"]), rel_tol=0.0, abs_tol=1e-15
|
||||
):
|
||||
raise ValueError(f"paired sampling_u mismatch: {(cell_id, probe_index)}")
|
||||
rate = float(probe["request_rate_per_gpu_req_s_gpu"])
|
||||
anchors.append(
|
||||
{
|
||||
"cell_id": cell_id,
|
||||
"tp": int(cell["tensor_parallel_size"]),
|
||||
"mns": int(cell["max_num_seqs"]),
|
||||
"probe_index": probe_index,
|
||||
"sampling_u": float(probe["sampling_u"]),
|
||||
"request_count": int(probe["request_count"]),
|
||||
"offered_req_s_per_gpu": rate,
|
||||
"real_feasible": bool(probe["feasible"]),
|
||||
"real_pass_rate": float(probe["pass_rate"]),
|
||||
"frontier_feasible": bool(scorer["slo"]["feasible"]),
|
||||
"frontier_pass_rate": float(scorer["slo"]["pass_rate"]),
|
||||
"frontier_completed_req_s_per_gpu": float(
|
||||
scorer["throughput_requests_per_second_per_gpu"]
|
||||
),
|
||||
}
|
||||
)
|
||||
return anchors
|
||||
|
||||
|
||||
def monotonic_violations(rows: list[dict[str, Any]], field: str) -> list[dict[str, Any]]:
|
||||
ordered = sorted(rows, key=lambda row: (row["offered_req_s_per_gpu"], row["probe_index"]))
|
||||
violations = []
|
||||
for lower_index, lower in enumerate(ordered):
|
||||
for upper in ordered[lower_index + 1 :]:
|
||||
if (
|
||||
lower["offered_req_s_per_gpu"] < upper["offered_req_s_per_gpu"]
|
||||
and not lower[field]
|
||||
and upper[field]
|
||||
):
|
||||
violations.append(
|
||||
{
|
||||
"lower_probe": lower["probe_index"],
|
||||
"lower_rate": lower["offered_req_s_per_gpu"],
|
||||
"upper_probe": upper["probe_index"],
|
||||
"upper_rate": upper["offered_req_s_per_gpu"],
|
||||
}
|
||||
)
|
||||
return violations
|
||||
|
||||
|
||||
def selected_anchor(rows: list[dict[str, Any]], field: str) -> dict[str, Any] | None:
|
||||
feasible = [row for row in rows if row[field]]
|
||||
if not feasible:
|
||||
return None
|
||||
return max(feasible, key=lambda row: (row["offered_req_s_per_gpu"], -row["probe_index"]))
|
||||
|
||||
|
||||
def summarize_cells(anchors: list[dict[str, Any]]) -> dict[str, dict[str, Any]]:
|
||||
grouped = {cell: [] for cell in sorted(EXPECTED_CELLS)}
|
||||
for row in anchors:
|
||||
grouped[row["cell_id"]].append(row)
|
||||
|
||||
summaries = {}
|
||||
for cell_id, rows in grouped.items():
|
||||
real_selected = selected_anchor(rows, "real_feasible")
|
||||
frontier_selected = selected_anchor(rows, "frontier_feasible")
|
||||
real_score = real_selected["offered_req_s_per_gpu"] if real_selected else 0.0
|
||||
frontier_score = frontier_selected["offered_req_s_per_gpu"] if frontier_selected else 0.0
|
||||
max_rate = max(row["offered_req_s_per_gpu"] for row in rows)
|
||||
real_violations = monotonic_violations(rows, "real_feasible")
|
||||
frontier_violations = monotonic_violations(rows, "frontier_feasible")
|
||||
summaries[cell_id] = {
|
||||
"tp": rows[0]["tp"],
|
||||
"mns": rows[0]["mns"],
|
||||
"anchor_count": len(rows),
|
||||
"max_common_anchor_req_s_per_gpu": max_rate,
|
||||
"real_feasible_anchor_count": sum(bool(row["real_feasible"]) for row in rows),
|
||||
"frontier_feasible_anchor_count": sum(
|
||||
bool(row["frontier_feasible"]) for row in rows
|
||||
),
|
||||
"real_score": real_score,
|
||||
"frontier_score": frontier_score,
|
||||
"absolute_error": frontier_score - real_score,
|
||||
"relative_error": (frontier_score - real_score) / real_score if real_score else None,
|
||||
"real_selected_probe": real_selected["probe_index"] if real_selected else None,
|
||||
"frontier_selected_probe": frontier_selected["probe_index"] if frontier_selected else None,
|
||||
"frontier_right_censored": bool(
|
||||
frontier_selected and math.isclose(frontier_score, max_rate, abs_tol=1e-15)
|
||||
),
|
||||
"frontier_boundary_bracketed": bool(
|
||||
frontier_selected
|
||||
and any(
|
||||
not row["frontier_feasible"]
|
||||
and row["offered_req_s_per_gpu"] > frontier_score
|
||||
for row in rows
|
||||
)
|
||||
),
|
||||
"real_monotonic_violation_count": len(real_violations),
|
||||
"frontier_monotonic_violation_count": len(frontier_violations),
|
||||
"real_monotonic_violations": real_violations,
|
||||
"frontier_monotonic_violations": frontier_violations,
|
||||
}
|
||||
return summaries
|
||||
|
||||
|
||||
def score_buckets(scores: dict[str, float]) -> tuple[float, dict[str, int]]:
|
||||
tolerance = max(1e-9, 1e-6 * max(abs(value) for value in scores.values()))
|
||||
return tolerance, {cell: math.floor(value / tolerance) for cell, value in scores.items()}
|
||||
|
||||
|
||||
def sign(value: int) -> int:
|
||||
return (value > 0) - (value < 0)
|
||||
|
||||
|
||||
def ranking_metrics(
|
||||
real_scores: dict[str, float], frontier_scores: dict[str, float]
|
||||
) -> dict[str, Any]:
|
||||
if set(real_scores) != set(frontier_scores):
|
||||
raise ValueError("ranking score cells differ")
|
||||
real_tolerance, real_buckets = score_buckets(real_scores)
|
||||
frontier_tolerance, frontier_buckets = score_buckets(frontier_scores)
|
||||
cells = sorted(real_scores)
|
||||
counts = Counter()
|
||||
exact = 0
|
||||
for left_index, left in enumerate(cells):
|
||||
for right in cells[left_index + 1 :]:
|
||||
real_sign = sign(real_buckets[left] - real_buckets[right])
|
||||
frontier_sign = sign(frontier_buckets[left] - frontier_buckets[right])
|
||||
exact += int(real_sign == frontier_sign)
|
||||
if real_sign == 0 and frontier_sign == 0:
|
||||
counts["both_tied"] += 1
|
||||
elif real_sign == 0:
|
||||
counts["real_only_tied"] += 1
|
||||
elif frontier_sign == 0:
|
||||
counts["frontier_only_tied"] += 1
|
||||
elif real_sign == frontier_sign:
|
||||
counts["concordant"] += 1
|
||||
else:
|
||||
counts["discordant"] += 1
|
||||
numerator = counts["concordant"] - counts["discordant"]
|
||||
denominator = math.sqrt(
|
||||
(counts["concordant"] + counts["discordant"] + counts["real_only_tied"])
|
||||
* (
|
||||
counts["concordant"]
|
||||
+ counts["discordant"]
|
||||
+ counts["frontier_only_tied"]
|
||||
)
|
||||
)
|
||||
tau = numerator / denominator if denominator else 0.0
|
||||
top_bucket = max(frontier_buckets.values())
|
||||
candidates = sorted(cell for cell, bucket in frontier_buckets.items() if bucket == top_bucket)
|
||||
real_best = max(real_scores.values())
|
||||
regrets = [(real_best - real_scores[cell]) / real_best for cell in candidates]
|
||||
return {
|
||||
"real_tolerance": real_tolerance,
|
||||
"frontier_tolerance": frontier_tolerance,
|
||||
"top1_candidate_cells": candidates,
|
||||
"top1_effective_k": len(candidates),
|
||||
"top1_optimistic_regret": min(regrets),
|
||||
"top1_worst_case_regret": max(regrets),
|
||||
"pair_count": len(cells) * (len(cells) - 1) // 2,
|
||||
"pairwise_exact_sign_accuracy": exact / (len(cells) * (len(cells) - 1) // 2),
|
||||
"kendall_tau_b": tau,
|
||||
"kendall_counts": dict(counts),
|
||||
}
|
||||
|
||||
|
||||
def confusion_metrics(anchors: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
counts = Counter()
|
||||
for row in anchors:
|
||||
real = bool(row["real_feasible"])
|
||||
frontier = bool(row["frontier_feasible"])
|
||||
label = (
|
||||
"true_feasible"
|
||||
if real and frontier
|
||||
else "true_infeasible"
|
||||
if not real and not frontier
|
||||
else "false_feasible"
|
||||
if not real and frontier
|
||||
else "false_infeasible"
|
||||
)
|
||||
counts[label] += 1
|
||||
total = len(anchors)
|
||||
positive = counts["true_feasible"] + counts["false_infeasible"]
|
||||
negative = counts["true_infeasible"] + counts["false_feasible"]
|
||||
return {
|
||||
**{name: counts[name] for name in (
|
||||
"true_feasible",
|
||||
"true_infeasible",
|
||||
"false_feasible",
|
||||
"false_infeasible",
|
||||
)},
|
||||
"total": total,
|
||||
"accuracy": (counts["true_feasible"] + counts["true_infeasible"]) / total,
|
||||
"feasible_recall": counts["true_feasible"] / positive if positive else None,
|
||||
"infeasible_recall": counts["true_infeasible"] / negative if negative else None,
|
||||
"false_feasible_rate": counts["false_feasible"] / negative if negative else None,
|
||||
"false_infeasible_rate": counts["false_infeasible"] / positive if positive else None,
|
||||
}
|
||||
|
||||
|
||||
def error_metrics(cell_summaries: dict[str, dict[str, Any]]) -> dict[str, float]:
|
||||
errors = [row["absolute_error"] for row in cell_summaries.values()]
|
||||
relative = [abs(row["relative_error"]) for row in cell_summaries.values()]
|
||||
return {
|
||||
"mean_absolute_error_req_s_per_gpu": sum(abs(value) for value in errors) / len(errors),
|
||||
"root_mean_square_error_req_s_per_gpu": math.sqrt(
|
||||
sum(value * value for value in errors) / len(errors)
|
||||
),
|
||||
"mean_absolute_percentage_error": sum(relative) / len(relative),
|
||||
"right_censored_cell_count": sum(
|
||||
bool(row["frontier_right_censored"]) for row in cell_summaries.values()
|
||||
),
|
||||
"bracketed_cell_count": sum(
|
||||
bool(row["frontier_boundary_bracketed"]) for row in cell_summaries.values()
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def grouped_error_metrics(
|
||||
cell_summaries: dict[str, dict[str, Any]], field: str
|
||||
) -> dict[str, dict[str, float]]:
|
||||
groups: dict[int, list[dict[str, Any]]] = {}
|
||||
for row in cell_summaries.values():
|
||||
groups.setdefault(int(row[field]), []).append(row)
|
||||
output = {}
|
||||
for key, rows in sorted(groups.items()):
|
||||
output[str(key)] = {
|
||||
"cell_count": len(rows),
|
||||
"mean_signed_error_req_s_per_gpu": sum(row["absolute_error"] for row in rows)
|
||||
/ len(rows),
|
||||
"mean_absolute_error_req_s_per_gpu": sum(
|
||||
abs(row["absolute_error"]) for row in rows
|
||||
)
|
||||
/ len(rows),
|
||||
"right_censored_cell_count": sum(bool(row["frontier_right_censored"]) for row in rows),
|
||||
}
|
||||
return output
|
||||
|
||||
|
||||
def old_proxy_summary(previous_metrics_path: Path) -> dict[str, Any]:
|
||||
previous = load_json(previous_metrics_path)
|
||||
analysis = previous["analyses"]["frozen-calibrated/throughput-proxy"]
|
||||
metrics = analysis["metrics"]
|
||||
return {
|
||||
"reading": "completed-throughput proxy (old, not SLO gated)",
|
||||
"top1_candidate_cells": metrics["top1"]["candidate_cells"],
|
||||
"top1_worst_case_regret": metrics["top1"]["worst_case_regret"],
|
||||
"kendall_tau_b": metrics["kendall_tau_b"]["tau_b"],
|
||||
"pairwise_exact_sign_accuracy": metrics["pairwise_direction"]["exact_sign_accuracy"],
|
||||
}
|
||||
|
||||
|
||||
def validate_previous_slo_reading(
|
||||
previous_metrics_path: Path,
|
||||
real_scores: dict[str, float],
|
||||
frontier_scores: dict[str, float],
|
||||
) -> dict[str, Any]:
|
||||
previous = load_json(previous_metrics_path)
|
||||
previous_real = {cell: float(value) for cell, value in previous["real_scores"].items()}
|
||||
previous_frontier = {
|
||||
cell: float(value)
|
||||
for cell, value in previous["analyses"]["frozen-calibrated/SLO-gated"][
|
||||
"simulated_scores"
|
||||
].items()
|
||||
}
|
||||
real_difference = max(abs(real_scores[cell] - previous_real[cell]) for cell in real_scores)
|
||||
frontier_difference = max(
|
||||
abs(frontier_scores[cell] - previous_frontier[cell]) for cell in frontier_scores
|
||||
)
|
||||
if real_difference > 1e-12 or frontier_difference > 1e-12:
|
||||
raise ValueError(
|
||||
"independent aligned reconstruction disagrees with previous secondary reading: "
|
||||
f"real={real_difference}, Frontier={frontier_difference}"
|
||||
)
|
||||
return {
|
||||
"status": "PASS",
|
||||
"previous_reading": "frozen-calibrated/SLO-gated",
|
||||
"maximum_real_score_absolute_difference": real_difference,
|
||||
"maximum_frontier_score_absolute_difference": frontier_difference,
|
||||
}
|
||||
|
||||
|
||||
def write_csv(path: Path, fieldnames: list[str], rows: Iterable[dict[str, Any]]) -> None:
|
||||
with path.open("w", encoding="utf-8", newline="") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
writer.writerows({field: row.get(field) for field in fieldnames} for row in rows)
|
||||
|
||||
|
||||
def render_report(metrics: dict[str, Any]) -> str:
|
||||
rank = metrics["aligned_ranking"]
|
||||
confusion = metrics["anchor_feasibility"]
|
||||
error = metrics["score_error"]
|
||||
grouped = metrics["grouped_score_error"]
|
||||
old = metrics["objective_comparison"]["old_proxy"]
|
||||
cells = metrics["cells"]
|
||||
lines = [
|
||||
"# Frontier SLO-aligned retrospective experiment",
|
||||
"",
|
||||
"## Result",
|
||||
"",
|
||||
(
|
||||
"After aligning both systems to the same paired-grid objective—maximum offered "
|
||||
"req/s/GPU with request-level SLO pass rate >= 0.95—the old 30.46% top-1 "
|
||||
"regret does not reproduce. Frontier nominates `"
|
||||
+ "`, `".join(rank["top1_candidate_cells"])
|
||||
+ f"` (tie), with real-evaluated regret {100 * rank['top1_optimistic_regret']:.3f}%"
|
||||
+ f"--{100 * rank['top1_worst_case_regret']:.3f}%."
|
||||
),
|
||||
"",
|
||||
(
|
||||
f"Ranking agreement is high on this fixed candidate grid: Kendall tau-b "
|
||||
f"{rank['kendall_tau_b']:.4f}, pairwise exact-sign accuracy "
|
||||
f"{100 * rank['pairwise_exact_sign_accuracy']:.2f}%. This is materially different "
|
||||
f"from the old completed-throughput proxy (tau-b {old['kendall_tau_b']:.4f}, "
|
||||
f"top-1 regret {100 * old['top1_worst_case_regret']:.2f}%)."
|
||||
),
|
||||
"",
|
||||
"## Per-cell paired-grid capacity",
|
||||
"",
|
||||
"| Cell | Real | Frontier | Error | Frontier boundary | Real monotonic violations |",
|
||||
"|---|---:|---:|---:|---|---:|",
|
||||
]
|
||||
for cell_id in sorted(cells, key=lambda cell: (cells[cell]["tp"], cells[cell]["mns"])):
|
||||
row = cells[cell_id]
|
||||
boundary = "right-censored" if row["frontier_right_censored"] else (
|
||||
"bracketed" if row["frontier_boundary_bracketed"] else "unresolved"
|
||||
)
|
||||
lines.append(
|
||||
f"| `{cell_id}` | {row['real_score']:.6f} | {row['frontier_score']:.6f} | "
|
||||
f"{row['absolute_error']:+.6f} | {boundary} | "
|
||||
f"{row['real_monotonic_violation_count']} |"
|
||||
)
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"The paired-grid score MAE is "
|
||||
f"{error['mean_absolute_error_req_s_per_gpu']:.4f} req/s/GPU and MAPE is "
|
||||
f"{100 * error['mean_absolute_percentage_error']:.2f}%. These error aggregates "
|
||||
"must be read as grid-clipped because "
|
||||
f"{int(error['right_censored_cell_count'])}/12 Frontier cells remain feasible at "
|
||||
"their highest common anchor.",
|
||||
"",
|
||||
"The error is configuration-dependent rather than a uniform scale offset. Mean "
|
||||
f"signed error is {grouped['by_mns']['8']['mean_signed_error_req_s_per_gpu']:+.4f} "
|
||||
"req/s/GPU for MNS=8, but rises to "
|
||||
f"{grouped['by_mns']['32']['mean_signed_error_req_s_per_gpu']:+.4f} and "
|
||||
f"{grouped['by_mns']['64']['mean_signed_error_req_s_per_gpu']:+.4f} for MNS=32/64. "
|
||||
"The aligned score therefore still exposes a missing or distorted MNS response.",
|
||||
"",
|
||||
"## Anchor-level feasibility",
|
||||
"",
|
||||
f"Across {confusion['total']} paired anchors: true-feasible="
|
||||
f"{confusion['true_feasible']}, true-infeasible={confusion['true_infeasible']}, "
|
||||
f"false-feasible={confusion['false_feasible']}, false-infeasible="
|
||||
f"{confusion['false_infeasible']}. Accuracy is {100 * confusion['accuracy']:.2f}%, "
|
||||
f"but the false-feasible rate among real-infeasible anchors is "
|
||||
f"{100 * confusion['false_feasible_rate']:.2f}%.",
|
||||
"",
|
||||
"Therefore the aligned experiment supports a narrower conclusion: Frontier can "
|
||||
"recover the top configuration family on this frozen, ragged candidate grid, but "
|
||||
"it is not yet a reliable SLO feasibility oracle. Good top-1 ranking is compatible "
|
||||
"with many wrong boundary labels.",
|
||||
"",
|
||||
"## Validity limits",
|
||||
"",
|
||||
"- This is a retrospective single-run analysis; no confidence interval is valid.",
|
||||
"- Real probes within a cell reused one vLLM process and prefix-cache history; "
|
||||
"Frontier anchors were independent runs. History-dependent real pass-rate values "
|
||||
"are retained rather than monotonicized; binary feasibility happens to remain "
|
||||
"monotone on the observed per-cell grids.",
|
||||
"- Right-censored cells expose only a lower bound on Frontier's own capacity. "
|
||||
"They are valid for paired-grid selection, not for claiming an exact simulator knee.",
|
||||
"- The frozen per-TP calibration and evaluation surface are not fully independent "
|
||||
"across workload families, so this is not a generalization result.",
|
||||
"",
|
||||
"## Required prospective experiment",
|
||||
"",
|
||||
"Repeat a small set of boundary cells with one engine restart per anchor, explicit "
|
||||
"cold-cache state, identical offered-load grid, and repeated trials. Extend the grid "
|
||||
"until both real and Frontier have at least one feasible and one infeasible point. "
|
||||
"Only that experiment can estimate boundary displacement and its uncertainty.",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def analyze(
|
||||
ground_truth_path: Path,
|
||||
results_dir: Path,
|
||||
previous_metrics_path: Path,
|
||||
repository_root: Path,
|
||||
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
real_cells = load_real_cells(ground_truth_path)
|
||||
frontier_runs = load_frontier_runs(results_dir)
|
||||
anchors = build_anchor_records(real_cells, frontier_runs)
|
||||
cells = summarize_cells(anchors)
|
||||
real_scores = {cell: row["real_score"] for cell, row in cells.items()}
|
||||
frontier_scores = {cell: row["frontier_score"] for cell, row in cells.items()}
|
||||
frontier_revisions = {
|
||||
run["manifest"]["frontier"]["git_head"] for run in frontier_runs.values()
|
||||
}
|
||||
if len(frontier_revisions) != 1:
|
||||
raise ValueError(f"mixed Frontier revisions: {sorted(frontier_revisions)}")
|
||||
metrics = {
|
||||
"schema_version": SCHEMA,
|
||||
"generated_utc": datetime.now(timezone.utc).isoformat(),
|
||||
"status": "VALID_RETROSPECTIVE_ALIGNED_ANALYSIS",
|
||||
"claim": (
|
||||
"On the frozen common anchor grid, rank configurations by maximum offered "
|
||||
"req/s/GPU whose request-level SLO pass rate is at least 0.95 in each system."
|
||||
),
|
||||
"estimand": READING,
|
||||
"inputs": {
|
||||
"ground_truth": {
|
||||
"path": str(ground_truth_path),
|
||||
"sha256": sha256_file(ground_truth_path),
|
||||
},
|
||||
"frontier_results": str(results_dir),
|
||||
"previous_metrics": {
|
||||
"path": str(previous_metrics_path),
|
||||
"sha256": sha256_file(previous_metrics_path),
|
||||
},
|
||||
},
|
||||
"provenance": {
|
||||
"analysis_repository": git_revision(repository_root),
|
||||
"frontier_git_head_recorded_by_runs": next(iter(frontier_revisions)),
|
||||
"frontier_mode": MODE,
|
||||
"real_engine_version": "vLLM 0.20.0",
|
||||
"anchor_count": len(anchors),
|
||||
"cell_count": len(cells),
|
||||
},
|
||||
"objective_comparison": {"old_proxy": old_proxy_summary(previous_metrics_path)},
|
||||
"aligned_ranking": ranking_metrics(real_scores, frontier_scores),
|
||||
"anchor_feasibility": confusion_metrics(anchors),
|
||||
"score_error": error_metrics(cells),
|
||||
"grouped_score_error": {
|
||||
"by_tp": grouped_error_metrics(cells, "tp"),
|
||||
"by_mns": grouped_error_metrics(cells, "mns"),
|
||||
},
|
||||
"independent_reconstruction_crosscheck": validate_previous_slo_reading(
|
||||
previous_metrics_path, real_scores, frontier_scores
|
||||
),
|
||||
"cells": cells,
|
||||
"statistics": {
|
||||
"real_trials_per_anchor": 1,
|
||||
"frontier_trials_per_anchor": 1,
|
||||
"confidence_intervals": None,
|
||||
"reason": "single retrospective observation per paired anchor",
|
||||
},
|
||||
"known_limits": [
|
||||
"real probes reused a process and prefix-cache history within each cell",
|
||||
"Frontier anchors were independent simulator runs",
|
||||
"right-censored cells do not identify Frontier's exact capacity knee",
|
||||
"the anchor grid is ragged across cells",
|
||||
],
|
||||
}
|
||||
return metrics, anchors
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--ground-truth", type=Path, required=True)
|
||||
parser.add_argument("--frontier-results", type=Path, required=True)
|
||||
parser.add_argument("--previous-metrics", type=Path, required=True)
|
||||
parser.add_argument("--output-dir", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
repository_root = Path(__file__).resolve().parents[2]
|
||||
metrics, anchors = analyze(
|
||||
args.ground_truth.resolve(),
|
||||
args.frontier_results.resolve(),
|
||||
args.previous_metrics.resolve(),
|
||||
repository_root,
|
||||
)
|
||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
metrics_path = args.output_dir / "metrics.json"
|
||||
metrics_path.write_text(json.dumps(metrics, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
write_csv(
|
||||
args.output_dir / "cell_scores.csv",
|
||||
[
|
||||
"cell_id",
|
||||
"tp",
|
||||
"mns",
|
||||
"anchor_count",
|
||||
"real_score",
|
||||
"frontier_score",
|
||||
"absolute_error",
|
||||
"relative_error",
|
||||
"frontier_right_censored",
|
||||
"frontier_boundary_bracketed",
|
||||
"real_monotonic_violation_count",
|
||||
"frontier_monotonic_violation_count",
|
||||
],
|
||||
({"cell_id": cell_id, **row} for cell_id, row in sorted(metrics["cells"].items())),
|
||||
)
|
||||
write_csv(
|
||||
args.output_dir / "anchor_labels.csv",
|
||||
[
|
||||
"cell_id",
|
||||
"tp",
|
||||
"mns",
|
||||
"probe_index",
|
||||
"sampling_u",
|
||||
"request_count",
|
||||
"offered_req_s_per_gpu",
|
||||
"real_feasible",
|
||||
"real_pass_rate",
|
||||
"frontier_feasible",
|
||||
"frontier_pass_rate",
|
||||
"frontier_completed_req_s_per_gpu",
|
||||
],
|
||||
anchors,
|
||||
)
|
||||
(args.output_dir / "report.md").write_text(render_report(metrics), encoding="utf-8")
|
||||
print(json.dumps({"status": metrics["status"], "metrics": str(metrics_path)}, sort_keys=True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
93
runs/frontier-slo-alignment-v0/results/anchor_labels.csv
Normal file
93
runs/frontier-slo-alignment-v0/results/anchor_labels.csv
Normal file
@@ -0,0 +1,93 @@
|
||||
cell_id,tp,mns,probe_index,sampling_u,request_count,offered_req_s_per_gpu,real_feasible,real_pass_rate,frontier_feasible,frontier_pass_rate,frontier_completed_req_s_per_gpu
|
||||
tp1_mns16,1,16,0,0.5,276,4.6,False,0.09057971014492754,False,0.14855072463768115,3.2422237998507493
|
||||
tp1_mns16,1,16,1,0.25,143,2.3833333333333333,False,0.8461538461538461,True,1.0,2.2847815885743166
|
||||
tp1_mns16,1,16,2,0.125,66,1.1,True,1.0,True,1.0,1.0641771799103616
|
||||
tp1_mns16,1,16,3,0.1875,103,1.7166666666666666,True,1.0,True,1.0,1.6601410596627497
|
||||
tp1_mns16,1,16,4,0.21875,121,2.0166666666666666,True,1.0,True,1.0,1.9360040008720856
|
||||
tp1_mns16,1,16,5,0.234375,132,2.2,True,0.9924242424242424,True,1.0,2.1118607283809685
|
||||
tp1_mns16,1,16,6,0.2421875,137,2.283333333333333,True,0.9927007299270073,True,1.0,2.1914280693644232
|
||||
tp1_mns16,1,16,7,0.24609375,141,2.35,True,0.9574468085106383,True,1.0,2.2549713872507646
|
||||
tp1_mns32,1,32,0,0.5,276,4.6,False,0.014492753623188406,False,0.6666666666666666,4.297017390853387
|
||||
tp1_mns32,1,32,1,0.25,143,2.3833333333333333,False,0.7132867132867133,True,1.0,2.284832740403262
|
||||
tp1_mns32,1,32,2,0.125,66,1.1,True,1.0,True,1.0,1.0641771799103616
|
||||
tp1_mns32,1,32,3,0.1875,103,1.7166666666666666,True,1.0,True,1.0,1.6601410596627497
|
||||
tp1_mns32,1,32,4,0.21875,121,2.0166666666666666,True,1.0,True,1.0,1.9360040008720856
|
||||
tp1_mns32,1,32,5,0.234375,132,2.2,True,0.9621212121212122,True,1.0,2.11158903487612
|
||||
tp1_mns32,1,32,6,0.2421875,137,2.283333333333333,True,0.9635036496350365,True,1.0,2.1911815099759533
|
||||
tp1_mns32,1,32,7,0.24609375,141,2.35,False,0.723404255319149,True,1.0,2.2550534049233777
|
||||
tp1_mns64,1,64,0,0.5,276,4.6,False,0.057971014492753624,False,0.6666666666666666,4.356763578770651
|
||||
tp1_mns64,1,64,1,0.25,143,2.3833333333333333,False,0.7132867132867133,True,1.0,2.284832740403262
|
||||
tp1_mns64,1,64,2,0.125,66,1.1,True,1.0,True,1.0,1.0641771799103616
|
||||
tp1_mns64,1,64,3,0.1875,103,1.7166666666666666,True,1.0,True,1.0,1.6601410596627497
|
||||
tp1_mns64,1,64,4,0.21875,121,2.0166666666666666,True,1.0,True,1.0,1.9360040008720856
|
||||
tp1_mns64,1,64,5,0.234375,132,2.2,True,0.9621212121212122,True,1.0,2.11158903487612
|
||||
tp1_mns64,1,64,6,0.2421875,137,2.283333333333333,True,0.9635036496350365,True,1.0,2.1911815099759533
|
||||
tp1_mns64,1,64,7,0.24609375,141,2.35,False,0.723404255319149,True,1.0,2.2550534049233777
|
||||
tp1_mns8,1,8,0,0.5,276,4.6,False,0.06521739130434782,False,0.06884057971014493,2.1725065543636846
|
||||
tp1_mns8,1,8,1,0.25,143,2.3833333333333333,False,0.6993006993006993,False,0.6153846153846154,2.0743292183634
|
||||
tp1_mns8,1,8,2,0.125,66,1.1,True,1.0,True,1.0,1.0641771799103616
|
||||
tp1_mns8,1,8,3,0.1875,103,1.7166666666666666,True,1.0,True,1.0,1.660069637502782
|
||||
tp1_mns8,1,8,4,0.21875,121,2.0166666666666666,True,1.0,False,0.8677685950413223,1.8977642969225545
|
||||
tp1_mns8,1,8,5,0.234375,132,2.2,False,0.7121212121212122,False,0.7575757575757576,2.0012415074420837
|
||||
tp1_mns8,1,8,6,0.2265625,126,2.1,True,1.0,False,0.7936507936507936,1.942525444038986
|
||||
tp1_mns8,1,8,7,0.23046875,130,2.1666666666666665,False,0.8846153846153846,False,0.7846153846153846,1.9778490149228842
|
||||
tp2_mns16,2,16,0,0.5,276,2.3,False,0.9492753623188406,True,1.0,2.2439374417654148
|
||||
tp2_mns16,2,16,1,0.25,143,1.1916666666666667,True,1.0,True,1.0,1.1652366234427673
|
||||
tp2_mns16,2,16,2,0.375,209,1.7416666666666667,True,1.0,True,1.0,1.7024011425654029
|
||||
tp2_mns16,2,16,3,0.4375,243,2.025,True,1.0,True,1.0,1.978602236135322
|
||||
tp2_mns16,2,16,4,0.46875,256,2.1333333333333333,True,1.0,True,1.0,2.083276142822733
|
||||
tp2_mns16,2,16,5,0.484375,265,2.2083333333333335,True,1.0,True,1.0,2.15467356806669
|
||||
tp2_mns16,2,16,6,0.4921875,269,2.2416666666666667,True,1.0,True,1.0,2.187119822377907
|
||||
tp2_mns16,2,16,7,0.49609375,273,2.275,True,1.0,True,1.0,2.219545738132131
|
||||
tp2_mns32,2,32,0,0.5,276,2.3,True,0.9855072463768116,True,1.0,2.2439374417654148
|
||||
tp2_mns32,2,32,1,0.75,391,3.2583333333333333,True,1.0,True,1.0,3.171147514698164
|
||||
tp2_mns32,2,32,2,0.875,450,3.75,False,0.42,True,1.0,3.6496440319651886
|
||||
tp2_mns32,2,32,3,0.8125,417,3.475,False,0.7745803357314148,True,1.0,3.381833218203857
|
||||
tp2_mns32,2,32,4,0.78125,407,3.3916666666666666,False,0.20147420147420148,True,1.0,3.3006981169615908
|
||||
tp2_mns32,2,32,5,0.765625,400,3.3333333333333335,False,0.695,True,1.0,3.243915834454301
|
||||
tp2_mns32,2,32,6,0.7578125,396,3.3,False,0.8535353535353535,True,1.0,3.2118868660091686
|
||||
tp2_mns32,2,32,7,0.75390625,394,3.283333333333333,True,1.0,True,1.0,3.195580748135602
|
||||
tp2_mns64,2,64,0,0.5,276,2.3,True,0.9855072463768116,True,1.0,2.2439374417654148
|
||||
tp2_mns64,2,64,1,0.75,391,3.2583333333333333,True,1.0,True,1.0,3.171147514698164
|
||||
tp2_mns64,2,64,2,0.875,450,3.75,False,0.34,True,1.0,3.6496440319651886
|
||||
tp2_mns64,2,64,3,0.8125,417,3.475,False,0.6282973621103117,True,1.0,3.381833218203857
|
||||
tp2_mns64,2,64,4,0.78125,407,3.3916666666666666,False,0.8427518427518428,True,1.0,3.3006981169615908
|
||||
tp2_mns64,2,64,5,0.765625,400,3.3333333333333335,False,0.0775,True,1.0,3.243915834454301
|
||||
tp2_mns64,2,64,6,0.7578125,396,3.3,False,0.29292929292929293,True,1.0,3.2118868660091686
|
||||
tp2_mns64,2,64,7,0.75390625,394,3.283333333333333,False,0.6040609137055838,True,1.0,3.195580748135602
|
||||
tp2_mns8,2,8,0,0.5,276,2.3,False,0.2028985507246377,False,0.39492753623188404,2.037309149499862
|
||||
tp2_mns8,2,8,1,0.25,143,1.1916666666666667,True,1.0,True,1.0,1.1652509326668725
|
||||
tp2_mns8,2,8,2,0.375,209,1.7416666666666667,True,1.0,True,1.0,1.6993884308462768
|
||||
tp2_mns8,2,8,3,0.4375,243,2.025,True,1.0,False,0.9465020576131687,1.9636944417416549
|
||||
tp2_mns8,2,8,4,0.46875,256,2.1333333333333333,True,1.0,False,0.890625,2.0353351328293683
|
||||
tp2_mns8,2,8,5,0.484375,265,2.2083333333333335,True,1.0,False,0.6943396226415094,2.0340077558764986
|
||||
tp2_mns8,2,8,6,0.4921875,269,2.2416666666666667,True,1.0,False,0.5204460966542751,2.0394007508896705
|
||||
tp2_mns8,2,8,7,0.49609375,273,2.275,True,1.0,False,0.4358974358974359,2.0358394808879736
|
||||
tp4_mns16,4,16,0,0.034252608017,600,2.5,False,0.16,True,1.0,2.4366476386646814
|
||||
tp4_mns16,4,16,1,0.017126304009,317,1.3208333333333333,True,1.0,True,1.0,1.3046624956163406
|
||||
tp4_mns16,4,16,2,0.025689456013,453,1.8875,True,1.0,True,1.0,1.8463926715458043
|
||||
tp4_mns16,4,16,3,0.029971032015,516,2.15,True,1.0,True,1.0,2.0988716639818685
|
||||
tp4_mns16,4,16,4,0.032111820016,552,2.3,True,1.0,True,1.0,2.2421056884527815
|
||||
tp4_mns16,4,16,5,0.033182214016,575,2.3958333333333335,True,1.0,True,1.0,2.335243044645089
|
||||
tp4_mns16,4,16,6,0.033717411016,586,2.441666666666667,True,1.0,True,1.0,2.3799430022241697
|
||||
tp4_mns32,4,32,0,0.034252608017,600,2.5,False,0.3466666666666667,True,1.0,2.462327006841985
|
||||
tp4_mns32,4,32,1,0.017126304009,317,1.3208333333333333,True,1.0,True,1.0,1.3046624956163406
|
||||
tp4_mns32,4,32,2,0.025689456013,453,1.8875,True,1.0,True,1.0,1.859491597548273
|
||||
tp4_mns32,4,32,3,0.029971032015,516,2.15,True,1.0,True,1.0,2.117703981656738
|
||||
tp4_mns32,4,32,4,0.032111820016,552,2.3,True,1.0,True,1.0,2.2653938418034003
|
||||
tp4_mns32,4,32,5,0.033182214016,575,2.3958333333333335,True,1.0,True,1.0,2.359733397875389
|
||||
tp4_mns32,4,32,6,0.033717411016,586,2.441666666666667,True,1.0,True,1.0,2.4049469442687075
|
||||
tp4_mns64,4,64,0,0.034252608017,600,2.5,False,0.3466666666666667,True,1.0,2.462327006841985
|
||||
tp4_mns64,4,64,1,0.017126304009,317,1.3208333333333333,True,1.0,True,1.0,1.3046624956163406
|
||||
tp4_mns64,4,64,2,0.025689456013,453,1.8875,True,1.0,True,1.0,1.859491597548273
|
||||
tp4_mns64,4,64,3,0.029971032015,516,2.15,True,1.0,True,1.0,2.117703981656738
|
||||
tp4_mns64,4,64,4,0.032111820016,552,2.3,True,1.0,True,1.0,2.2653938418034003
|
||||
tp4_mns64,4,64,5,0.033182214016,575,2.3958333333333335,True,1.0,True,1.0,2.359733397875389
|
||||
tp4_mns64,4,64,6,0.033717411016,586,2.441666666666667,True,1.0,True,1.0,2.4049469442687075
|
||||
tp4_mns8,4,8,0,0.034252608017,600,2.5,False,0.056666666666666664,False,0.095,1.5449814460277083
|
||||
tp4_mns8,4,8,1,0.017126304009,317,1.3208333333333333,False,0.8517350157728707,True,1.0,1.277194040226129
|
||||
tp4_mns8,4,8,2,0.008563152005,159,0.6625,True,1.0,True,1.0,0.6553587315711599
|
||||
tp4_mns8,4,8,3,0.012844728007,243,1.0125,True,1.0,True,1.0,0.9892987097876506
|
||||
tp4_mns8,4,8,4,0.014985516008,274,1.1416666666666666,True,1.0,True,1.0,1.1153521862441735
|
||||
tp4_mns8,4,8,5,0.016055910008,301,1.2541666666666667,True,1.0,True,1.0,1.2252281710185793
|
||||
tp4_mns8,4,8,6,0.016591107009,308,1.2833333333333334,True,1.0,True,1.0,1.2459143726426762
|
||||
|
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user