Decompose good/bad selection split across frozen surfaces
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,60 @@
|
||||
# 实验 S0：good/bad case 分裂的统一分解（margin vs differential residual）
|
||||
|
||||
> **状态：** 已完成（2026-07-20；含 S0b 方向化修正与一轮 strict review 修复）
|
||||
>
|
||||
> 用户指令：核心要务是分析为什么部分 case 下 Frontier work、部分不 work 的 system 根因；本 card 是该诊断 campaign 的第一个 slice，仅使用 frozen artifacts，零 GPU 成本。人工 review 由用户的直接指令（"只有做好这个分析我们才能推进下一步"）满足。
|
||||
|
||||
## Claim 与决策
|
||||
|
||||
- **Parent claim：** ongoing.md H2——误差机制是 action-conditioned residual；本实验把它细化为"分裂从哪来"。
|
||||
- **现象（已冻结）：** 同一 best-effort Frontier 栈上，Q30/Q235 的 Trace-PD 与多数 PO 面 selection 近优，而 Fixed-PD 面 14–58% regret；失败 objective 随负载档切换（Q30 低压 TPOT/E2E 全反、高压 TTFT 56–58%）；A1 collective profile 修复了 Q235 Trace-PO p90 21.2%→0.3% 但对 Fixed-PD 33% 完全无效（本 card frozen-inputs/q235-ablation-a1）。
|
||||
- **Competing hypotheses：**
|
||||
- **H-SCALE：** 分裂完全由「config-differential residual vs 真机 decision margin」的关系解释：good case 的 sim/real 比值跨 config 近似均匀（乘性偏移，argmin 不变），bad case 的比值跨 config 分散且超过 margin。workload shape 本身不需要出现在解释里。
|
||||
- **H-THRESH：** 绝对 service-time 高估近似均匀，但与离散机制（MNS admission cap、MoE token-bucket、graph bucket）交互后被转换为 config-differential 误差；fixed uniform workload 把所有请求同步到同一 state 轨迹，使阈值交叉对整个 cell 相干生效；trace 的长度/到达异质性把阈值效应摊平。
|
||||
- **H-STATE：** 失败由 simulator 闭环 batch state 分布漂移主导（Q235：sim decode batch 13.5 vs real 3.9 + B4→B5 profile cliff 正反馈）；即使打破 workload 同步性，闭环漂移仍可翻转排序。
|
||||
- 三者关系：H-SCALE 是现象层（必要条件），H-THRESH/H-STATE 是 differential residual 的两种产生机制，可共存但可判别（见事前预测）。
|
||||
- **事前预测：**
|
||||
- H-SCALE 成立 ⟺ 对每个 case×objective，failure 恰好发生在「top 邻域 log-ratio spread > log1p(真机相对 margin)」处（两侧同为 log-space 尺度），无反例。
|
||||
- H-THRESH 独有：bad case 的 differential 误差集中于阈值语义分量（first-scheduling wait、bucket 跳变段），且 sim-only 反事实（去阈值/加 jitter）恢复排序——Q30 高压 TTFT 的 admission 反事实已支持一例。
|
||||
- H-STATE 独有：差异化误差在去掉阈值分量后仍在 execution 项内（Q235 Fixed-PD 的 own-composition −20.07 vs exact-state +10.90 已支持一例）。
|
||||
- **判定规则：** S0 只裁决 H-SCALE 与「分量定位」（queue vs execution）；H-THRESH/H-STATE 的干预判别属 S1+（sim-only 反事实）与 GPU 实验（需另行 review）。若 H-SCALE 出现反例（good case 有 spread>margin 仍选对，或 bad case spread<margin），必须原样报告，不得平滑。
|
||||
|
||||
## Setup
|
||||
|
||||
- **输入（全部 frozen，runs/frontier-split-rootcause-v0/frozen-inputs/）：** q30-trace-pd（graph-piecewise comparison）、q30-fixed-hi（fixed-pd/fixed-po 高压面）、q30-expansion-lo（低压 fixed-pd/fixed-po/trace-po）、q235-fourcase-a0、q235-ablation-a1（+provenance）、q235-state-diag、q30-admission-diag。来源 cpfs 路径与 SHA 见各目录内 manifest/launch 记录。
|
||||
- **计算（每 case×objective）：** per-config 比值 r_c=sim_c/real_c；config-uniform scale=geomean(r_c)；differential residual=log-ratio spread（全 surface 与真机 top-3 邻域各一）；真机 relative margin（best 与 2nd-best、best 与 sim-winner 的真机值差）；failure flag=regret>5%；H-SCALE 检验=failure ⟺ 邻域 spread>log1p(margin)（review 修正：初版直接以 ln 差比较普通 relative margin，尺度不一致；修正后 70 行 verdict 不变）。
|
||||
- **分量定位：** q30-admission-diag 提供 TTFT=first-scheduling wait+prefill execution 分解；q235-state-diag 提供 own-composition vs exact-state contrast；把这些已知分量证据合并进统一表。
|
||||
- **交叉核对：** 重算的 regret 必须与各 frozen comparison.md 表一致（抽查 58.0%、33.0%、0.0%）；A0 vs A1 的 Q235 对比必须复现 trace-po p90 21.2%→0.3%、fixed-pd 四项不变。
|
||||
|
||||
## 预期产物与 review
|
||||
|
||||
- runs/frontier-split-rootcause-v0/analyze_split_decomposition.py（只读 frozen-inputs，确定性输出）
|
||||
- runs/frontier-split-rootcause-v0/results/decomposition.{json,md}：统一表，每行 case×objective，列出 winner、regret、scale、spread（全/邻域）、margin、H-SCALE verdict、已知分量归因
|
||||
- runs/frontier-split-rootcause-v0/results/margin-vs-residual.png：x=真机 margin，y=邻域 differential residual，点色=selection 对错；H-SCALE 成立则对错点被对角线分离
|
||||
- 人工验收：编排者亲自重跑脚本、抽查交叉核对数字、亲自查看渲染图
|
||||
|
||||
## 复现信息
|
||||
|
||||
- **Code：** AITuner branch feature/sim，自 HEAD 18c0b25 起；worker/reviewer job-id 见下方「Review 与 provenance 补记」；脚本与产物随本 card 同一 commit 入库（含 frozen-inputs 本地拷贝）。
|
||||
- **Environment：** 本地 workstation，CPU-only，python3+matplotlib；不访问远端。
|
||||
- **已知 deviation：** frozen-inputs 是 cpfs 原件的本地拷贝（scp，2026-07-20）；q30-expansion-lo 的低压 Fixed-PD 面已被高压面取代为 primary，本分析将两档并列为独立观测，不混合。
|
||||
|
||||
## 结果
|
||||
|
||||
- **观察事实：**
|
||||
- 70 行（14 个 case surface）全部算出，无数据缺口；四组硬性交叉核对通过；连续运行产物 SHA 一致。
|
||||
- **H-SCALE 判为必要非充分**：23 个 material failure（regret>5%）全部满足「top-3 邻域 log spread > log1p(margin)」，无一例失败发生在 residual 小于 margin 处；但另有 40/70 行同样满足该条件却均非 material failure（其中仅 16 行 exact winner match，其余 24 行是小 regret 的 winner 错位——14 个 MNS 精确 tie 与 10 个 strict reversal）——无方向 spread 不携带决策信息。
|
||||
- **S0b 方向化后的机制普查**：23 个 material failure 的 winner-deciding pair 分布为 tp-axis 11、mixed 10、mns-axis 2（Q235 A0/A1 Fixed-PD 的 8 个 TPOT/E2E failure 全为 tp-axis；Q30 Fixed-PD 高低压为 tp/mixed）；仅有的 2 个 mns-axis failure 是 Q235 A0/A1 Trace-PD E2E p90（regret 6.2%，勉强越过 5% 门槛）。trace 面严格反序中 tp-axis 为 0（q30 trace-pd 三轴全 0）。
|
||||
- **A1 对照的轴分解**：serving-matched collective profile 把 Q235 两个 PO 面的 tp-axis 反序从 4/4 清零（trace-po p90 regret 21.2%→0.3%），但 Fixed-PD 仅 5→4、四项 regret 一位小数不动——prefill 路径的 TP-differential 误差源=collective profile（可修），decode 耦合的 TP-differential 误差另有来源。
|
||||
- **MNS 不敏感缺陷**：14 个 winner-label mismatch 是 simulator 逐位相等的 tie，全部 mns-axis（如 q30 fixed-po 的 MNS16↔32、q235 fixed-pd 的 MNS64↔128）；tie 计入后 MNS 边界误差 31 与 TP 严格反序 32 相当，但 MNS 侧 regret 小。
|
||||
- **成功的鲁棒性**：23 个 exact-winner success 中 17 个 margin-robust（margin≥1%），6 个 fragile（含 q30 trace-pd E2E p90 的 0.1% margin 与 q235 A1 fixed-po 四项）。
|
||||
- **面级 scale 对照**：prefill-only 面 geomean scale 0.96–1.37×（绝对预测基本准确），含 decode 的面 4.3–130×——绝对误差灾难集中于 decode。
|
||||
- **异常：** 无数据异常。strict review（FAIL：3 Major/1 Minor）指出 H-SCALE 尺度混用（log spread vs relative margin）、tie 轴普查缺失、card 状态过期、Q235 一致性表述过强；全部修复，修复后 70 行 verdict 逐行不变。
|
||||
- **含义：** 分裂的现象层解释是「margin 保护 + config-differential 误差」。机制层上，21/23 个 material failure 由 TP/mixed pair 决定，且所有大 regret（≥13%）failure 都发生在含 decode 的面上：其中 Q235 Fixed-PD 有 state-drift 直接证据、Q30 高压 TTFT 有 admission 反事实证据，而 **Q30 低压 TPOT/E2E 反转的机制尚未诊断**（S1 目标）。「decode 耦合的 TP-differential 误差是主要载体」是当前最强归纳，不是对全部 failure 的已证机制归因；2 个 mns-axis 边缘 failure（6.2%）在该归纳之外。trace 面成功伴随「TP 反序为零 + TP margin 宽」，但「误差小」与「margin 宽」谁是主因仍未判——这正是 H-THRESH vs H-STATE 的判别缺口。轴标签与机制不一一对应（Q30 admission 是 MNS 阈值机制但 deciding pair 为 tp/mixed，因 TP 改变到达压力）。
|
||||
- **Claim update：** H2（action-conditioned residual）supported 且被细化：residual 的决策相关分量集中在 TP 轴、由 decode 状态耦合产生；H-SCALE 降级为必要条件；H-THRESH/H-STATE 保持 competing，待 S1 判别。
|
||||
- **下一步：** S1（sim-only 反事实：Q30 低压 Fixed-PD TPOT 反转的分量定位——这是唯一无机制解释的 material failure；fixed workload jitter 判别 H-THRESH vs H-STATE）；GPU 判别实验（加压 Trace-PD、jittered Fixed-PD 真机面，dash1–4）另行出 card 供 review。
|
||||
|
||||
## Review 与 provenance 补记
|
||||
|
||||
- S0 worker：codex `task-mrsn1s9c-z6ha0w`；S0b：`task-mrsnjzn6-k18x4n`（resume）；strict reviewer（fresh 只读）：`task-mrsocmlb-386btc`（verdict FAIL）；修复轮：`task-mrsop06n-pwxo0b`（fresh writable）。编排者独立验收：脚本重跑、SHA 比对、两图目视检查。
|
||||
- 产物 SHA（修复后）：decomposition.json `81ea56b2…`、decomposition.md `4d19af54…`、margin-vs-residual.png `df2110c4…`、decision-pair-axis.png `2e787301…`。
|
||||
@@ -3,14 +3,16 @@
|
||||
> 2026-07-17｜写给未参与项目的读者，可直接作为 presentation 讲稿。历史过程与复现信息见 `../runs/*/` 各 experiment card、`../docs/` 各 campaign 文档。
|
||||
>
|
||||
> **2026-07-19 update：** Qwen235 Fixed-PD 的错误排序在 exact real state composition 下已经翻正，主因是 simulator closed-loop batch state，而不是 collective。Qwen30 Fixed-PD 的 56--58% TTFT regret 也已定位：Frontier 将 decode service time 高估 4--8×，使 TP4 的 modeled concurrency 越过 MNS admission cap并产生虚假排队；去掉该等待后 Frontier 与真机都判定 TP4 topology 更快。详见 [`experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md`](experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md)。
|
||||
>
|
||||
> **2026-07-20 update：** 对全部 14 个 frozen case surface（70 个 case×objective）做了统一的 margin-vs-residual 分解与方向化机制普查（[`experiments/frontier-split-rootcause-s0-20260720.md`](experiments/frontier-split-rootcause-s0-20260720.md)）。三个要点：(1) 「residual 超过 margin」是失败的必要条件但远非充分——good/bad 分裂不能用无方向误差量解释；(2) 23 个 material failure 的 winner-deciding pair 中 21 个落在 TP 轴或 mixed（其余 2 个是 6.2% regret 的边缘 mns-axis case），trace 面的 TP 反序为零；A1 measured collective 把 Qwen235 两个 prefill-only 面的 TP 反序清零（trace-PO p90 regret 21.2%→0.3%）却对 Fixed-PD 完全无效——prefill 路径的 TP 差异化误差源是 collective profile（可修），decode 耦合的 TP 差异化误差是当前所有 material failure 的载体；(3) 「Fixed-PD 失败因为高压」被否证：失败 Fixed-PD 的真机 in-flight（14.05）低于全对的 Trace-PD（38.69），且低压 Fixed-PD 同样失败、失败 objective 随负载切换。另有次要缺陷：14 个 winner 错位来自 simulator 对 MNS 逐位不敏感的精确 tie。
|
||||
|
||||
## 一眼看懂
|
||||
|
||||
- **Topic / problem：** LLM serving 的自动、低成本配置调优（AITuner）。当前主线问题：用 simulator 给部署配置（并行度、批量上限等）排序，什么时候可信？需要补多少真机证据？算上这些成本还划算吗？
|
||||
- **Central claim：** simulator 要能帮助配置调优，必须先满足 scheduler transition 的 liveness/coverage，再满足「配置相关残差小于真机 decision margin」；前者决定 capacity 是否有定义，后者决定排序是否正确。(ID: C0)
|
||||
- **当前结论：** 早先 35 个 trace stall 不是 Frontier scheduler liveness failure：adapter 为不满 16-token 的 prefix block 错误生成了 cache identity，Frontier 又没有 fail-fast。修正为完整 block、使用真实 graph buckets/KV blocks 和 `piecewise`/`KERNEL_ONLY` profile 后，Qwen30 Trace-PD 的全部 12 个 cell 完成 129/129 request，Frontier 对 TTFT/TPOT/E2E 的 6 个 argmin 均与三次 fresh-server 真机一致；但绝对 latency 仍高估 4--511×。这只证明一个 MoE Trace-PD surface 的 selection fidelity，不能外推到 prefill-only、fixed workload 或 235B。
|
||||
- **最大 uncertainty / risk：** 这个正确选择是否依赖 Trace-PD 的 queue/margin 偶然性；Fixed-PD、Trace-P、Fixed-P 与大型 FP8 MoE 是否仍保留正确 config ranking。Q235 还缺少同栈 profile/runtime contract，不能复用旧 vLLM 0.10.2 数据。
|
||||
- **下一项 critical action：** 完成 Qwen30 其余三项 fixed/trace × PD/P surface。Fixed case 已先启动无请求 vLLM runtime-state preflight；Trace-P 的 graph-aligned Frontier 12-cell CPU surface 正在运行。Q235 先做 vLLM 0.20 TP4/TP8 compatibility gate，随后才允许新 profile。
|
||||
- **最大 uncertainty / risk：** trace 面成功的原因未判——是 decode 耦合的 TP 差异化误差在 trace 状态分布下真的变小，还是只是被宽 TP margin 掩盖。这决定「sim 剪枝可信域」的边界怎么画，也决定加压 trace 是否会失败。
|
||||
- **下一项 critical action：** 判别两个失败机制假设（离散阈值转换 vs 闭环 state 漂移）：先做零 GPU 的 sim-only jitter 反事实与 Q30 低压 Fixed-PD TPOT 反转分量定位，再按结果决定 dash1–4 上的加压 Trace-PD / jittered Fixed-PD 真机判别面。
|
||||
- **停止条件：** T1 出 verdict 且成本账本建立后：pass 且摊销论证成立 → 转向「sim 剪枝 + 真机终选」的 hybrid 机制设计；fail → 转入失败机制归因；两条路都无 insight 增量 → 收敛写作。
|
||||
|
||||
## 核心概念
|
||||
@@ -34,9 +36,10 @@
|
||||
- **Supporting：** 235B prefill-only regret=0；235B fixed-shape mixed 的 top set 全中；30B 加 per-TP 校准后 regret 0.76%（但这是外部端到端 scale 给出的上界，不是原生 profile 保真度）。
|
||||
- **Counterevidence：** 修正 prefix trace contract 后的 TP2/MNS16 `none`-graph run 完成但 p50 TPOT 约 96 ms，真机为约 14 ms；然而该比较尚未对齐 real vLLM 的 `FULL_AND_PIECEWISE` graph path。
|
||||
- **下一项 discriminative experiment：** 补齐 `KERNEL_ONLY` graph family并以 `piecewise` 重跑相同 trace；若 full surface 仍错，graph omission 不再是可用解释。
|
||||
- **Hypothesis（机制，active）：** 误差机制是 action-conditioned residual——执行状态的转移（并行拓扑、kernel family、graph mode、batch 组成）使按算子 profile 的组合预测跨配置不可复合；残差大于 margin 时排序失败。(ID: H2；supported)
|
||||
- **Hypothesis（机制，active）：** 误差机制是 action-conditioned residual——执行状态的转移（并行拓扑、kernel family、graph mode、batch 组成）使按算子 profile 的组合预测跨配置不可复合；残差大于 margin 时排序失败。(ID: H2；supported，已细化)
|
||||
- **Supporting：** 三个 TP 档的端到端校准系数为 0.72/0.47/0.35，残差确实随配置剧烈变化；235B 的批量上限交互预测错误但被 2× margin 容忍；30B prefill-only 在低负载近似对齐、饱和后按 TP 反向放大，最终 τ-b=−1。
|
||||
- **下一步：** 按 collective 通信 → batch 组成 → 调度器逐步轨迹的单变量顺序定位组合性失效点（对应 experiment card 中 A1/A2/A3 消融阶梯）。
|
||||
- **细化（2026-07-20 统一普查）：** 决策相关的残差分量集中在 TP 轴且由 decode 状态耦合产生——prefill-only 面的绝对 scale 仅 0.96–1.37× 且 measured collective 即可清除其 TP 反序，而含 decode 的面 scale 4.3–130×、全部 material failure 都由 TP/mixed pair 决定。「residual>margin」只是必要条件；失败还需要残差对准 winner-deciding pair。
|
||||
- **下一步：** 判别两个 competing 机制假设——离散阈值转换（admission cap/profile cliff 把均匀高估转成差异化误差；打破 fixed workload 的同步性应恢复排序）vs 闭环 state 漂移（打破同步性也救不了）。最便宜路径：sim-only jitter 反事实 + Q30 低压 Fixed-PD TPOT 反转的分量定位（唯一无机制解释的 material failure）。
|
||||
- **Subclaim：** 成本论证只有在摊销前提下成立。(ID: C3)
|
||||
- **Hypothesis（active）：** 每个 model×硬件×runtime 的一次性对齐成本，摊销到大配置面、频繁重调（引擎版本 churn 的频率证据见 claim map）或禁止在线实验的场景后，低于重复真机调优。(ID: H3；untested——分母已实测，分子未入账)
|
||||
- **下一步：** 建 cost ledger（见「下一步」）。
|
||||
@@ -52,7 +55,7 @@
|
||||
## Key evidence（最多 3 条）
|
||||
|
||||
- **E1（否证「prefill-only 是充分 easy condition」，支持 H2）：** 30B BF16、去掉 decode/prefix/混合 batch 后，真机最优是 TP4（8 vs 7 req/s/GPU），simulator 却把 TP4 排最差（6 vs 8）：top set 无交集，regret 12.5%，τ-b=−1。产物：`../runs/frontier-phase-factorial-v0/results/final/`（dash0，12.07 H20-GPUh）。
|
||||
- **E2（margin 可掩盖残差，支持 C0/H2）：** 235B FP8 的 prefill-only 与 fixed-shape mixed 都选对 top set，但 simulator 漏掉并发上限（max-num-seqs）×批 token 上限（max-num-batched-tokens）的交互棋盘格，并把 10/34 个真机 anchor 误判为不可行；真机头部 margin 有 2×，残差被容忍。产物：`../runs/frontier-multicase-sufficiency-v0/best_effort/fixed_cohort_evidence/`、`../runs/frontier-multicase-sufficiency-v1/results/t0-final/`。
|
||||
- **E2（统一机制普查：material failure 全部由 decode 耦合的 TP 差异化误差决定，支持 H2 细化）：** 对 14 个 frozen surface、70 个 case×objective 的方向化分解显示：23 个 material failure 中 21 个由 TP/mixed pair 决定（仅 2 个 6.2% 边缘 mns-axis case）、trace 面 TP 反序为零；measured collective（A1）把 Qwen235 两个 prefill-only 面的 TP 反序清零（trace-PO p90 regret 21.2%→0.3%）但对 Fixed-PD 的 33% 无效；「residual>margin」仅为失败的必要条件。产物：[`../runs/frontier-split-rootcause-v0/results/`](../runs/frontier-split-rootcause-v0/results/decomposition.md)（实验 card：[`experiments/frontier-split-rootcause-s0-20260720.md`](experiments/frontier-split-rootcause-s0-20260720.md)）。
|
||||
- **E3（selection 与 calibration 分开）：** 用完整 block projection、`piecewise` 和 graph-compatible KERNEL_ONLY profile 后，Qwen30 Trace-PD 的 12/12 sim cells 完成，6 个 mean/p90 latency objective 的 argmin 都与真机一致；但 sim/real latency ratio 仍为 4--511×。产物：[`../runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md`](../runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md)。
|
||||
|
||||
## 下一步（最多 3 项）
|
||||
|
||||
1444
runs/frontier-split-rootcause-v0/analyze_split_decomposition.py
Normal file
1444
runs/frontier-split-rootcause-v0/analyze_split_decomposition.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1 @@
|
||||
10567da5238db57c29a45e09ebd40343a7d1a543
|
||||
@@ -0,0 +1,180 @@
|
||||
bfbfb84d0310b312c3f3f0047219d5b615850d8b5b465282d6207ebe7b5ba987 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/analysis/comparison.json
|
||||
aa0295fcd1a0465483ca3655aae94de543acf5b39bde2fcfff6ff40a7fb27f2a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/analysis/comparison.md
|
||||
f2c2a94627194431497c751e1ed0e3768f1f7971c3667e1f70563f1d7c5118f3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/controller.log
|
||||
30bdc9da5c8e638d6278b3ae3f812613a1f7ef296439e34d28dcb72ae30b0d1b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/profiles/qwen235-serving-allreduce.csv
|
||||
87a92ee3ba6955b95cf0fcdbf5dd8675a8ec85bf4a0dfd61ae12a75c32679fac /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/profiles/qwen235-serving-allreduce.manifest.json
|
||||
13a6cd30307a6dcdd12dc460949ce1e618cbd167cd976ad1b71ad6477869b55d /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/provenance/aituner.commit
|
||||
e3f7b627aa262187e7e02b703595f32fb1ced508724484ff515a68fe037d4f03 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/provenance/frontier.commit
|
||||
cd3d7ea44155f2d3e801926b35134d9735777256d72ba3f6a8f0cff2770ba09a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/provenance/input.sha256
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/138b2eb7_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/479e1bac_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/5ae40466_model_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/6e9cd646_model_lock.file
|
||||
a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_138b2eb7_predictions.csv
|
||||
f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_138b2eb7_predictions.pkl
|
||||
628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_479e1bac_predictions.csv
|
||||
7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_479e1bac_predictions.pkl
|
||||
7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_5ae40466.pkl
|
||||
125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_6e9cd646.pkl
|
||||
c56228d3654b51ea42c73f0883d58c67c19b1024c14f1677c63cb882e4dc3334 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/frontier_surface.json
|
||||
cf54e197cfcb6b480ccbd658f73dad0a90597bd58399d1c07f25e3987ca02227 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/command.json
|
||||
7796df845a96bdb9603baf372118466e27200fc10ee83b26b469d4e2c089308f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv
|
||||
01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv
|
||||
ab0ee72a7e2fbb3bf87c1653a9b1805b7805fd0f9ae3676ce69a0d9b112a4545 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json
|
||||
84a609818002bcfb648a236a65b1d40dcd3c964360cb488dfb7549eddc546533 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/stderr.log
|
||||
469ad61c0082534fa069a3ddc38510b4dc2180d016518834d3ac69f31c3ef7e5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/stdout.log
|
||||
080edcceed5f26160cd073a33c87aa8399cb0fe7cc9227bacb442aa1f90c5a30 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/command.json
|
||||
db4e6b4632fc16a368407e458d26fd7a0adc9ae930755ac7784b899da60ee9fc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv
|
||||
01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv
|
||||
ab0ee72a7e2fbb3bf87c1653a9b1805b7805fd0f9ae3676ce69a0d9b112a4545 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json
|
||||
eb987dc5b2222740cef7577350190033cccee41946638aa5e9a5fd20c6be8b45 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/stderr.log
|
||||
62312ef79e6302bf7be0c6ba276211065950b46eb4b4aef0a031358a804e17ce /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/stdout.log
|
||||
89b4fc5eb8ccd1d5dc58b778352acaec5d496dded0181990fd3bce30c557ab48 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/command.json
|
||||
bcc034a01b617f26aa4d7d760082fc1baa1e62b1b63ba50e11c9e234124d2c37 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv
|
||||
51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv
|
||||
758424010e61780cbcbb320d68caa22c2e35c067654f9c4ef98217ff90e07b30 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json
|
||||
463057bcb54084ab86eb35fc4f33ad62d1b3ebd05b6b04a1510ce8df6e686807 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/stderr.log
|
||||
0a0433b5c338632e35516e34b262cf8b714cea73488ba9e3459f4881a0fe327e /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/stdout.log
|
||||
2a69949a89e4dcedbede612b8beab834d63eb8683d97975bc779764304b1a627 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/command.json
|
||||
88def2ba5bd00b375e0d907976bdd45c36faf53ad7980d8a1eb7e46057ccc8ec /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv
|
||||
51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv
|
||||
758424010e61780cbcbb320d68caa22c2e35c067654f9c4ef98217ff90e07b30 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json
|
||||
6ed18c855e41284d27a8aeca93799ba55745e7527344671edfed449693b29758 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/stderr.log
|
||||
30bc99671a0bedac259b8be1d980a2981a48439e2d04a742d7baa4cc82638b39 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/stdout.log
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/138b2eb7_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/479e1bac_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/5ae40466_model_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/6e9cd646_model_lock.file
|
||||
a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_138b2eb7_predictions.csv
|
||||
f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_138b2eb7_predictions.pkl
|
||||
628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_479e1bac_predictions.csv
|
||||
7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_479e1bac_predictions.pkl
|
||||
7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_5ae40466.pkl
|
||||
125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_6e9cd646.pkl
|
||||
e83e951102f893a83c7ef4ebbae6e6f998a1a43a8d9f7a6ed165f5db96609980 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/frontier_surface.json
|
||||
e3f2699e0b9e182ab440acc690ddd8aaf0c954c5a83b7e1f444b3d494fcb2379 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/command.json
|
||||
4a28f5aa74e7f9909545b1d1bc00a66098d85f7fff6b72fbb98d11d4a3f786c0 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv
|
||||
0030204dc1047a02d601f7c1d57eee9fc2234080afd4e499c016b902a8da7573 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv
|
||||
d5b01e9e1689106d8174b1246da2c0c7a35ab7b83b1f8f2f3919753ac320e1e3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json
|
||||
599bb360437de9ebb16c1b29958c27b2c7f8e95eb047f9b8f26cb5709003755a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/stderr.log
|
||||
386b21c7695ae4159fb60f5c8772fd91512319f1511769e9b675955b7f26a223 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/stdout.log
|
||||
3db0c671d763a7db9565b151464b96060455bafe7f065b35719e27c7d45b12b5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/command.json
|
||||
dfe34eef677ee24edd43ef01008aa30ed02fd7f251876149a55b9436130f179b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv
|
||||
0030204dc1047a02d601f7c1d57eee9fc2234080afd4e499c016b902a8da7573 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv
|
||||
d5b01e9e1689106d8174b1246da2c0c7a35ab7b83b1f8f2f3919753ac320e1e3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json
|
||||
9cceef591ceae33b03dddd558387bf4e1853c2a3e74913dd7839fb47dacfcb9c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/stderr.log
|
||||
1d5494b859450660d3f06050a91af27072ec4d15f4fe4bfb159f32f2e6769b89 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/stdout.log
|
||||
439fe52e1d3c10c9c5cb4db20308e62da26a44866085d44768244d82ff0d22d1 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/command.json
|
||||
9c1211fc6032dfa395933dbd2977115c5494a37b814c335dd2ed1ce70716c7ce /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv
|
||||
d3ebd523ddf95f021d98c043d290670b20d8c0ffc2fdbca27fc5740ae46981f5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv
|
||||
264cd59b0b0b9797f5054f8527e13412a9c9b5fd3a4608d3f81b228a6c08d0bc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json
|
||||
80adf7553fe587b542d0c43acf6850412199503a0eaa17ebbb04ac2870326059 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/stderr.log
|
||||
3b86967c56b8dbd8db2edbbbbae367864ef41c703ee51fc4636811ca43f5c1ac /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/stdout.log
|
||||
0ee66ca4a8363be762616e69329287c5a74993f36283ed5a470a4b012ec9a000 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/command.json
|
||||
98b3db100b934f7b3473fa340f1c368f510f55071f7f79d458264a6d3aac5209 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv
|
||||
d3ebd523ddf95f021d98c043d290670b20d8c0ffc2fdbca27fc5740ae46981f5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv
|
||||
264cd59b0b0b9797f5054f8527e13412a9c9b5fd3a4608d3f81b228a6c08d0bc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json
|
||||
6b2fad699b75bf96907cf6408da1cb5c34ba7429da791df8f5c55ab24a949240 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/stderr.log
|
||||
05475a151dca4d87d060c2714d7bd8837964d77b1e4ba335f1363b38a74e3a01 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/stdout.log
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/138b2eb7_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/479e1bac_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/5ae40466_model_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/6e9cd646_model_lock.file
|
||||
a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_138b2eb7_predictions.csv
|
||||
f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_138b2eb7_predictions.pkl
|
||||
628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_479e1bac_predictions.csv
|
||||
7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_479e1bac_predictions.pkl
|
||||
7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_5ae40466.pkl
|
||||
125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_6e9cd646.pkl
|
||||
42420125d3a8ce4aed8dd1c859d39ff840b4e77a52ad5752de73d00a0aed5c3b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/frontier_surface.json
|
||||
a222ec73d311c5d4e57b0ed3ed2054cb83ec31ff78031eb2b8bde6031472f9ca /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/command.json
|
||||
9ff45df909e4f94da9435e85d893c0582fbb57ecc28ab15dddadce26604c0f1b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv
|
||||
12984c1c3ac1ab1d8fb35e7bcbacb147d827cdba43e934fdc757ee8ea7baf6fb /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv
|
||||
343388af242d66c2e8e46a4462dd23862d6437725c80961588fa6cdc328acdec /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json
|
||||
b64186eaaa0f2fa566d181b40ed42cd9303b8131ce6fd7c4ae3d77855533a645 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/stderr.log
|
||||
ece0b457d6c6fb30280d003e209912b918d06abc5e0781296ce29c6855542ed5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/stdout.log
|
||||
837819c6bcb470bc6067a3e5b24713166e4e3b1164090793690874597ea60e03 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/command.json
|
||||
8db5d447bfc3506e4cf76a511a5a4760c567e2303240f878212aad44ff6f7cdc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv
|
||||
d244788351f45f3473a8a2705b232e1a951485b5a24e09029331f42eb47bbafc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv
|
||||
5795a7d8f48ea0cde14f5c724d5a77d0903398a75151a95c39ad99c2ba2bfb68 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json
|
||||
5baba11faf64f5c9e4ebbd123aa745c46d73c3f0f81cdd0a16249d212020d443 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/stderr.log
|
||||
4d70795d321dff067aca5e9ead098726df129d556ceca8eb437d63cefa0c900c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/stdout.log
|
||||
40b8d7eb4a946406f37532bb308514bc4982088d872374d078635ce6ef764001 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/command.json
|
||||
658539186152881399746213af4e3b041b5589cf3a8559346a109d444149767b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv
|
||||
00e4ed87800ce6054cf4818a4bac9b5db72c5e6dff52ff57d3b7b2489e97ffda /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv
|
||||
8947f894df28ac9f9cec7be09e79767a685f2ef96835e495d78eadcf5092633a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json
|
||||
04134b8170da075e36d33766ea9d520dd4db57abcf188116e05aaee1d125f8f3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/stderr.log
|
||||
bf6e1295c814c455d1c0b9f02ba0669636b58475efe8873907c6f53677e75fce /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/stdout.log
|
||||
8719be2b20a1d0b35fc5bf23f398801737725527e6ca502a6f343566741517ca /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/command.json
|
||||
5280da88ab96a0da6e54dc6dcf17d783822b8a49387a696e79830dfd7e81d2d2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv
|
||||
20adfc333507bbe98f607b1075b0f790e4651ad3185f96906009f6177839de96 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv
|
||||
1b8a2c5be9bb1c5ab551768a9bbe1c95a6c3f93c9f2b1cb6e1a53a86d3ffb239 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json
|
||||
683afcccfbcfe232313965d985217bbc8acce32426bc31df749c6b8177cee8a9 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/stderr.log
|
||||
02b34698d18bb18b46e4792b6208c6277249399e80e5f32c42a112f2fdc4a0e2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/stdout.log
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/138b2eb7_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/479e1bac_prediction_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/5ae40466_model_lock.file
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/6e9cd646_model_lock.file
|
||||
a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_138b2eb7_predictions.csv
|
||||
f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_138b2eb7_predictions.pkl
|
||||
628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_479e1bac_predictions.csv
|
||||
7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_479e1bac_predictions.pkl
|
||||
7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_5ae40466.pkl
|
||||
125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_6e9cd646.pkl
|
||||
aafb5ba279ba8d0f693e12778851c563de4702ffba51e908324ad0e613468d3c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/frontier_surface.json
|
||||
1ae2c8f734d6bde7740d06953f21c69af00b6307bcb8fc15b3a6a9415e6d29d8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/command.json
|
||||
b8f251e1ae5034352f35d61cbfdcad77c48ff978452b43092102ad785dc59930 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv
|
||||
c164e64b1f0b2eb009185b07247d7ce3abbe4baf377ad62ef071a4767395b704 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv
|
||||
3b65bd4a043ab4c395b988fa94172123e851638b1f250ca8761f8725d1cf8cf5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json
|
||||
2b61dc88137c817f96e7dcd45c65f451b8031647e56c6f8e554c9b91aee11982 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/stderr.log
|
||||
0fb063af9bf0cc706e42f12ac1883a309bb4e7ca41924dac8b9ae8d9d580b223 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/stdout.log
|
||||
08e6f686731ff6ceda8577c088ade825182777cff2528e207b444f2fe575f7ef /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/command.json
|
||||
2f870b68c0820e241e43ebe435d301c88e2abf68bf9e29ff22c08baf39aeb574 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv
|
||||
9349cb9f1ab478343b1b8d079cc175b50328b7c0bffb37018b3a7dc3e208c915 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv
|
||||
141593e073d0247455c73de04da5d82e918e3bf4128c225153a3e85faf63420c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json
|
||||
c6faee6e21ba387608c5387e1a05eae3158a6edda0c530c38d4d8664b5e34a3c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/stderr.log
|
||||
cd8ee4fb98de672f9b7550dc39e0bf3e80a44c41c65f80c2f5192a9083d1b0cb /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/stdout.log
|
||||
961af5f501fcc9606d0ba69924aa50b06e8030dc4239091216780e036702c95d /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/command.json
|
||||
d9330a115bd2bf15f84986c44fdb35aecf67b8f8138807ed7c2256f36e1a6543 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv
|
||||
6c6f40eba2795ef41f027c7412134694a91baa484933afc9b300803412d376cf /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv
|
||||
d7a7c8fe3fdcfb5d33589cf4ac1706e9fc011c95302180d6d7a1b874f6fd19f4 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json
|
||||
8aed33f0fc90d82c63f0a3c17dc0ef37ffd555046a4155f6333fca8ca4250df0 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/stderr.log
|
||||
756930da43bef6ec443ebe6d6360ad15069a3a43cc0c0c6ef07d2795b83cfdb2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/stdout.log
|
||||
3dad31841fddfff0dc4c8dec00f78f9ffc75de82fc94eec50a354ee64902aa6d /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/command.json
|
||||
b0437765d7f3c27102ea54833ff86d9c14c4f2b76185e770cf1276a53edb5669 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json
|
||||
9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv
|
||||
015d5b1b48dbe1bb0de1a8a3265a1292e75af1f58d5ac076486e0b85fb133968 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv
|
||||
c0cb828fcaf067076c82a68eeb4c180e5e38d6f0f43dfcf666b0a4515b78a70c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json
|
||||
8e4de34f2714e2b0cc1b7a7cc7e657662c3f99231adcea922dca73503a1d9ca2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/result.json
|
||||
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/stderr.log
|
||||
336c646cf9d88d4059f6a4584dab312a2e55401a808627fa8011280c7eb39c3a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/stdout.log
|
||||
@@ -0,0 +1 @@
|
||||
6e8e0d845bceff11b0b62cb29df3a1a93411fdd4
|
||||
@@ -0,0 +1,4 @@
|
||||
8eadf8b39becfcbe103c6328ca616902689e9694cc6cb6c1f08609f0e758ec41 /home/admin/cpfs/wjh/aituner/aituner-q235-collective-20260719/runs/frontier-fidelity-envelope-v1/run_qwen235_collective_profile_ablation.sh
|
||||
30bdc9da5c8e638d6278b3ae3f812613a1f7ef296439e34d28dcb72ae30b0d1b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/profiles/qwen235-serving-allreduce.csv
|
||||
5aa8657c5271abec6fca78a9f8fa2cfc0696f24c61033a850d7289f53f48632e /home/admin/cpfs/wjh/aituner/qwen235-v020-profiles-20260719-r2/frozen/manifest.json
|
||||
99befe893683e3a638ea28048d00fa6c7b3d6a209921fe19ad51e78dfbc39459 /home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/provenance/runtime-contract.json
|
||||
@@ -0,0 +1,799 @@
|
||||
{
|
||||
"cases": {
|
||||
"fixed-pd": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.3067524317507717,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.3456375801249614,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.33025510615656906,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.37228890038064755,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.002088257708717345,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 5729.31677040774,
|
||||
"e2e_p90_ms": 5871.668232022785,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 21.044200828709027,
|
||||
"tpot_p90_ms": 21.59612646244247,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 363.00995954954925,
|
||||
"ttft_p90_ms": 369.09779999405146
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 5769.497427325912,
|
||||
"e2e_p90_ms": 5888.945628073998,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 21.187139407074238,
|
||||
"tpot_p90_ms": 21.653656356547977,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 366.7432916322386,
|
||||
"ttft_p90_ms": 369.99153497163206
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 7486.7986220007915,
|
||||
"e2e_p90_ms": 7901.13743103575,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 27.994155607374484,
|
||||
"tpot_p90_ms": 29.63612463562658,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 348.25438050341427,
|
||||
"ttft_p90_ms": 365.8816539682448
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 7484.121415326708,
|
||||
"e2e_p90_ms": 7908.651847974397,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 27.978544004773987,
|
||||
"tpot_p90_ms": 29.650389521346227,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 349.5577253122725,
|
||||
"ttft_p90_ms": 365.11919100303203
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 19261.684938205322,
|
||||
"e2e_p50_ms": 19681.421068277567,
|
||||
"e2e_p90_ms": 19712.678973231064,
|
||||
"e2e_p95_ms": 19717.18006692948,
|
||||
"tpot_mean_ms": 74.22292738142716,
|
||||
"tpot_p50_ms": 75.8976524911963,
|
||||
"tpot_p90_ms": 75.8976524912086,
|
||||
"tpot_p95_ms": 75.89765249120967,
|
||||
"ttft_mean_ms": 334.8384559413977,
|
||||
"ttft_p50_ms": 334.75193988430976,
|
||||
"ttft_p90_ms": 359.7779859328796,
|
||||
"ttft_p95_ms": 363.2786816744229
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 19261.684938205322,
|
||||
"e2e_p50_ms": 19681.421068277567,
|
||||
"e2e_p90_ms": 19712.678973231064,
|
||||
"e2e_p95_ms": 19717.18006692948,
|
||||
"tpot_mean_ms": 74.22292738142716,
|
||||
"tpot_p50_ms": 75.8976524911963,
|
||||
"tpot_p90_ms": 75.8976524912086,
|
||||
"tpot_p95_ms": 75.89765249120967,
|
||||
"ttft_mean_ms": 334.8384559413977,
|
||||
"ttft_p50_ms": 334.75193988430976,
|
||||
"ttft_p90_ms": 359.7779859328796,
|
||||
"ttft_p95_ms": 363.2786816744229
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 14056.464119438264,
|
||||
"e2e_p50_ms": 14650.653983203298,
|
||||
"e2e_p90_ms": 14674.997135818061,
|
||||
"e2e_p95_ms": 14677.859760866291,
|
||||
"tpot_mean_ms": 54.15667269956665,
|
||||
"tpot_p50_ms": 56.52350191837838,
|
||||
"tpot_p90_ms": 56.52350850045728,
|
||||
"tpot_p95_ms": 56.523546100039816,
|
||||
"ttft_mean_ms": 246.51258104876885,
|
||||
"ttft_p50_ms": 246.7781280760306,
|
||||
"ttft_p90_ms": 263.0087113349084,
|
||||
"ttft_p95_ms": 264.77576248352364
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 14056.464119438264,
|
||||
"e2e_p50_ms": 14650.653983203298,
|
||||
"e2e_p90_ms": 14674.997135818061,
|
||||
"e2e_p95_ms": 14677.859760866291,
|
||||
"tpot_mean_ms": 54.15667269956665,
|
||||
"tpot_p50_ms": 56.52350191837838,
|
||||
"tpot_p90_ms": 56.52350850045728,
|
||||
"tpot_p95_ms": 56.523546100039816,
|
||||
"ttft_mean_ms": 246.51258104876885,
|
||||
"ttft_p50_ms": 246.7781280760306,
|
||||
"ttft_p90_ms": 263.0087113349084,
|
||||
"ttft_p95_ms": 264.77576248352364
|
||||
}
|
||||
}
|
||||
},
|
||||
"fixed-po": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 345.40159726161653,
|
||||
"e2e_p90_ms": 346.7378600034863,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 345.35322551680514,
|
||||
"ttft_p90_ms": 346.6896569589153
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 345.8011649264437,
|
||||
"e2e_p90_ms": 347.9283060878515,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 345.75379653310387,
|
||||
"ttft_p90_ms": 347.88418107200414
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 326.2389304512842,
|
||||
"e2e_p90_ms": 341.0220619989559,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 326.19059710668336,
|
||||
"ttft_p90_ms": 340.98013502079993
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 326.89427194042366,
|
||||
"e2e_p90_ms": 341.91650594584644,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 326.846390089937,
|
||||
"ttft_p90_ms": 341.87235799618065
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 303.2842141052072,
|
||||
"e2e_p50_ms": 303.28421410520434,
|
||||
"e2e_p90_ms": 303.28421410521855,
|
||||
"e2e_p95_ms": 303.28421410521855,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 303.2842141052072,
|
||||
"ttft_p50_ms": 303.28421410520434,
|
||||
"ttft_p90_ms": 303.28421410521855,
|
||||
"ttft_p95_ms": 303.28421410521855
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 303.2842141052072,
|
||||
"e2e_p50_ms": 303.28421410520434,
|
||||
"e2e_p90_ms": 303.28421410521855,
|
||||
"e2e_p95_ms": 303.28421410521855,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 303.2842141052072,
|
||||
"ttft_p50_ms": 303.28421410520434,
|
||||
"ttft_p90_ms": 303.28421410521855,
|
||||
"ttft_p95_ms": 303.28421410521855
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 224.0004422359394,
|
||||
"e2e_p50_ms": 224.00044223590854,
|
||||
"e2e_p90_ms": 224.00044223607017,
|
||||
"e2e_p95_ms": 224.00044223607017,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 224.0004422359394,
|
||||
"ttft_p50_ms": 224.00044223590854,
|
||||
"ttft_p90_ms": 224.00044223607017,
|
||||
"ttft_p95_ms": 224.00044223607017
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 224.0004422359394,
|
||||
"e2e_p50_ms": 224.00044223590854,
|
||||
"e2e_p90_ms": 224.00044223607017,
|
||||
"e2e_p95_ms": 224.00044223607017,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 224.0004422359394,
|
||||
"ttft_p50_ms": 224.00044223590854,
|
||||
"ttft_p90_ms": 224.00044223607017,
|
||||
"ttft_p95_ms": 224.00044223607017
|
||||
}
|
||||
}
|
||||
},
|
||||
"trace-pd": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.8333333333333334,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.006154214824712234,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.8333333333333334,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.062164651568153984,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns64",
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_p90_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns64",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 203026.22048668287,
|
||||
"e2e_p90_ms": 344183.7428959552,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 58.09653520572589,
|
||||
"tpot_p90_ms": 75.82137218925735,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 26487.547554662306,
|
||||
"ttft_p90_ms": 93044.83354603872
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 198593.25371425177,
|
||||
"e2e_p90_ms": 332386.17597706616,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 53.60856802543236,
|
||||
"tpot_p90_ms": 64.87020396877685,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 31557.98602907994,
|
||||
"ttft_p90_ms": 112666.82129900437
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 182457.35910624202,
|
||||
"e2e_p90_ms": 318000.89644105174,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 61.88514020253078,
|
||||
"tpot_p90_ms": 78.6175951982538,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1141.539354895564,
|
||||
"ttft_p90_ms": 2616.6939180111513
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 181341.34550937495,
|
||||
"e2e_p90_ms": 299389.4552709535,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 45.89371812653489,
|
||||
"tpot_p90_ms": 51.37046630467056,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 37215.26084531421,
|
||||
"ttft_p90_ms": 132333.42359995004
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 350485.31604335306,
|
||||
"e2e_p50_ms": 329886.2419457557,
|
||||
"e2e_p90_ms": 618683.6003279199,
|
||||
"e2e_p95_ms": 719028.7082294676,
|
||||
"tpot_mean_ms": 92.88846066050185,
|
||||
"tpot_p50_ms": 91.14780565610876,
|
||||
"tpot_p90_ms": 104.48481783284998,
|
||||
"tpot_p95_ms": 117.7639165127043,
|
||||
"ttft_mean_ms": 51997.2508135414,
|
||||
"ttft_p50_ms": 915.3872825012286,
|
||||
"ttft_p90_ms": 175964.46056759413,
|
||||
"ttft_p95_ms": 254194.45368846075
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 350205.0884431818,
|
||||
"e2e_p50_ms": 345577.391630324,
|
||||
"e2e_p90_ms": 612408.2764512596,
|
||||
"e2e_p95_ms": 709831.5675796239,
|
||||
"tpot_mean_ms": 86.77595832694777,
|
||||
"tpot_p50_ms": 87.01966249041537,
|
||||
"tpot_p90_ms": 95.27552612769624,
|
||||
"tpot_p95_ms": 98.22236811591964,
|
||||
"ttft_mean_ms": 64276.51238724817,
|
||||
"ttft_p50_ms": 2913.567965463905,
|
||||
"ttft_p90_ms": 231348.6370051864,
|
||||
"ttft_p95_ms": 250924.94176651328
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 197537.45276613487,
|
||||
"e2e_p50_ms": 184538.43666142275,
|
||||
"e2e_p90_ms": 356508.87615749374,
|
||||
"e2e_p95_ms": 426676.71753314673,
|
||||
"tpot_mean_ms": 62.78094439594437,
|
||||
"tpot_p50_ms": 61.98322714624318,
|
||||
"tpot_p90_ms": 73.38270919472093,
|
||||
"tpot_p95_ms": 79.82017533221833,
|
||||
"ttft_mean_ms": 496.4562724801789,
|
||||
"ttft_p50_ms": 258.18551545088656,
|
||||
"ttft_p90_ms": 1185.2339118913733,
|
||||
"ttft_p95_ms": 1563.6528594879735
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 230232.00122833299,
|
||||
"e2e_p50_ms": 225231.54242404614,
|
||||
"e2e_p90_ms": 399325.5400221139,
|
||||
"e2e_p95_ms": 451116.17307457025,
|
||||
"tpot_mean_ms": 56.01681518539994,
|
||||
"tpot_p50_ms": 56.94928232211729,
|
||||
"tpot_p90_ms": 60.38839798597023,
|
||||
"tpot_p95_ms": 62.67163079918898,
|
||||
"ttft_mean_ms": 46908.98176860714,
|
||||
"ttft_p50_ms": 1623.170360660838,
|
||||
"ttft_p90_ms": 166051.82069777363,
|
||||
"ttft_p95_ms": 181013.0283352888
|
||||
}
|
||||
}
|
||||
},
|
||||
"trace-po": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp4_ep1_mns128"
|
||||
],
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0137482658615673,
|
||||
"sim_winner": "tp8_ep8_mns64",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.0025194410024500513,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp4_ep1_mns128"
|
||||
],
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.013750984200462835,
|
||||
"sim_winner": "tp8_ep8_mns64",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.002607797240047338,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 1083.6389664896267,
|
||||
"e2e_p90_ms": 3156.966591021046,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1083.561832460368,
|
||||
"ttft_p90_ms": 3156.9307829486206
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 1102.0341126990652,
|
||||
"e2e_p90_ms": 3162.989347940311,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1101.9558890756384,
|
||||
"ttft_p90_ms": 3162.9534389358014
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 1029.581229927413,
|
||||
"e2e_p90_ms": 2612.0267439400777,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1029.4865175674956,
|
||||
"ttft_p90_ms": 2611.9744409807026
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 1043.7361864025345,
|
||||
"e2e_p90_ms": 2605.4624350508675,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1043.6429704051557,
|
||||
"ttft_p90_ms": 2605.180658050813
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 698.2455408721717,
|
||||
"e2e_p50_ms": 208.8217362068434,
|
||||
"e2e_p90_ms": 2102.42229741975,
|
||||
"e2e_p95_ms": 2868.6579248910534,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 698.2455408721717,
|
||||
"ttft_p50_ms": 208.8217362068434,
|
||||
"ttft_p90_ms": 2102.42229741975,
|
||||
"ttft_p95_ms": 2868.6579248910534
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 698.0132356513792,
|
||||
"e2e_p50_ms": 208.8217362068434,
|
||||
"e2e_p90_ms": 2102.42229741975,
|
||||
"e2e_p95_ms": 2868.6579248910534,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 698.0132356513792,
|
||||
"ttft_p50_ms": 208.8217362068434,
|
||||
"ttft_p90_ms": 2102.42229741975,
|
||||
"ttft_p95_ms": 2868.6579248910534
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 487.42679595918156,
|
||||
"e2e_p50_ms": 253.39288448911512,
|
||||
"e2e_p90_ms": 1264.1466431938486,
|
||||
"e2e_p95_ms": 1656.1837032209396,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 487.42679595918156,
|
||||
"ttft_p50_ms": 253.39288448911512,
|
||||
"ttft_p90_ms": 1264.1466431938486,
|
||||
"ttft_p95_ms": 1656.1837032209396
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 487.3699604726311,
|
||||
"e2e_p50_ms": 253.41330144894124,
|
||||
"e2e_p90_ms": 1264.1466431938486,
|
||||
"e2e_p95_ms": 1656.1837032209396,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 487.3699604726311,
|
||||
"ttft_p50_ms": 253.41330144894124,
|
||||
"ttft_p90_ms": 1264.1466431938486,
|
||||
"ttft_p95_ms": 1656.1837032209396
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"real_root": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1",
|
||||
"schema": "qwen235-v020-simulator-real-comparison-v1",
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
# Qwen235 vLLM 0.20 Frontier vs real
|
||||
|
||||
| case | metric | Frontier winner | real winner | match | regret | pair agreement |
|
||||
|---|---|---|---|---:|---:|---:|
|
||||
| fixed-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% |
|
||||
| fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.2% | 100.0% |
|
||||
| fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 33.0% | 0.0% |
|
||||
| fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 37.2% | 0.0% |
|
||||
| fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 30.7% | 0.0% |
|
||||
| fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 34.6% | 0.0% |
|
||||
| fixed-po | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% |
|
||||
| fixed-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% |
|
||||
| fixed-po | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% |
|
||||
| fixed-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% |
|
||||
| trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.6% | 83.3% |
|
||||
| trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 6.2% | 83.3% |
|
||||
| trace-po | ttft_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | no | 1.4% | 66.7% |
|
||||
| trace-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.3% | 100.0% |
|
||||
| trace-po | e2e_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | no | 1.4% | 66.7% |
|
||||
| trace-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.3% | 100.0% |
|
||||
@@ -0,0 +1,952 @@
|
||||
{
|
||||
"cases": {
|
||||
"fixed-pd": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.3067524317507717,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.3456375801249614,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.33025510615656906,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp4_ep1_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.37228890038064755,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.04237011756982145,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 1.0,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [],
|
||||
"selected_real_regret": 0.002088257708717345,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 5729.31677040774,
|
||||
"e2e_p90_ms": 5871.668232022785,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 21.044200828709027,
|
||||
"tpot_p90_ms": 21.59612646244247,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 363.00995954954925,
|
||||
"ttft_p90_ms": 369.09779999405146
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 5769.497427325912,
|
||||
"e2e_p90_ms": 5888.945628073998,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 21.187139407074238,
|
||||
"tpot_p90_ms": 21.653656356547977,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 366.7432916322386,
|
||||
"ttft_p90_ms": 369.99153497163206
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 7486.7986220007915,
|
||||
"e2e_p90_ms": 7901.13743103575,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 27.994155607374484,
|
||||
"tpot_p90_ms": 29.63612463562658,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 348.25438050341427,
|
||||
"ttft_p90_ms": 365.8816539682448
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 7484.121415326708,
|
||||
"e2e_p90_ms": 7908.651847974397,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 27.978544004773987,
|
||||
"tpot_p90_ms": 29.650389521346227,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 349.5577253122725,
|
||||
"ttft_p90_ms": 365.11919100303203
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 22734.095381397616,
|
||||
"e2e_p50_ms": 23441.595611495926,
|
||||
"e2e_p90_ms": 23484.306701188103,
|
||||
"e2e_p95_ms": 23489.511452611594,
|
||||
"tpot_mean_ms": 87.77349881741111,
|
||||
"tpot_p50_ms": 90.6067822139126,
|
||||
"tpot_p90_ms": 90.60678221391389,
|
||||
"tpot_p95_ms": 90.60678221391389,
|
||||
"ttft_mean_ms": 351.8531829577845,
|
||||
"ttft_p50_ms": 351.44748294051453,
|
||||
"ttft_p90_ms": 381.14669406792245,
|
||||
"ttft_p95_ms": 385.3185272085611
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 22734.095381397616,
|
||||
"e2e_p50_ms": 23441.595611495926,
|
||||
"e2e_p90_ms": 23484.306701188103,
|
||||
"e2e_p95_ms": 23489.511452611594,
|
||||
"tpot_mean_ms": 87.77349881741111,
|
||||
"tpot_p50_ms": 90.6067822139126,
|
||||
"tpot_p90_ms": 90.60678221391389,
|
||||
"tpot_p95_ms": 90.60678221391389,
|
||||
"ttft_mean_ms": 351.8531829577845,
|
||||
"ttft_p50_ms": 351.44748294051453,
|
||||
"ttft_p90_ms": 381.14669406792245,
|
||||
"ttft_p95_ms": 385.3185272085611
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 16059.899811799207,
|
||||
"e2e_p50_ms": 17256.45308650381,
|
||||
"e2e_p90_ms": 17280.553823012277,
|
||||
"e2e_p95_ms": 17282.746891876577,
|
||||
"tpot_mean_ms": 61.58781555229474,
|
||||
"tpot_p50_ms": 66.32324900760649,
|
||||
"tpot_p90_ms": 66.3246446898248,
|
||||
"tpot_p95_ms": 66.3246446898248,
|
||||
"ttft_mean_ms": 355.00684596404665,
|
||||
"ttft_p50_ms": 355.16695480338,
|
||||
"ttft_p90_ms": 368.8582878708644,
|
||||
"ttft_p95_ms": 370.503089520021
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 16059.899811799207,
|
||||
"e2e_p50_ms": 17256.45308650381,
|
||||
"e2e_p90_ms": 17280.553823012277,
|
||||
"e2e_p95_ms": 17282.746891876577,
|
||||
"tpot_mean_ms": 61.58781555229474,
|
||||
"tpot_p50_ms": 66.32324900760649,
|
||||
"tpot_p90_ms": 66.3246446898248,
|
||||
"tpot_p95_ms": 66.3246446898248,
|
||||
"ttft_mean_ms": 355.00684596404665,
|
||||
"ttft_p50_ms": 355.16695480338,
|
||||
"ttft_p90_ms": 368.8582878708644,
|
||||
"ttft_p95_ms": 370.503089520021
|
||||
}
|
||||
}
|
||||
},
|
||||
"fixed-po": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.05873813644442971,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.016760786592592503,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.058746722254088946,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.016744441542810407,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 345.40159726161653,
|
||||
"e2e_p90_ms": 346.7378600034863,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 345.35322551680514,
|
||||
"ttft_p90_ms": 346.6896569589153
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 345.8011649264437,
|
||||
"e2e_p90_ms": 347.9283060878515,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 345.75379653310387,
|
||||
"ttft_p90_ms": 347.88418107200414
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 326.2389304512842,
|
||||
"e2e_p90_ms": 341.0220619989559,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 326.19059710668336,
|
||||
"ttft_p90_ms": 340.98013502079993
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 326.89427194042366,
|
||||
"e2e_p90_ms": 341.91650594584644,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 326.846390089937,
|
||||
"ttft_p90_ms": 341.87235799618065
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 313.84871569626137,
|
||||
"e2e_p50_ms": 313.848715696261,
|
||||
"e2e_p90_ms": 313.8487156962668,
|
||||
"e2e_p95_ms": 313.8487156962668,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 313.84871569626137,
|
||||
"ttft_p50_ms": 313.848715696261,
|
||||
"ttft_p90_ms": 313.8487156962668,
|
||||
"ttft_p95_ms": 313.8487156962668
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 313.84871569626137,
|
||||
"e2e_p50_ms": 313.848715696261,
|
||||
"e2e_p90_ms": 313.8487156962668,
|
||||
"e2e_p95_ms": 313.8487156962668,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 313.84871569626137,
|
||||
"ttft_p50_ms": 313.848715696261,
|
||||
"ttft_p90_ms": 313.8487156962668,
|
||||
"ttft_p95_ms": 313.8487156962668
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 334.2963928223074,
|
||||
"e2e_p50_ms": 334.2963928222744,
|
||||
"e2e_p90_ms": 334.2963928224325,
|
||||
"e2e_p95_ms": 334.2963928224325,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 334.2963928223074,
|
||||
"ttft_p50_ms": 334.2963928222744,
|
||||
"ttft_p90_ms": 334.2963928224325,
|
||||
"ttft_p95_ms": 334.2963928224325
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 334.2963928223074,
|
||||
"e2e_p50_ms": 334.2963928222744,
|
||||
"e2e_p90_ms": 334.2963928224325,
|
||||
"e2e_p95_ms": 334.2963928224325,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 334.2963928223074,
|
||||
"ttft_p50_ms": 334.2963928222744,
|
||||
"ttft_p90_ms": 334.2963928224325,
|
||||
"ttft_p95_ms": 334.2963928224325
|
||||
}
|
||||
}
|
||||
},
|
||||
"trace-pd": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.8333333333333334,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.006154214824712234,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp4_ep1_mns128"
|
||||
],
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.062164651568153984,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns64",
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_p90_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns64",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.6666666666666666,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.0,
|
||||
"sim_winner": "tp8_ep8_mns128",
|
||||
"winner_match": true
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 203026.22048668287,
|
||||
"e2e_p90_ms": 344183.7428959552,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 58.09653520572589,
|
||||
"tpot_p90_ms": 75.82137218925735,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 26487.547554662306,
|
||||
"ttft_p90_ms": 93044.83354603872
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 198593.25371425177,
|
||||
"e2e_p90_ms": 332386.17597706616,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 53.60856802543236,
|
||||
"tpot_p90_ms": 64.87020396877685,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 31557.98602907994,
|
||||
"ttft_p90_ms": 112666.82129900437
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 182457.35910624202,
|
||||
"e2e_p90_ms": 318000.89644105174,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 61.88514020253078,
|
||||
"tpot_p90_ms": 78.6175951982538,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1141.539354895564,
|
||||
"ttft_p90_ms": 2616.6939180111513
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 181341.34550937495,
|
||||
"e2e_p90_ms": 299389.4552709535,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": 45.89371812653489,
|
||||
"tpot_p90_ms": 51.37046630467056,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 37215.26084531421,
|
||||
"ttft_p90_ms": 132333.42359995004
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 385157.8895230749,
|
||||
"e2e_p50_ms": 367024.8243163692,
|
||||
"e2e_p90_ms": 673951.1520207045,
|
||||
"e2e_p95_ms": 785718.9717121953,
|
||||
"tpot_mean_ms": 100.38179934319204,
|
||||
"tpot_p50_ms": 100.42433178975269,
|
||||
"tpot_p90_ms": 113.67517808832463,
|
||||
"tpot_p95_ms": 121.63558651348922,
|
||||
"ttft_mean_ms": 59067.296381594686,
|
||||
"ttft_p50_ms": 908.4160127742393,
|
||||
"ttft_p90_ms": 200789.2407497713,
|
||||
"ttft_p95_ms": 289881.2920798696
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 380604.1804371099,
|
||||
"e2e_p50_ms": 378158.20614125655,
|
||||
"e2e_p90_ms": 678448.1689084907,
|
||||
"e2e_p95_ms": 763969.1713963022,
|
||||
"tpot_mean_ms": 93.54579486627527,
|
||||
"tpot_p50_ms": 92.86650455014843,
|
||||
"tpot_p90_ms": 102.66261969117096,
|
||||
"tpot_p95_ms": 112.87824516729198,
|
||||
"ttft_mean_ms": 72259.33174260205,
|
||||
"ttft_p50_ms": 2897.238841988653,
|
||||
"ttft_p90_ms": 265395.14114254655,
|
||||
"ttft_p95_ms": 273163.7753733101
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 174939.1805624549,
|
||||
"e2e_p50_ms": 166723.75338099696,
|
||||
"e2e_p90_ms": 317439.7347771481,
|
||||
"e2e_p95_ms": 371471.8509246631,
|
||||
"tpot_mean_ms": 58.37204729277776,
|
||||
"tpot_p50_ms": 54.29394782230334,
|
||||
"tpot_p90_ms": 75.91440592454555,
|
||||
"tpot_p95_ms": 84.58610544605006,
|
||||
"ttft_mean_ms": 945.9090659083846,
|
||||
"ttft_p50_ms": 669.9533543429972,
|
||||
"ttft_p90_ms": 2348.607298432925,
|
||||
"ttft_p95_ms": 2781.46390848989
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 208352.4497977196,
|
||||
"e2e_p50_ms": 207573.22184026084,
|
||||
"e2e_p90_ms": 353236.8601573669,
|
||||
"e2e_p95_ms": 402705.2991150841,
|
||||
"tpot_mean_ms": 52.06723645713691,
|
||||
"tpot_p50_ms": 51.64653531173072,
|
||||
"tpot_p90_ms": 60.374786839964116,
|
||||
"tpot_p95_ms": 62.94158800335912,
|
||||
"ttft_mean_ms": 43679.663681651284,
|
||||
"ttft_p50_ms": 2772.0384805371,
|
||||
"ttft_p90_ms": 151027.16878398336,
|
||||
"ttft_p95_ms": 165014.86227423933
|
||||
}
|
||||
}
|
||||
},
|
||||
"trace-po": {
|
||||
"comparison": {
|
||||
"e2e_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp4_ep1_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.07037121566091509,
|
||||
"sim_winner": "tp4_ep1_mns64",
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.21167227304868486,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_mean_ms": {
|
||||
"informative_pairs": 6,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns128",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp4_ep1_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp8_ep8_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.07039370625209918,
|
||||
"sim_winner": "tp4_ep1_mns64",
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_p90_ms": {
|
||||
"informative_pairs": 4,
|
||||
"pair_direction_agreement": 0.0,
|
||||
"real_winner": "tp8_ep8_mns64",
|
||||
"reversals": [
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns64",
|
||||
"tp8_ep8_mns128"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns64"
|
||||
],
|
||||
[
|
||||
"tp4_ep1_mns128",
|
||||
"tp8_ep8_mns128"
|
||||
]
|
||||
],
|
||||
"selected_real_regret": 0.2117895828808376,
|
||||
"sim_winner": "tp4_ep1_mns128",
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"real": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 1083.6389664896267,
|
||||
"e2e_p90_ms": 3156.966591021046,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1083.561832460368,
|
||||
"ttft_p90_ms": 3156.9307829486206
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 1102.0341126990652,
|
||||
"e2e_p90_ms": 3162.989347940311,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1101.9558890756384,
|
||||
"ttft_p90_ms": 3162.9534389358014
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 1029.581229927413,
|
||||
"e2e_p90_ms": 2612.0267439400777,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1029.4865175674956,
|
||||
"ttft_p90_ms": 2611.9744409807026
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 1043.7361864025345,
|
||||
"e2e_p90_ms": 2605.4624350508675,
|
||||
"request_samples": 387,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"trials": [
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial1/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial2/results/result.json",
|
||||
"/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial3/results/result.json"
|
||||
],
|
||||
"ttft_mean_ms": 1043.6429704051557,
|
||||
"ttft_p90_ms": 2605.180658050813
|
||||
}
|
||||
},
|
||||
"sim": {
|
||||
"tp4_ep1_mns128": {
|
||||
"e2e_mean_ms": 691.7873605232029,
|
||||
"e2e_p50_ms": 211.81272326305088,
|
||||
"e2e_p90_ms": 2068.417017674378,
|
||||
"e2e_p95_ms": 2843.8890088954663,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 691.7873605232029,
|
||||
"ttft_p50_ms": 211.81272326305088,
|
||||
"ttft_p90_ms": 2068.417017674378,
|
||||
"ttft_p95_ms": 2843.8890088954663
|
||||
},
|
||||
"tp4_ep1_mns64": {
|
||||
"e2e_mean_ms": 691.5722223601863,
|
||||
"e2e_p50_ms": 210.89579132517144,
|
||||
"e2e_p90_ms": 2068.417017674378,
|
||||
"e2e_p95_ms": 2843.8890088954663,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 691.5722223601863,
|
||||
"ttft_p50_ms": 210.89579132517144,
|
||||
"ttft_p90_ms": 2068.417017674378,
|
||||
"ttft_p95_ms": 2843.8890088954663
|
||||
},
|
||||
"tp8_ep8_mns128": {
|
||||
"e2e_mean_ms": 928.7998403823643,
|
||||
"e2e_p50_ms": 593.0899264387222,
|
||||
"e2e_p90_ms": 2341.4108333476474,
|
||||
"e2e_p95_ms": 2898.493085937801,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 928.7998403823643,
|
||||
"ttft_p50_ms": 593.0899264387222,
|
||||
"ttft_p90_ms": 2341.4108333476474,
|
||||
"ttft_p95_ms": 2898.493085937801
|
||||
},
|
||||
"tp8_ep8_mns64": {
|
||||
"e2e_mean_ms": 928.7182564423873,
|
||||
"e2e_p50_ms": 593.0899264387222,
|
||||
"e2e_p90_ms": 2341.4108333476474,
|
||||
"e2e_p95_ms": 2898.493085937801,
|
||||
"tpot_mean_ms": null,
|
||||
"tpot_p50_ms": null,
|
||||
"tpot_p90_ms": null,
|
||||
"tpot_p95_ms": null,
|
||||
"ttft_mean_ms": 928.7182564423873,
|
||||
"ttft_p50_ms": 593.0899264387222,
|
||||
"ttft_p90_ms": 2341.4108333476474,
|
||||
"ttft_p95_ms": 2898.493085937801
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"schema": "qwen235-v020-simulator-real-comparison-v1"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
# Qwen235 vLLM 0.20 Frontier vs real
|
||||
|
||||
| case | metric | Frontier winner | real winner | match | regret | pair agreement |
|
||||
|---|---|---|---|---:|---:|---:|
|
||||
| fixed-pd | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 4.2% | 0.0% |
|
||||
| fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.2% | 100.0% |
|
||||
| fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 33.0% | 0.0% |
|
||||
| fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 37.2% | 0.0% |
|
||||
| fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 30.7% | 0.0% |
|
||||
| fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 34.6% | 0.0% |
|
||||
| fixed-po | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 5.9% | 0.0% |
|
||||
| fixed-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 1.7% | 0.0% |
|
||||
| fixed-po | e2e_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 5.9% | 0.0% |
|
||||
| fixed-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 1.7% | 0.0% |
|
||||
| trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% |
|
||||
| trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.6% | 83.3% |
|
||||
| trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 6.2% | 66.7% |
|
||||
| trace-po | ttft_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | no | 7.0% | 0.0% |
|
||||
| trace-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | no | 21.2% | 0.0% |
|
||||
| trace-po | e2e_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | no | 7.0% | 0.0% |
|
||||
| trace-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | no | 21.2% | 0.0% |
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,54 @@
|
||||
# Qwen235 Fixed-PD state diagnosis
|
||||
|
||||
**Verdict:** Exact real state composition flips Frontier to the correct TP4 winner; the ranking error is caused by closed-loop state/composition mismatch, not a conditional stage reversal.
|
||||
|
||||
| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage |
|
||||
|---|---:|---:|---:|
|
||||
| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% |
|
||||
| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% |
|
||||
|
||||
| Config | Real exact decode batch mean | Real token-weighted iteration ms | Matched simulator ms | Exact coverage |
|
||||
|---|---:|---:|---:|---:|
|
||||
| tp4_ep1_mns64 | 3.890 | 18.4710 | 34.2575 | 100.0% |
|
||||
| tp8_ep8_mns64 | 10.921 | 27.5887 | 45.1603 | 100.0% |
|
||||
|
||||
## Frontier internal component contrast over its own executed composition
|
||||
|
||||
Decode-token-weighted over both pure-decode and mixed prefill/decode steps.
|
||||
|
||||
| Component | TP8 - TP4 (ms/decoded token step) |
|
||||
|---|---:|
|
||||
| moe_compute | -32.4544 |
|
||||
| total | -20.0663 |
|
||||
| attention | -7.4619 |
|
||||
| tp_dp_communication | -6.5711 |
|
||||
| ep_communication | +6.2354 |
|
||||
| moe_routing | -0.2862 |
|
||||
| dense_mlp_compute | -0.1231 |
|
||||
| pipeline_communication | +0.0000 |
|
||||
| runtime_overhead | +0.0000 |
|
||||
|
||||
## Frontier component contrast at exact real token composition
|
||||
|
||||
Positive means TP8 slower; negative means Frontier gives TP8 an advantage.
|
||||
|
||||
| Component | TP8 - TP4 (ms/step) |
|
||||
|---|---:|
|
||||
| total | +10.9028 |
|
||||
| ep_communication | +6.5355 |
|
||||
| tp_dp_communication | -6.4718 |
|
||||
| attention | -2.1304 |
|
||||
| moe_compute | -1.9774 |
|
||||
| moe_routing | +0.1384 |
|
||||
| dense_mlp_compute | -0.0308 |
|
||||
| pipeline_communication | +0.0000 |
|
||||
| runtime_overhead | +0.0000 |
|
||||
|
||||
## Interpretation boundary
|
||||
|
||||
- Real observed TPOT contrast: +6.9500 ms/token.
|
||||
- Real exact-iteration contrast: +9.1177 ms/decode-token-weighted step.
|
||||
- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs.
|
||||
- Frontier state outputs have no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors.
|
||||
- For TP8 shared-domain sync, total is ledger stage_end-start (critical path); op categories are serialized work estimates and are not additive because overlap and lane-summed wait diagnostics are represented separately.
|
||||
- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,48 @@
|
||||
# Qwen235 Fixed-PD state diagnosis
|
||||
|
||||
**Verdict:** State-composition mismatch is insufficient: after reweighting Frontier to measured real decode composition, it still predicts TP8 faster.
|
||||
|
||||
| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage |
|
||||
|---|---:|---:|---:|
|
||||
| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% |
|
||||
| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% |
|
||||
|
||||
## Frontier internal component contrast over its own executed composition
|
||||
|
||||
Decode-token-weighted over both pure-decode and mixed prefill/decode steps.
|
||||
|
||||
| Component | TP8 - TP4 (ms/decoded token step) |
|
||||
|---|---:|
|
||||
| moe_compute | -32.4544 |
|
||||
| total | -20.0663 |
|
||||
| attention | -7.4619 |
|
||||
| tp_dp_communication | -6.5711 |
|
||||
| ep_communication | +6.2354 |
|
||||
| moe_routing | -0.2862 |
|
||||
| dense_mlp_compute | -0.1231 |
|
||||
| pipeline_communication | +0.0000 |
|
||||
| runtime_overhead | +0.0000 |
|
||||
|
||||
## Frontier internal component contrast at real Running proxy
|
||||
|
||||
Positive means TP8 slower; negative means Frontier gives TP8 an advantage.
|
||||
|
||||
| Component | TP8 - TP4 (ms/step) |
|
||||
|---|---:|
|
||||
| moe_compute | -16.0581 |
|
||||
| total | -8.5517 |
|
||||
| ep_communication | +6.8248 |
|
||||
| tp_dp_communication | -5.4010 |
|
||||
| moe_routing | +0.3109 |
|
||||
| attention | -0.2407 |
|
||||
| dense_mlp_compute | +0.0018 |
|
||||
| pipeline_communication | +0.0000 |
|
||||
| runtime_overhead | +0.0000 |
|
||||
|
||||
## Interpretation boundary
|
||||
|
||||
- Real observed TPOT contrast: +6.9500 ms/token.
|
||||
- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs.
|
||||
- Frontier state outputs have no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors.
|
||||
- For TP8 shared-domain sync, total is ledger stage_end-start (critical path); op categories are serialized work estimates and are not additive because overlap and lane-summed wait diagnostics are represented separately.
|
||||
- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,47 @@
|
||||
# Qwen235 Fixed-PD state diagnosis
|
||||
|
||||
**Verdict:** Active-batch-count mismatch alone is insufficient: after exact reweighting to each config's real Running histogram, Frontier still predicts TP8 faster.
|
||||
|
||||
| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage |
|
||||
|---|---:|---:|---:|
|
||||
| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% |
|
||||
| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% |
|
||||
|
||||
## Frontier internal component contrast over its own executed composition
|
||||
|
||||
Decode-token-weighted over both pure-decode and mixed prefill/decode steps.
|
||||
|
||||
| Component | TP8 - TP4 (ms/decoded token step) |
|
||||
|---|---:|
|
||||
| total | -73.6733 |
|
||||
| moe_compute | -45.2710 |
|
||||
| attention | -13.6950 |
|
||||
| tp_dp_communication | -12.7244 |
|
||||
| moe_routing | -1.7759 |
|
||||
| dense_mlp_compute | -0.2890 |
|
||||
| ep_communication | +0.0821 |
|
||||
| pipeline_communication | +0.0000 |
|
||||
| runtime_overhead | +0.0000 |
|
||||
|
||||
## Frontier internal component contrast at real Running proxy
|
||||
|
||||
Positive means TP8 slower; negative means Frontier gives TP8 an advantage.
|
||||
|
||||
| Component | TP8 - TP4 (ms/step) |
|
||||
|---|---:|
|
||||
| total | -46.0760 |
|
||||
| moe_compute | -26.3387 |
|
||||
| tp_dp_communication | -12.1532 |
|
||||
| attention | -6.2393 |
|
||||
| moe_routing | -1.2405 |
|
||||
| dense_mlp_compute | -0.1769 |
|
||||
| ep_communication | +0.0726 |
|
||||
| pipeline_communication | +0.0000 |
|
||||
| runtime_overhead | +0.0000 |
|
||||
|
||||
## Interpretation boundary
|
||||
|
||||
- Real observed TPOT contrast: +6.9500 ms/token.
|
||||
- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs.
|
||||
- Frontier ledger exposes additive execution components but no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors.
|
||||
- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof.
|
||||
@@ -0,0 +1,173 @@
|
||||
{
|
||||
"boundary": "This identifies where the selection error is created, but the frozen real logs do not attribute service-time overprediction to one operator.",
|
||||
"cells": {
|
||||
"tp2_mns64": {
|
||||
"arrival_rate_requests_per_s": 2.25,
|
||||
"mns": 64,
|
||||
"real": {
|
||||
"e2e_ms": 3354.8816545874283,
|
||||
"periodic_queue": {
|
||||
"running_max": 10,
|
||||
"running_mean": 7.257142857142857,
|
||||
"samples": 35,
|
||||
"waiting_max": 0,
|
||||
"waiting_mean": 0.0
|
||||
},
|
||||
"required_slots_upper_bound": 7.548483722821714,
|
||||
"slot_margin_lower_bound": 56.451516277178285,
|
||||
"tpot_ms": 12.393283087182931,
|
||||
"ttft_ms": 194.56193759030603
|
||||
},
|
||||
"simulator": {
|
||||
"first_scheduling_delay_ms": 38.22329108338011,
|
||||
"queue_free_ttft_ms": 191.02937373774816,
|
||||
"request_execution_ms": 25761.91467886552,
|
||||
"required_slots": 57.96430802744742,
|
||||
"slot_margin": 6.035691972552577,
|
||||
"tpot_ms": 100.27798158873513,
|
||||
"ttft_ms": 229.2526648211283,
|
||||
"ttft_queue_fraction": 0.16672997503956338
|
||||
},
|
||||
"tp": 2,
|
||||
"tpot_overprediction_ratio": 8.091316956395687
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"arrival_rate_requests_per_s": 4.5,
|
||||
"mns": 16,
|
||||
"real": {
|
||||
"e2e_ms": 3252.7993474094965,
|
||||
"periodic_queue": {
|
||||
"running_max": 16,
|
||||
"running_mean": 15.166666666666666,
|
||||
"samples": 18,
|
||||
"waiting_max": 0,
|
||||
"waiting_mean": 0.0
|
||||
},
|
||||
"required_slots_upper_bound": 14.637597063342735,
|
||||
"slot_margin_lower_bound": 1.362402936657265,
|
||||
"tpot_ms": 12.267737219892378,
|
||||
"ttft_ms": 124.49453797538061
|
||||
},
|
||||
"simulator": {
|
||||
"first_scheduling_delay_ms": 64906.25631809643,
|
||||
"queue_free_ttft_ms": 134.74066995065758,
|
||||
"request_execution_ms": 12108.627705569606,
|
||||
"required_slots": 54.488824675063235,
|
||||
"slot_margin": -38.488824675063235,
|
||||
"tpot_ms": 46.95641974752248,
|
||||
"ttft_ms": 65040.99698804708,
|
||||
"ttft_queue_fraction": 0.9979283732385681
|
||||
},
|
||||
"tp": 4,
|
||||
"tpot_overprediction_ratio": 3.8276349505907024
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"arrival_rate_requests_per_s": 4.5,
|
||||
"mns": 32,
|
||||
"real": {
|
||||
"e2e_ms": 3227.015463910283,
|
||||
"periodic_queue": {
|
||||
"running_max": 16,
|
||||
"running_mean": 14.944444444444445,
|
||||
"samples": 18,
|
||||
"waiting_max": 0,
|
||||
"waiting_mean": 0.0
|
||||
},
|
||||
"required_slots_upper_bound": 14.521569587596273,
|
||||
"slot_margin_lower_bound": 17.478430412403725,
|
||||
"tpot_ms": 12.171863019540417,
|
||||
"ttft_ms": 123.15859724408143
|
||||
},
|
||||
"simulator": {
|
||||
"first_scheduling_delay_ms": 27180.676177927482,
|
||||
"queue_free_ttft_ms": 135.62147333612225,
|
||||
"request_execution_ms": 14643.080360640608,
|
||||
"required_slots": 65.89386162288272,
|
||||
"slot_margin": -33.893861622882724,
|
||||
"tpot_ms": 56.89199563649036,
|
||||
"ttft_ms": 27316.2976512636,
|
||||
"ttft_queue_fraction": 0.9950351444010626
|
||||
},
|
||||
"tp": 4,
|
||||
"tpot_overprediction_ratio": 4.6740581573385525
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"arrival_rate_requests_per_s": 4.5,
|
||||
"mns": 64,
|
||||
"real": {
|
||||
"e2e_ms": 3258.202659738964,
|
||||
"periodic_queue": {
|
||||
"running_max": 17,
|
||||
"running_mean": 14.833333333333334,
|
||||
"samples": 18,
|
||||
"waiting_max": 0,
|
||||
"waiting_mean": 0.0
|
||||
},
|
||||
"required_slots_upper_bound": 14.661911968825338,
|
||||
"slot_margin_lower_bound": 49.338088031174664,
|
||||
"tpot_ms": 12.29409478300949,
|
||||
"ttft_ms": 123.17715882596677
|
||||
},
|
||||
"simulator": {
|
||||
"first_scheduling_delay_ms": 6367.273779218055,
|
||||
"queue_free_ttft_ms": 136.7304324624105,
|
||||
"request_execution_ms": 17736.957363112386,
|
||||
"required_slots": 79.81630813400574,
|
||||
"slot_margin": -15.816308134005737,
|
||||
"tpot_ms": 69.02049776725552,
|
||||
"ttft_ms": 6504.004211680465,
|
||||
"ttft_queue_fraction": 0.9789774993969319
|
||||
},
|
||||
"tp": 4,
|
||||
"tpot_overprediction_ratio": 5.614117914776633
|
||||
}
|
||||
},
|
||||
"contrasts": {
|
||||
"observed_sim_ttft_tp4_minus_tp2_ms": 27087.044986442474,
|
||||
"queue_free_sim_ttft_tp4_minus_tp2_ms": -55.40790040162591,
|
||||
"real_ttft_tp4_minus_tp2_ms": -71.4033403462246
|
||||
},
|
||||
"schema": "qwen30-fixed-pd-ttft-admission-diagnosis-v1",
|
||||
"selection": {
|
||||
"agreement": 0.5909090909090909,
|
||||
"concordant_pairs": 39,
|
||||
"discordant_pairs": 27,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.5797674051508896,
|
||||
"sim_ranking": [
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns64",
|
||||
"tp1_mns32",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp2_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": false
|
||||
},
|
||||
"status": "PASS",
|
||||
"verdict": "Frontier overpredicts decode service time, so TP4's higher global arrival rate crosses the MNS admission cap only in simulation. The resulting first-scheduling queue reverses the TP topology ranking; without that queue Frontier itself predicts TP4 faster than TP2, matching real hardware.",
|
||||
"workload": "Qwen3-30B Fixed-PD 4096->256 at 1.125 req/s/GPU"
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
# Qwen30 Fixed-PD TTFT admission diagnosis
|
||||
|
||||
**Verdict:** Frontier overpredicts decode service time, so TP4's higher global arrival rate crosses the MNS admission cap only in simulation. The resulting first-scheduling queue reverses the TP topology ranking; without that queue Frontier itself predicts TP4 faster than TP2, matching real hardware.
|
||||
|
||||
| Config | Sim TPOT / real | Sim required slots / MNS | Real slots upper bound / MNS | Sim first-schedule wait | Real waiting max |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| tp2_mns64 | 8.09x | 58.0/64 | 7.5/64 | 38.2 ms | 0 |
|
||||
| tp4_mns16 | 3.83x | 54.5/16 | 14.6/16 | 64906.3 ms | 0 |
|
||||
| tp4_mns32 | 4.67x | 65.9/32 | 14.5/32 | 27180.7 ms | 0 |
|
||||
| tp4_mns64 | 5.61x | 79.8/64 | 14.7/64 | 6367.3 ms | 0 |
|
||||
|
||||
## TP4/MNS32 minus TP2/MNS64 TTFT
|
||||
|
||||
- Observed Frontier: +27087.0 ms.
|
||||
- Frontier without first-scheduling queue: -55.4 ms.
|
||||
- Real hardware: -71.4 ms.
|
||||
|
||||
**Boundary:** This identifies where the selection error is created, but the frozen real logs do not attribute service-time overprediction to one operator.
|
||||
@@ -0,0 +1,2 @@
|
||||
Q30_ANALYSIS_ECHO source_real=/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6 source_sim=/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim cases=trace-po,fixed-pd,fixed-po requests=129 configs=12 trials=3 metrics=mean,p90(TTFT,TPOT-if-OSL-gt-1,E2E) gpu_cost=0 output=/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6
|
||||
Q30_ANALYSIS_COMPLETE
|
||||
@@ -0,0 +1,453 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"tpot_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"cells": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2624.2011811364387,
|
||||
"p90": 2624.2011811379484
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 19.350639886513438,
|
||||
"p90": 19.3506398865253
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns16/tp1/result.json"
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2624.2011811364387,
|
||||
"p90": 2624.2011811379484
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 19.350639886513438,
|
||||
"p90": 19.3506398865253
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns32/tp1/result.json"
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2624.2011811364387,
|
||||
"p90": 2624.2011811379484
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 19.350639886513438,
|
||||
"p90": 19.3506398865253
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns64/tp1/result.json"
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2624.2011811364387,
|
||||
"p90": 2624.2011811379484
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 19.350639886513438,
|
||||
"p90": 19.3506398865253
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns8/tp1/result.json"
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2808.3844446026824,
|
||||
"p90": 2817.9486849812747
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 21.06177881019329,
|
||||
"p90": 21.09350467849898
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 133.53853570813533,
|
||||
"p90": 142.1523942165379
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns16/tp2/result.json"
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2808.3844446026824,
|
||||
"p90": 2817.9486849812747
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 21.06177881019329,
|
||||
"p90": 21.09350467849898
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 133.53853570813533,
|
||||
"p90": 142.1523942165379
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns32/tp2/result.json"
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2808.3844446026824,
|
||||
"p90": 2817.9486849812747
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 21.06177881019329,
|
||||
"p90": 21.09350467849898
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 133.53853570813533,
|
||||
"p90": 142.1523942165379
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns64/tp2/result.json"
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 2808.3844446026824,
|
||||
"p90": 2817.9486849812747
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 21.06177881019329,
|
||||
"p90": 21.09350467849898
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 133.53853570813533,
|
||||
"p90": 142.1523942165379
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns8/tp2/result.json"
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 3752.684495467103,
|
||||
"p90": 3794.6349827560653
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 28.70502970984996,
|
||||
"p90": 28.949734665706984
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 107.14572231615826,
|
||||
"p90": 118.03470639992497
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns16/tp4/result.json"
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 3752.684495467103,
|
||||
"p90": 3794.6349827560653
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 28.70502970984996,
|
||||
"p90": 28.949734665706984
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 107.14572231615826,
|
||||
"p90": 118.03470639992497
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns32/tp4/result.json"
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 3752.684495467103,
|
||||
"p90": 3794.6349827560653
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 28.70502970984996,
|
||||
"p90": 28.949734665706984
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 107.14572231615826,
|
||||
"p90": 118.03470639992497
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns64/tp4/result.json"
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 3752.684495467103,
|
||||
"p90": 3794.6349827560653
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 28.70502970984996,
|
||||
"p90": 28.949734665706984
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 107.14572231615826,
|
||||
"p90": 118.03470639992497
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns8/tp4/result.json"
|
||||
}
|
||||
},
|
||||
"prefill_only": false,
|
||||
"real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/fixed-pd-real-audit.json",
|
||||
"schema": "qwen30-latency-case-frontier-real-comparison-v1",
|
||||
"selection": {
|
||||
"e2e_ms:mean": {
|
||||
"agreement": 0.0,
|
||||
"concordant_pairs": 0,
|
||||
"discordant_pairs": 48,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.26921939465653383,
|
||||
"sim_ranking": [
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"sim_winner": "tp1_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"agreement": 0.0,
|
||||
"concordant_pairs": 0,
|
||||
"discordant_pairs": 48,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.25730483004000526,
|
||||
"sim_ranking": [
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"sim_winner": "tp1_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_ms:mean": {
|
||||
"agreement": 0.0,
|
||||
"concordant_pairs": 0,
|
||||
"discordant_pairs": 48,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.13962124906187545,
|
||||
"sim_ranking": [
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"sim_winner": "tp1_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_ms:p90": {
|
||||
"agreement": 0.0,
|
||||
"concordant_pairs": 0,
|
||||
"discordant_pairs": 48,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.13357103696654363,
|
||||
"sim_ranking": [
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"sim_winner": "tp1_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns16",
|
||||
"selected_config_real_regret": 0.0,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64",
|
||||
"tp1_mns16"
|
||||
],
|
||||
"real_winner": "tp4_mns16",
|
||||
"selected_config_real_regret": 0.0,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": true
|
||||
}
|
||||
},
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30 Frontier vs real latency selection
|
||||
|
||||
| Objective | Frontier | Real | Match | Regret | Pairwise |
|
||||
|---|---|---|---:|---:|---:|
|
||||
| ttft_ms:mean | tp4_mns16 | tp4_mns16 | yes | 0.0% | 100.0% |
|
||||
| ttft_ms:p90 | tp4_mns16 | tp4_mns16 | yes | 0.0% | 100.0% |
|
||||
| tpot_ms:mean | tp1_mns16 | tp4_mns32 | no | 14.0% | 0.0% |
|
||||
| tpot_ms:p90 | tp1_mns16 | tp4_mns8 | no | 13.4% | 0.0% |
|
||||
| e2e_ms:mean | tp1_mns16 | tp4_mns32 | no | 26.9% | 0.0% |
|
||||
| e2e_ms:p90 | tp1_mns16 | tp4_mns8 | no | 25.7% | 0.0% |
|
||||
@@ -0,0 +1 @@
|
||||
{"e2e_ms:mean": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns32", "selected_config_real_regret": 0.26921939465653383, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "e2e_ms:p90": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.25730483004000526, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "tpot_ms:mean": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns32", "tp2_mns8", "tp2_mns64", "tp1_mns64", "tp1_mns32", "tp1_mns16", "tp1_mns8"], "real_winner": "tp4_mns32", "selected_config_real_regret": 0.13962124906187545, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "tpot_ms:p90": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns32", "tp2_mns16", "tp2_mns8", "tp2_mns64", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.13357103696654363, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:mean": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns32", "tp4_mns8", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns16", "selected_config_real_regret": 0.0, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": true}, "ttft_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns8", "tp4_mns32", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64", "tp1_mns16"], "real_winner": "tp4_mns16", "selected_config_real_regret": 0.0, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": true}}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,12 @@
|
||||
# Qwen30 real latency case audit
|
||||
|
||||
Prefill-only: `False`.
|
||||
|
||||
| Objective | Real winner | Value (ms) |
|
||||
|---|---|---:|
|
||||
| ttft_ms:pooled_mean_ms | tp4_mns16 | 63.04 |
|
||||
| ttft_ms:pooled_p90_ms | tp4_mns16 | 66.42 |
|
||||
| tpot_ms:pooled_mean_ms | tp4_mns32 | 4.25 |
|
||||
| tpot_ms:pooled_p90_ms | tp4_mns8 | 4.27 |
|
||||
| e2e_ms:pooled_mean_ms | tp4_mns32 | 604.41 |
|
||||
| e2e_ms:pooled_p90_ms | tp4_mns8 | 610.01 |
|
||||
@@ -0,0 +1 @@
|
||||
{"e2e_ms:pooled_mean_ms": {"ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns32", "winner_value_ms": 604.4058500071964}, "e2e_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns8", "winner_value_ms": 610.0081680342555}, "tpot_ms:pooled_mean_ms": {"ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns32", "tp2_mns8", "tp2_mns64", "tp1_mns64", "tp1_mns32", "tp1_mns16", "tp1_mns8"], "winner": "tp4_mns32", "winner_value_ms": 4.254808547439789}, "tpot_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns32", "tp2_mns16", "tp2_mns8", "tp2_mns64", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns8", "winner_value_ms": 4.268226101921182}, "ttft_ms:pooled_mean_ms": {"ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns32", "tp4_mns8", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns16", "winner_value_ms": 63.03574686938565}, "ttft_ms:pooled_p90_ms": {"ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns8", "tp4_mns32", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64", "tp1_mns16"], "winner": "tp4_mns16", "winner_value_ms": 66.4233659626916}}
|
||||
@@ -0,0 +1,326 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"cells": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns16/tp1/result.json"
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns32/tp1/result.json"
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns64/tp1/result.json"
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 166.66991554923203,
|
||||
"p90": 166.6699155492779
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns8/tp1/result.json"
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns16/tp2/result.json"
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns32/tp2/result.json"
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns64/tp2/result.json"
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 118.55592059231998,
|
||||
"p90": 118.55592059232833
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns8/tp2/result.json"
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns16/tp4/result.json"
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns32/tp4/result.json"
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns64/tp4/result.json"
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 86.52545838364681,
|
||||
"p90": 86.52545838364745
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns8/tp4/result.json"
|
||||
}
|
||||
},
|
||||
"prefill_only": true,
|
||||
"real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/fixed-po-real-audit.json",
|
||||
"schema": "qwen30-latency-case-frontier-real-comparison-v1",
|
||||
"selection": {
|
||||
"e2e_ms:mean": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.014895639803871355,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.03590282591969048,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.014940997472395743,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.035908453871728634,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po"
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
# Qwen30 Frontier vs real latency selection
|
||||
|
||||
| Objective | Frontier | Real | Match | Regret | Pairwise |
|
||||
|---|---|---|---:|---:|---:|
|
||||
| ttft_ms:mean | tp4_mns16 | tp4_mns8 | no | 1.5% | 100.0% |
|
||||
| ttft_ms:p90 | tp4_mns16 | tp4_mns8 | no | 3.6% | 100.0% |
|
||||
| e2e_ms:mean | tp4_mns16 | tp4_mns8 | no | 1.5% | 100.0% |
|
||||
| e2e_ms:p90 | tp4_mns16 | tp4_mns8 | no | 3.6% | 100.0% |
|
||||
@@ -0,0 +1 @@
|
||||
{"e2e_ms:mean": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.014895639803871355, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "e2e_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.03590282591969048, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:mean": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.014940997472395743, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.035908453871728634, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}}
|
||||
@@ -0,0 +1,997 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"case_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po",
|
||||
"configs": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 151.54830888467202,
|
||||
"pooled_p90_ms": 152.33826905023307,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 151.54830888467202,
|
||||
"trial_stddev_of_means_ms": 0.33839071804993504
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 151.50490087229585,
|
||||
"pooled_p90_ms": 152.29528304189444,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 151.50490087229585,
|
||||
"trial_stddev_of_means_ms": 0.3392777547312034
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 151.25221894176806,
|
||||
"p90_ms": 152.03110210131854,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 151.20715750724278,
|
||||
"p90_ms": 151.9814640050754,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 151.4755454836702,
|
||||
"p90_ms": 152.21454401034862,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 151.4332707151915,
|
||||
"p90_ms": 152.16313605196774,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 151.9171622285778,
|
||||
"p90_ms": 152.64088998083025,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 151.87427439445327,
|
||||
"p90_ms": 152.5908849434927,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 151.4782552960514,
|
||||
"pooled_p90_ms": 152.49883604701608,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 151.4782552960514,
|
||||
"trial_stddev_of_means_ms": 0.5555788512092701
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 151.43368434935846,
|
||||
"pooled_p90_ms": 152.45853806845844,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 151.43368434935846,
|
||||
"trial_stddev_of_means_ms": 0.5560791927635007
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 151.00544339222762,
|
||||
"p90_ms": 151.96295792702585,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 150.96285229624618,
|
||||
"p90_ms": 151.92402398679405,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 151.33915553880178,
|
||||
"p90_ms": 152.3007940268144,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 151.29099727893436,
|
||||
"p90_ms": 152.25290204398334,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 152.09016695712484,
|
||||
"p90_ms": 152.93943404685706,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 152.04720347289486,
|
||||
"p90_ms": 152.89701998699456,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 152.62579285143815,
|
||||
"pooled_p90_ms": 153.7184970220551,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 152.62579285143815,
|
||||
"trial_stddev_of_means_ms": 0.5522776932799126
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 152.58183204256076,
|
||||
"pooled_p90_ms": 153.6727879429236,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 152.58183204256076,
|
||||
"trial_stddev_of_means_ms": 0.552880728503269
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 152.98813100463835,
|
||||
"p90_ms": 153.7133390083909,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 152.9461515773189,
|
||||
"p90_ms": 153.67183403577656,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 152.89909483832415,
|
||||
"p90_ms": 154.20521004125476,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 152.85368830727253,
|
||||
"p90_ms": 154.1550400434062,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 151.99015271135195,
|
||||
"p90_ms": 152.67854696139693,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 151.94565624309084,
|
||||
"p90_ms": 152.6353599037975,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 152.22744446955332,
|
||||
"pooled_p90_ms": 152.97825809102505,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 152.22744446955332,
|
||||
"trial_stddev_of_means_ms": 0.15844676271042335
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 152.18195707226792,
|
||||
"pooled_p90_ms": 152.93638303410262,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 152.18195707226792,
|
||||
"trial_stddev_of_means_ms": 0.1592296957231754
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 152.08507488055747,
|
||||
"p90_ms": 152.94339996762574,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 152.04033442473315,
|
||||
"p90_ms": 152.89624000433832,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 152.3981458541771,
|
||||
"p90_ms": 153.33553205709904,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 152.3543132889597,
|
||||
"p90_ms": 153.2915000570938,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 152.19911267392538,
|
||||
"p90_ms": 152.6885419152677,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 152.15122350311094,
|
||||
"p90_ms": 152.64206391293555,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 97.51495216234325,
|
||||
"pooled_p90_ms": 98.90919097233564,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 97.51495216234325,
|
||||
"trial_stddev_of_means_ms": 0.41813696851903276
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 97.46812211917545,
|
||||
"pooled_p90_ms": 98.85558602400124,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 97.46812211917545,
|
||||
"trial_stddev_of_means_ms": 0.41503784318372017
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 97.09843800785814,
|
||||
"p90_ms": 98.22496003471315,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 97.05583205674017,
|
||||
"p90_ms": 98.16953900735825,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 97.93469326682533,
|
||||
"p90_ms": 99.07599200960249,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 97.88585422369255,
|
||||
"p90_ms": 99.02978304307908,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 97.51172521234628,
|
||||
"p90_ms": 99.18248606845737,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 97.46268007709364,
|
||||
"p90_ms": 99.13825406692922,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 97.09498990484747,
|
||||
"pooled_p90_ms": 98.50096399895847,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 97.09498990484747,
|
||||
"trial_stddev_of_means_ms": 0.27051136106210416
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 97.04873380816761,
|
||||
"pooled_p90_ms": 98.45811000559479,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 97.0487338081676,
|
||||
"trial_stddev_of_means_ms": 0.27130252759105145
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 96.8018446240372,
|
||||
"p90_ms": 97.90838800836354,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 96.75326022814035,
|
||||
"p90_ms": 97.85841300617903,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 97.33497442735373,
|
||||
"p90_ms": 98.68263697717339,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 97.28661816668256,
|
||||
"p90_ms": 98.63975096959621,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 97.14815066315147,
|
||||
"p90_ms": 98.70580595452338,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 97.1063230296799,
|
||||
"p90_ms": 98.66256301756948,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 94.78371690426897,
|
||||
"pooled_p90_ms": 95.95681494101882,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 94.78371690426899,
|
||||
"trial_stddev_of_means_ms": 0.09464944657839716
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 94.73932527026128,
|
||||
"pooled_p90_ms": 95.90691793709993,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 94.73932527026129,
|
||||
"trial_stddev_of_means_ms": 0.09810418936504331
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 94.80111797740899,
|
||||
"p90_ms": 95.66861705388874,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 94.75897805848297,
|
||||
"p90_ms": 95.61934601515532,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 94.68157430154149,
|
||||
"p90_ms": 95.95681494101882,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 94.63288232525305,
|
||||
"p90_ms": 95.90691793709993,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 94.86845843385645,
|
||||
"p90_ms": 96.28609393257648,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 94.82611542704782,
|
||||
"p90_ms": 96.24539199285209,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 95.65880776170006,
|
||||
"pooled_p90_ms": 97.55863400641829,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 95.65880776170006,
|
||||
"trial_stddev_of_means_ms": 1.2232525642643362
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 95.61450884031319,
|
||||
"pooled_p90_ms": 97.50729007646441,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 95.61450884031319,
|
||||
"trial_stddev_of_means_ms": 1.2245345925842415
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 97.06745062893357,
|
||||
"p90_ms": 99.17883900925517,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 97.02484949686846,
|
||||
"p90_ms": 99.13210896775126,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 94.86425999920209,
|
||||
"p90_ms": 96.06455289758742,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 94.82164932920259,
|
||||
"p90_ms": 96.02120891213417,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 95.0447126569645,
|
||||
"p90_ms": 97.03064302448183,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 94.99702769486855,
|
||||
"p90_ms": 96.98293695691973,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 65.28745120113297,
|
||||
"pooled_p90_ms": 70.42863499373198,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 65.28745120113297,
|
||||
"trial_stddev_of_means_ms": 3.4421132551731515
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 65.24474184372936,
|
||||
"pooled_p90_ms": 70.38743107113987,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 65.24474184372936,
|
||||
"trial_stddev_of_means_ms": 3.440733102883723
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 64.19509824892405,
|
||||
"p90_ms": 67.92093603871763,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 64.15208354917203,
|
||||
"p90_ms": 67.87848006933928,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 69.1431922154039,
|
||||
"p90_ms": 78.7114379927516,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 69.09912444791908,
|
||||
"p90_ms": 78.66681204177439,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 62.524063139070954,
|
||||
"p90_ms": 63.537837006151676,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 62.48301753409698,
|
||||
"p90_ms": 63.49677708931267,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 64.54067908164875,
|
||||
"pooled_p90_ms": 69.61812602821738,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 64.54067908164875,
|
||||
"trial_stddev_of_means_ms": 0.8919869576553885
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 64.4944863910101,
|
||||
"pooled_p90_ms": 69.50995896477252,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 64.49448639101008,
|
||||
"trial_stddev_of_means_ms": 0.8942503909647054
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 65.55543364614046,
|
||||
"p90_ms": 71.84744393453002,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 65.51214804161744,
|
||||
"p90_ms": 71.80331903509796,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 63.88050908684846,
|
||||
"p90_ms": 66.97378400713205,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 63.83413878312811,
|
||||
"p90_ms": 66.91711395978928,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 64.18609451195731,
|
||||
"p90_ms": 69.55521402414888,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 64.13717234828474,
|
||||
"p90_ms": 69.50995896477252,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 65.03269931623949,
|
||||
"pooled_p90_ms": 68.43663903418928,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 65.03269931623947,
|
||||
"trial_stddev_of_means_ms": 1.1339679108088652
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 64.98623655348516,
|
||||
"pooled_p90_ms": 68.38861701544374,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 64.98623655348516,
|
||||
"trial_stddev_of_means_ms": 1.1341449568113504
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 66.32654335125991,
|
||||
"p90_ms": 70.97568700555712,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 66.28015979965643,
|
||||
"p90_ms": 70.93840499874204,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 64.56001544761104,
|
||||
"p90_ms": 67.99875595606863,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 64.51422280213225,
|
||||
"p90_ms": 67.94618198182434,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 64.21153914984748,
|
||||
"p90_ms": 67.55471904762089,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 64.16432705866679,
|
||||
"p90_ms": 67.51206307671964,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 64.32922621852013,
|
||||
"pooled_p90_ms": 67.98768497537822,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 64.32922621852013,
|
||||
"trial_stddev_of_means_ms": 1.8052761808674234
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 64.2842707174255,
|
||||
"pooled_p90_ms": 67.94753996655345,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 64.2842707174255,
|
||||
"trial_stddev_of_means_ms": 1.80289690665978
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 62.84650166739183,
|
||||
"p90_ms": 63.89774801209569,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 62.80200301792271,
|
||||
"p90_ms": 63.856702065095305,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 66.3395124548945,
|
||||
"p90_ms": 70.76063798740506,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 66.29134260132447,
|
||||
"p90_ms": 70.7101090811193,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 63.801664533274064,
|
||||
"p90_ms": 67.48319591861218,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 63.759466533029325,
|
||||
"p90_ms": 67.4400629941374,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"prefill_only": true,
|
||||
"schema": "qwen30-latency-case-real-audit-v1",
|
||||
"trace_manifests": {
|
||||
"tp1": {
|
||||
"description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse",
|
||||
"first_arrival_s": 0.0,
|
||||
"global_offered_request_rate": 0.215,
|
||||
"last_arrival_s": 595.3488372093024,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 0.215,
|
||||
"prefix_caching": false,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp1/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "ce7582fc13050078a243d7f6adfc2b2e071dbe87b5f760c7bf111af57718ccfb",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp1/public/frontier.csv",
|
||||
"public_csv_sha256": "e6f1ca0354e9d93c60ff18261ae6e08e2e54dbc888e30e244776f9cc8695a8cf",
|
||||
"requests": 129,
|
||||
"row_vector_sha256": "7c8dc7cccd2bd874768c1b06ca40d2327452390577b9907bacb76270b46998fa",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 1
|
||||
},
|
||||
"tp2": {
|
||||
"description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse",
|
||||
"first_arrival_s": 0.0,
|
||||
"global_offered_request_rate": 0.43,
|
||||
"last_arrival_s": 297.6744186046512,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 0.215,
|
||||
"prefix_caching": false,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp2/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "3fe3c01ac31cd776b2458cafb818d889b20c5e14948154bbd716cce7200c8d5b",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp2/public/frontier.csv",
|
||||
"public_csv_sha256": "78b900d8f47fec3a1c83b748f82cb9623ce3f0cf08265aa20c3ef1511a4a97e0",
|
||||
"requests": 129,
|
||||
"row_vector_sha256": "c9382d9007ea1dc572898836f2d0efb8e2244c4febc1ed59d5aeacfcc5475c0a",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 2
|
||||
},
|
||||
"tp4": {
|
||||
"description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse",
|
||||
"first_arrival_s": 0.0,
|
||||
"global_offered_request_rate": 0.86,
|
||||
"last_arrival_s": 148.8372093023256,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 0.215,
|
||||
"prefix_caching": false,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp4/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "af3ba5a7888ea5f283bb360dd36f1f455906f69f5ad82cef84e82018545710db",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp4/public/frontier.csv",
|
||||
"public_csv_sha256": "8b1100ccab34d9dac62b7ff1508d36bdf75aa3c3135e6f6642a0d3ac77af2505",
|
||||
"requests": 129,
|
||||
"row_vector_sha256": "a5e4959f28790eaa99a59a1b86c449a4e484967f1e35e47900d1a27fa061607f",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 4
|
||||
}
|
||||
},
|
||||
"traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po",
|
||||
"winners": {
|
||||
"e2e_ms:pooled_mean_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"winner": "tp4_mns8",
|
||||
"winner_value_ms": 64.32922621852013
|
||||
},
|
||||
"e2e_ms:pooled_p90_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"winner": "tp4_mns8",
|
||||
"winner_value_ms": 67.98768497537822
|
||||
},
|
||||
"ttft_ms:pooled_mean_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"winner": "tp4_mns8",
|
||||
"winner_value_ms": 64.2842707174255
|
||||
},
|
||||
"ttft_ms:pooled_p90_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"winner": "tp4_mns8",
|
||||
"winner_value_ms": 67.94753996655345
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30 real latency case audit
|
||||
|
||||
Prefill-only: `True`.
|
||||
|
||||
| Objective | Real winner | Value (ms) |
|
||||
|---|---|---:|
|
||||
| ttft_ms:pooled_mean_ms | tp4_mns8 | 64.28 |
|
||||
| ttft_ms:pooled_p90_ms | tp4_mns8 | 67.95 |
|
||||
| e2e_ms:pooled_mean_ms | tp4_mns8 | 64.33 |
|
||||
| e2e_ms:pooled_p90_ms | tp4_mns8 | 67.99 |
|
||||
@@ -0,0 +1 @@
|
||||
{"e2e_ms:pooled_mean_ms": {"ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 64.32922621852013}, "e2e_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 67.98768497537822}, "ttft_ms:pooled_mean_ms": {"ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 64.2842707174255}, "ttft_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 67.94753996655345}}
|
||||
@@ -0,0 +1,326 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"cells": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 340.75522510609915,
|
||||
"p90": 832.752247743656
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 340.75522510609915,
|
||||
"p90": 832.752247743656
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns16/tp1/result.json"
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 340.75598360435777,
|
||||
"p90": 832.752247743656
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 340.75598360435777,
|
||||
"p90": 832.752247743656
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns32/tp1/result.json"
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 340.7569941487134,
|
||||
"p90": 832.752247743656
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 340.7569941487134,
|
||||
"p90": 832.752247743656
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns64/tp1/result.json"
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 340.75522510609915,
|
||||
"p90": 832.752247743656
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 340.75522510609915,
|
||||
"p90": 832.752247743656
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns8/tp1/result.json"
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 222.73961839402438,
|
||||
"p90": 513.3421414172687
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 222.73961839402438,
|
||||
"p90": 513.3421414172687
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns16/tp2/result.json"
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 222.7393083124536,
|
||||
"p90": 513.3421414172687
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 222.7393083124536,
|
||||
"p90": 513.3421414172687
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns32/tp2/result.json"
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 222.73932355314494,
|
||||
"p90": 513.3421414172687
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 222.73932355314494,
|
||||
"p90": 513.3421414172687
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns64/tp2/result.json"
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 222.73961839402438,
|
||||
"p90": 513.3421414172687
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 222.73961839402438,
|
||||
"p90": 513.3421414172687
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns8/tp2/result.json"
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 164.583607484367,
|
||||
"p90": 355.93888195481327
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 164.583607484367,
|
||||
"p90": 355.93888195481327
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns16/tp4/result.json"
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 164.58396574139448,
|
||||
"p90": 355.93888195481327
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 164.58396574139448,
|
||||
"p90": 355.93888195481327
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns32/tp4/result.json"
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 164.58471606968033,
|
||||
"p90": 355.93888195481327
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 164.58471606968033,
|
||||
"p90": 355.93888195481327
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns64/tp4/result.json"
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 164.583607484367,
|
||||
"p90": 355.93888195481327
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 164.583607484367,
|
||||
"p90": 355.93888195481327
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns8/tp4/result.json"
|
||||
}
|
||||
},
|
||||
"prefill_only": true,
|
||||
"real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/trace-po-real-audit.json",
|
||||
"schema": "qwen30-latency-case-frontier-real-comparison-v1",
|
||||
"selection": {
|
||||
"e2e_ms:mean": {
|
||||
"agreement": 0.8412698412698413,
|
||||
"concordant_pairs": 53,
|
||||
"discordant_pairs": 10,
|
||||
"informative_pairs": 63,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"selected_config_real_regret": 0.0318784105178655,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 3,
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.0033158177113489728,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"agreement": 0.8412698412698413,
|
||||
"concordant_pairs": 53,
|
||||
"discordant_pairs": 10,
|
||||
"informative_pairs": 63,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"selected_config_real_regret": 0.03187213306057872,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 3,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.0037779133092611917,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po"
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
# Qwen30 Frontier vs real latency selection
|
||||
|
||||
| Objective | Frontier | Real | Match | Regret | Pairwise |
|
||||
|---|---|---|---:|---:|---:|
|
||||
| ttft_ms:mean | tp4_mns16 | tp4_mns64 | no | 3.2% | 84.1% |
|
||||
| ttft_ms:p90 | tp4_mns16 | tp4_mns8 | no | 0.4% | 100.0% |
|
||||
| e2e_ms:mean | tp4_mns16 | tp4_mns64 | no | 3.2% | 84.1% |
|
||||
| e2e_ms:p90 | tp4_mns16 | tp4_mns8 | no | 0.3% | 100.0% |
|
||||
@@ -0,0 +1 @@
|
||||
{"e2e_ms:mean": {"agreement": 0.8412698412698413, "concordant_pairs": 53, "discordant_pairs": 10, "informative_pairs": 63, "real_ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns64", "selected_config_real_regret": 0.0318784105178655, "sim_ranking": ["tp4_mns16", "tp4_mns8", "tp4_mns32", "tp4_mns64", "tp2_mns32", "tp2_mns64", "tp2_mns16", "tp2_mns8", "tp1_mns16", "tp1_mns8", "tp1_mns32", "tp1_mns64"], "sim_winner": "tp4_mns16", "tied_pairs": 3, "winner_match": false}, "e2e_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.0033158177113489728, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:mean": {"agreement": 0.8412698412698413, "concordant_pairs": 53, "discordant_pairs": 10, "informative_pairs": 63, "real_ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns64", "selected_config_real_regret": 0.03187213306057872, "sim_ranking": ["tp4_mns16", "tp4_mns8", "tp4_mns32", "tp4_mns64", "tp2_mns32", "tp2_mns64", "tp2_mns16", "tp2_mns8", "tp1_mns16", "tp1_mns8", "tp1_mns32", "tp1_mns64"], "sim_winner": "tp4_mns16", "tied_pairs": 3, "winner_match": false}, "ttft_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.0037779133092611917, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}}
|
||||
@@ -0,0 +1,997 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"case_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po",
|
||||
"configs": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 552.8720716136033,
|
||||
"pooled_p90_ms": 1849.5772210881114,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 552.8720716136035,
|
||||
"trial_stddev_of_means_ms": 2.521785864921604
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 552.8045099093855,
|
||||
"pooled_p90_ms": 1849.544870085083,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 552.8045099093856,
|
||||
"trial_stddev_of_means_ms": 2.520195001336989
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 551.4591356410912,
|
||||
"p90_ms": 1849.5772210881114,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 551.3877547012152,
|
||||
"p90_ms": 1849.544870085083,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 551.3735196675904,
|
||||
"p90_ms": 1854.1427809977904,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 551.311527398711,
|
||||
"p90_ms": 1854.09467399586,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 555.7835595321286,
|
||||
"p90_ms": 1865.3576440410689,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 555.7142476282306,
|
||||
"p90_ms": 1865.324669983238,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 553.9655365312751,
|
||||
"pooled_p90_ms": 1858.5613030008972,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 553.9655365312751,
|
||||
"trial_stddev_of_means_ms": 0.562579474030608
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 553.8995644734638,
|
||||
"pooled_p90_ms": 1858.5308340843767,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 553.8995644734638,
|
||||
"trial_stddev_of_means_ms": 0.5595598051891147
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 554.533127208002,
|
||||
"p90_ms": 1863.7738740071654,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 554.4650907711263,
|
||||
"p90_ms": 1863.7414400000125,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 553.4081058838412,
|
||||
"p90_ms": 1860.4962331010029,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 553.3461683987488,
|
||||
"p90_ms": 1860.462275100872,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 553.9553765019821,
|
||||
"p90_ms": 1858.5613030008972,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 553.8874342505163,
|
||||
"p90_ms": 1858.5308340843767,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 553.5116215725009,
|
||||
"pooled_p90_ms": 1862.3707490041852,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 553.5116215725008,
|
||||
"trial_stddev_of_means_ms": 2.551946974085908
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 553.4431933351968,
|
||||
"pooled_p90_ms": 1862.3419440118596,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 553.4431933351968,
|
||||
"trial_stddev_of_means_ms": 2.558476651513527
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 551.9927626163446,
|
||||
"p90_ms": 1863.004481070675,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 551.9189011646695,
|
||||
"p90_ms": 1862.9671899834648,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 556.4578829970312,
|
||||
"p90_ms": 1876.7878300277516,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 556.3969604667392,
|
||||
"p90_ms": 1876.7514559440315,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 552.0842191041266,
|
||||
"p90_ms": 1862.3707490041852,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 552.0137183741818,
|
||||
"p90_ms": 1862.3419440118596,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 554.3623829256859,
|
||||
"pooled_p90_ms": 1849.5380710810423,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 554.3623829256859,
|
||||
"trial_stddev_of_means_ms": 1.2895027720806873
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 554.2973810096947,
|
||||
"pooled_p90_ms": 1849.499023053795,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 554.2973810096947,
|
||||
"trial_stddev_of_means_ms": 1.2879745625238963
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 552.8872783905863,
|
||||
"p90_ms": 1849.5380710810423,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 552.8246594192243,
|
||||
"p90_ms": 1849.499023053795,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 555.2756262775293,
|
||||
"p90_ms": 1865.9542039968073,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 555.2131765045572,
|
||||
"p90_ms": 1865.9242549911141,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 554.9242441089419,
|
||||
"p90_ms": 1863.5734919225797,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 554.8543071053027,
|
||||
"p90_ms": 1863.5354900034145,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 354.36655512622264,
|
||||
"pooled_p90_ms": 1137.23775499966,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 354.36655512622264,
|
||||
"trial_stddev_of_means_ms": 1.8755237488897505
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 354.2987044818834,
|
||||
"pooled_p90_ms": 1137.2028539190069,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 354.29870448188336,
|
||||
"trial_stddev_of_means_ms": 1.8738441773930947
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 352.474492817554,
|
||||
"p90_ms": 1146.7004970181733,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 352.4049650534785,
|
||||
"p90_ms": 1146.5505469823256,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 356.2250907998532,
|
||||
"p90_ms": 1142.300286097452,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 356.1519986188198,
|
||||
"p90_ms": 1142.2789900097996,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 354.40008176126076,
|
||||
"p90_ms": 1137.23775499966,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 354.3391497733519,
|
||||
"p90_ms": 1137.2028539190069,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 353.6182792498718,
|
||||
"pooled_p90_ms": 1133.6436610436067,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 353.6182792498718,
|
||||
"trial_stddev_of_means_ms": 1.6113304304047518
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 353.5506138878872,
|
||||
"pooled_p90_ms": 1133.6100719636306,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 353.5506138878872,
|
||||
"trial_stddev_of_means_ms": 1.6067873099412449
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 355.363455723769,
|
||||
"p90_ms": 1137.4261770397425,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 355.2912532689425,
|
||||
"p90_ms": 1137.4054140178487,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 353.3043994003358,
|
||||
"p90_ms": 1133.6436610436067,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 353.2365282921684,
|
||||
"p90_ms": 1133.6100719636306,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 352.1869826255106,
|
||||
"p90_ms": 1137.214289046824,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 352.12406010255074,
|
||||
"p90_ms": 1137.1923619881272,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 344.2724434967521,
|
||||
"pooled_p90_ms": 1112.3684010235593,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 344.27244349675215,
|
||||
"trial_stddev_of_means_ms": 1.2293359433030469
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 344.2076703236325,
|
||||
"pooled_p90_ms": 1112.340173101984,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 344.20767032363256,
|
||||
"trial_stddev_of_means_ms": 1.2250537007477627
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 342.85445375793955,
|
||||
"p90_ms": 1112.3684010235593,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 342.79442101431823,
|
||||
"p90_ms": 1112.340173101984,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 345.03840929702244,
|
||||
"p90_ms": 1119.2332479404286,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 344.96722287271,
|
||||
"p90_ms": 1119.2107619717717,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 344.92446743529433,
|
||||
"p90_ms": 1121.5684060007334,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 344.86136708386937,
|
||||
"p90_ms": 1121.5376510517672,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 347.7476487716525,
|
||||
"pooled_p90_ms": 1112.5693990616128,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 347.74764877165256,
|
||||
"trial_stddev_of_means_ms": 0.717530137997859
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 347.67954972228495,
|
||||
"pooled_p90_ms": 1112.5206090509892,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 347.67954972228495,
|
||||
"trial_stddev_of_means_ms": 0.7137879732940912
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 347.8964949396706,
|
||||
"p90_ms": 1126.5470329672098,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 347.825654325718,
|
||||
"p90_ms": 1126.4982499415055,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 348.37908198508296,
|
||||
"p90_ms": 1108.5135659668595,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 348.3089811246221,
|
||||
"p90_ms": 1108.4930589422584,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 346.967369390204,
|
||||
"p90_ms": 1112.5693990616128,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 346.90401371651467,
|
||||
"p90_ms": 1112.5206090509892,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 230.30740981957652,
|
||||
"pooled_p90_ms": 669.3113599903882,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 230.30740981957652,
|
||||
"trial_stddev_of_means_ms": 2.164263687280578
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 230.23591205587212,
|
||||
"pooled_p90_ms": 669.2857249872759,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 230.2359120558721,
|
||||
"trial_stddev_of_means_ms": 2.1675472527430966
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 230.70611610156737,
|
||||
"p90_ms": 682.0242760004476,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 230.64041384333356,
|
||||
"p90_ms": 681.8402550416067,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 232.2445988110014,
|
||||
"p90_ms": 669.8487129760906,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 232.17271348053603,
|
||||
"p90_ms": 669.6453359909356,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 227.9715145461608,
|
||||
"p90_ms": 668.2263769907877,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 227.89460884374674,
|
||||
"p90_ms": 667.8900560364127,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 227.01489424738475,
|
||||
"pooled_p90_ms": 668.1078639812768,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 227.01489424738475,
|
||||
"trial_stddev_of_means_ms": 1.0302265785061928
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 226.9508410934046,
|
||||
"pooled_p90_ms": 668.0838000029325,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 226.9508410934046,
|
||||
"trial_stddev_of_means_ms": 1.0309199236641802
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 227.36105044253256,
|
||||
"p90_ms": 668.9074879977852,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 227.29304120623212,
|
||||
"p90_ms": 668.8842560397461,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 227.82746249586054,
|
||||
"p90_ms": 668.1078639812768,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 227.7671468305192,
|
||||
"p90_ms": 668.0838000029325,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 225.8561698037611,
|
||||
"p90_ms": 668.5177159961313,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 225.79233524346247,
|
||||
"p90_ms": 668.1753649609163,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 223.1923911500318,
|
||||
"pooled_p90_ms": 668.0814319988713,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 223.1923911500318,
|
||||
"trial_stddev_of_means_ms": 1.0834054130133073
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 223.12445959072676,
|
||||
"pooled_p90_ms": 668.0591680342332,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 223.1244595907268,
|
||||
"trial_stddev_of_means_ms": 1.078830920161661
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 224.0322316110553,
|
||||
"p90_ms": 668.6453759903088,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 223.96008559961103,
|
||||
"p90_ms": 668.1041569681838,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 223.57544556978453,
|
||||
"p90_ms": 697.0440980512649,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 223.50675472538455,
|
||||
"p90_ms": 697.0191639848053,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 221.96949626925553,
|
||||
"p90_ms": 668.0814319988713,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 221.9065384471847,
|
||||
"p90_ms": 668.0591680342332,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 229.07142115104702,
|
||||
"pooled_p90_ms": 667.0993800507858,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 229.07142115104702,
|
||||
"trial_stddev_of_means_ms": 1.6982436653662605
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 229.00724306236953,
|
||||
"pooled_p90_ms": 666.7667380534112,
|
||||
"pooled_samples": 387,
|
||||
"trial_mean_of_means_ms": 229.00724306236953,
|
||||
"trial_stddev_of_means_ms": 1.6999412520227009
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 227.73505919955872,
|
||||
"p90_ms": 667.3817320261151,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 227.67112628275225,
|
||||
"p90_ms": 667.0240580569953,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 230.98243522013775,
|
||||
"p90_ms": 667.0993800507858,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 230.92065048909407,
|
||||
"p90_ms": 666.7667380534112,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 228.4967690334446,
|
||||
"p90_ms": 667.9027479840443,
|
||||
"samples": 129
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 228.42995241526225,
|
||||
"p90_ms": 667.7060039946809,
|
||||
"samples": 129
|
||||
}
|
||||
},
|
||||
"requests": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"prefill_only": true,
|
||||
"schema": "qwen30-latency-case-real-audit-v1",
|
||||
"trace_manifests": {
|
||||
"tp1": {
|
||||
"description": "trace-derived: exact input/arrival/session/prefix; output override only",
|
||||
"first_arrival_s": 6.698,
|
||||
"global_offered_request_rate": 0.215,
|
||||
"last_arrival_s": 597.037,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 0.215,
|
||||
"prefix_caching": true,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp1/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "525bf779bbd111bc3c42ac65af5cfed36ce59806acc521ee16d1c5ce7b3a6152",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp1/public/frontier.csv",
|
||||
"public_csv_sha256": "eccab2e2f50f566eb4ec78291d126fce5fe9cb426eb89993c1b63b18f83dfc29",
|
||||
"requests": 129,
|
||||
"row_vector_sha256": "3df3a57e6a796e76d42f846fbfef59049f2bd804f4d3eda8ecbc9d96b43dc6ce",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 1
|
||||
},
|
||||
"tp2": {
|
||||
"description": "trace-derived: exact input/arrival/session/prefix; output override only",
|
||||
"first_arrival_s": 3.349,
|
||||
"global_offered_request_rate": 0.43,
|
||||
"last_arrival_s": 298.5185,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 0.215,
|
||||
"prefix_caching": true,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp2/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "61bdb8a7245426a7c283d0f2944657d3eda46b1c04998c88b025a2411fe15340",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp2/public/frontier.csv",
|
||||
"public_csv_sha256": "26701486cfb655a301b129acd14ab8d78b94ec843b4ceca8e222cee466e40535",
|
||||
"requests": 129,
|
||||
"row_vector_sha256": "f20dd473240fd243d38cb727f7896457e86da9e965b4f23f565aef5f3fc70351",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 2
|
||||
},
|
||||
"tp4": {
|
||||
"description": "trace-derived: exact input/arrival/session/prefix; output override only",
|
||||
"first_arrival_s": 1.6745,
|
||||
"global_offered_request_rate": 0.86,
|
||||
"last_arrival_s": 149.25925,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 0.215,
|
||||
"prefix_caching": true,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp4/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "c38514fa524fc215cd9097d990a2985532f83ac6314bc46ad89ebfc1bd89b4f8",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp4/public/frontier.csv",
|
||||
"public_csv_sha256": "f4112a2b0366b559bfced6a0986a01e2b7b39899a650d030c3e680b3dc63c92e",
|
||||
"requests": 129,
|
||||
"row_vector_sha256": "851e796f6d3e87444b58da87e0ef4b7482a13bb6502cb6873fa80523877ab74e",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 4
|
||||
}
|
||||
},
|
||||
"traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po",
|
||||
"winners": {
|
||||
"e2e_ms:pooled_mean_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"winner": "tp4_mns64",
|
||||
"winner_value_ms": 223.1923911500318
|
||||
},
|
||||
"e2e_ms:pooled_p90_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"winner": "tp4_mns8",
|
||||
"winner_value_ms": 667.0993800507858
|
||||
},
|
||||
"ttft_ms:pooled_mean_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"winner": "tp4_mns64",
|
||||
"winner_value_ms": 223.12445959072676
|
||||
},
|
||||
"ttft_ms:pooled_p90_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"winner": "tp4_mns8",
|
||||
"winner_value_ms": 666.7667380534112
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30 real latency case audit
|
||||
|
||||
Prefill-only: `True`.
|
||||
|
||||
| Objective | Real winner | Value (ms) |
|
||||
|---|---|---:|
|
||||
| ttft_ms:pooled_mean_ms | tp4_mns64 | 223.12 |
|
||||
| ttft_ms:pooled_p90_ms | tp4_mns8 | 666.77 |
|
||||
| e2e_ms:pooled_mean_ms | tp4_mns64 | 223.19 |
|
||||
| e2e_ms:pooled_p90_ms | tp4_mns8 | 667.10 |
|
||||
@@ -0,0 +1 @@
|
||||
{"e2e_ms:pooled_mean_ms": {"ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns64", "winner_value_ms": 223.1923911500318}, "e2e_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 667.0993800507858}, "ttft_ms:pooled_mean_ms": {"ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns64", "winner_value_ms": 223.12445959072676}, "ttft_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 666.7667380534112}}
|
||||
@@ -0,0 +1,453 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"tpot_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"cells": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 128201.44370110576,
|
||||
"p90": 216221.8716312423
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 106.74127671151493,
|
||||
"p90": 107.53204073371177
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 100982.41813966946,
|
||||
"p90": 188801.20124415262
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns16/tp1/result.json"
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 54134.6392206195,
|
||||
"p90": 70757.16788718625
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 131.73148052557949,
|
||||
"p90": 134.09269340154515
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 20543.111686596734,
|
||||
"p90": 40978.20939546926
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns32/tp1/result.json"
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 36992.839178769755,
|
||||
"p90": 38253.17393690583
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 143.73592141724885,
|
||||
"p90": 148.50084455017992
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 340.1792173712992,
|
||||
"p90": 390.1667904730175
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns64/tp1/result.json"
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 231355.5665072295,
|
||||
"p90": 401099.3133242681
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 79.72075104221494,
|
||||
"p90": 80.05841803805441
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 211026.77499146466,
|
||||
"p90": 380684.4167245642
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns8/tp1/result.json"
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 99415.72204671855,
|
||||
"p90": 170023.18522015438
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 69.3294093088488,
|
||||
"p90": 69.78173796567563
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 81736.72267296212,
|
||||
"p90": 152228.84203890432
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns16/tp2/result.json"
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 47656.99443299207,
|
||||
"p90": 71358.22494490076
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 83.38213205445625,
|
||||
"p90": 84.68169232450846
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 26394.55075910572,
|
||||
"p90": 52520.58767404907
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns32/tp2/result.json"
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 25800.137969948584,
|
||||
"p90": 26994.699921135867
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 100.27798158873513,
|
||||
"p90": 104.85288258852235
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 229.2526648211283,
|
||||
"p90": 261.37367146904467
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns64/tp2/result.json"
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 173225.23923299377,
|
||||
"p90": 301133.20586389
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 53.275061700270285,
|
||||
"p90": 53.466591976089035
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 159640.09849942484,
|
||||
"p90": 287499.22490998777
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns8/tp2/result.json"
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 77014.8840236653,
|
||||
"p90": 132893.6304418779
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 46.95641974752248,
|
||||
"p90": 47.23194326182317
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 65040.99698804708,
|
||||
"p90": 120849.48491011641
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns16/tp4/result.json"
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 41823.75653856865,
|
||||
"p90": 66900.19803040712
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 56.89199563649036,
|
||||
"p90": 57.75799749592927
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 27316.2976512636,
|
||||
"p90": 54230.649753904916
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns32/tp4/result.json"
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 24104.231142330624,
|
||||
"p90": 29174.179941975835
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 69.02049776725552,
|
||||
"p90": 72.06184134879742
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 6504.004211680465,
|
||||
"p90": 13004.945097644395
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns64/tp4/result.json"
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 133968.39678804282,
|
||||
"p90": 233355.7233519645
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean": 37.645089303294014,
|
||||
"p90": 37.75522144488961
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 124368.89901570287,
|
||||
"p90": 223728.14188352478
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns8/tp4/result.json"
|
||||
}
|
||||
},
|
||||
"prefill_only": false,
|
||||
"real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/analysis/fixed-pd/real-audit.json",
|
||||
"schema": "qwen30-latency-case-frontier-real-comparison-v1",
|
||||
"selection": {
|
||||
"e2e_ms:mean": {
|
||||
"agreement": 0.5151515151515151,
|
||||
"concordant_pairs": 34,
|
||||
"discordant_pairs": 32,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp1_mns32",
|
||||
"tp4_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns64",
|
||||
"tp1_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp1_mns32",
|
||||
"selected_config_real_regret": 0.017005214686575575,
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"agreement": 0.5151515151515151,
|
||||
"concordant_pairs": 34,
|
||||
"discordant_pairs": 32,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp1_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp1_mns32",
|
||||
"selected_config_real_regret": 0.054933980018932745,
|
||||
"sim_ranking": [
|
||||
"tp2_mns64",
|
||||
"tp4_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp1_mns32",
|
||||
"tp2_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp2_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": false
|
||||
},
|
||||
"tpot_ms:mean": {
|
||||
"agreement": 0.48484848484848486,
|
||||
"concordant_pairs": 32,
|
||||
"discordant_pairs": 34,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.0,
|
||||
"sim_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns8",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_ms:p90": {
|
||||
"agreement": 0.42424242424242425,
|
||||
"concordant_pairs": 28,
|
||||
"discordant_pairs": 38,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns64",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp2_mns64",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"selected_config_real_regret": 0.0,
|
||||
"sim_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp2_mns16",
|
||||
"tp4_mns64",
|
||||
"tp1_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns8",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"agreement": 0.5909090909090909,
|
||||
"concordant_pairs": 39,
|
||||
"discordant_pairs": 27,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.5797674051508896,
|
||||
"sim_ranking": [
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns64",
|
||||
"tp1_mns32",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp2_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"agreement": 0.5909090909090909,
|
||||
"concordant_pairs": 39,
|
||||
"discordant_pairs": 27,
|
||||
"informative_pairs": 66,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp1_mns8",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"selected_config_real_regret": 0.5620663582204116,
|
||||
"sim_ranking": [
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns64",
|
||||
"tp1_mns32",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp2_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30 Frontier vs real latency selection
|
||||
|
||||
| Objective | Frontier | Real | Match | Regret | Pairwise |
|
||||
|---|---|---|---:|---:|---:|
|
||||
| ttft_ms:mean | tp2_mns64 | tp4_mns32 | no | 58.0% | 59.1% |
|
||||
| ttft_ms:p90 | tp2_mns64 | tp4_mns64 | no | 56.2% | 59.1% |
|
||||
| tpot_ms:mean | tp4_mns8 | tp4_mns8 | yes | 0.0% | 48.5% |
|
||||
| tpot_ms:p90 | tp4_mns8 | tp4_mns8 | yes | 0.0% | 42.4% |
|
||||
| e2e_ms:mean | tp4_mns64 | tp1_mns32 | no | 1.7% | 51.5% |
|
||||
| e2e_ms:p90 | tp2_mns64 | tp1_mns32 | no | 5.5% | 51.5% |
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,12 @@
|
||||
# Qwen30 real latency case audit
|
||||
|
||||
Prefill-only: `False`.
|
||||
|
||||
| Objective | Real winner | Value (ms) |
|
||||
|---|---|---:|
|
||||
| ttft_ms:pooled_mean_ms | tp4_mns32 | 123.16 |
|
||||
| ttft_ms:pooled_p90_ms | tp4_mns64 | 126.84 |
|
||||
| tpot_ms:pooled_mean_ms | tp4_mns8 | 8.42 |
|
||||
| tpot_ms:pooled_p90_ms | tp4_mns8 | 8.77 |
|
||||
| e2e_ms:pooled_mean_ms | tp1_mns32 | 3203.72 |
|
||||
| e2e_ms:pooled_p90_ms | tp1_mns32 | 3389.19 |
|
||||
@@ -0,0 +1,326 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"cells": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns16/tp1/result.json"
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns32/tp1/result.json"
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns64/tp1/result.json"
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 267.46878507222704,
|
||||
"p90": 267.46878507223215
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns8/tp1/result.json"
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns16/tp2/result.json"
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns32/tp2/result.json"
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns64/tp2/result.json"
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 175.26431810890125,
|
||||
"p90": 175.2643181089013
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns8/tp2/result.json"
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns16/tp4/result.json"
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns32/tp4/result.json"
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns64/tp4/result.json"
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean": 127.7919646347385,
|
||||
"p90": 127.79196463473852
|
||||
}
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns8/tp4/result.json"
|
||||
}
|
||||
},
|
||||
"prefill_only": true,
|
||||
"real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/analysis/fixed-po/real-audit.json",
|
||||
"schema": "qwen30-latency-case-frontier-real-comparison-v1",
|
||||
"selection": {
|
||||
"e2e_ms:mean": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.0034531419781375313,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.005277055274057908,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.003463041945107654,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"agreement": 1.0,
|
||||
"concordant_pairs": 48,
|
||||
"discordant_pairs": 0,
|
||||
"informative_pairs": 48,
|
||||
"real_ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns32",
|
||||
"selected_config_real_regret": 0.005266030780421245,
|
||||
"sim_ranking": [
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns16",
|
||||
"tied_pairs": 18,
|
||||
"winner_match": false
|
||||
}
|
||||
},
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po"
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
# Qwen30 Frontier vs real latency selection
|
||||
|
||||
| Objective | Frontier | Real | Match | Regret | Pairwise |
|
||||
|---|---|---|---:|---:|---:|
|
||||
| ttft_ms:mean | tp4_mns16 | tp4_mns32 | no | 0.3% | 100.0% |
|
||||
| ttft_ms:p90 | tp4_mns16 | tp4_mns32 | no | 0.5% | 100.0% |
|
||||
| e2e_ms:mean | tp4_mns16 | tp4_mns32 | no | 0.3% | 100.0% |
|
||||
| e2e_ms:p90 | tp4_mns16 | tp4_mns32 | no | 0.5% | 100.0% |
|
||||
@@ -0,0 +1,997 @@
|
||||
{
|
||||
"applicable_metrics": [
|
||||
"ttft_ms",
|
||||
"e2e_ms"
|
||||
],
|
||||
"case_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po",
|
||||
"configs": {
|
||||
"tp1_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 311.5405924270961,
|
||||
"pooled_p90_ms": 313.7628498952836,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 311.5405924270961,
|
||||
"trial_stddev_of_means_ms": 1.3039412575877052
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 311.49589566449873,
|
||||
"pooled_p90_ms": 313.7203580699861,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 311.49589566449873,
|
||||
"trial_stddev_of_means_ms": 1.3039649961427384
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 310.6955070939125,
|
||||
"p90_ms": 311.31368898786604,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 310.65253322356983,
|
||||
"p90_ms": 311.2668279791251,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 310.8839521212648,
|
||||
"p90_ms": 311.4629740593955,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 310.837354891071,
|
||||
"p90_ms": 311.4192739594728,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 313.04231806611097,
|
||||
"p90_ms": 314.1881680348888,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 312.99779887885535,
|
||||
"p90_ms": 314.14560007397085,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 311.5105919916504,
|
||||
"pooled_p90_ms": 312.67331005074084,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 311.5105919916504,
|
||||
"trial_stddev_of_means_ms": 0.8370728024636384
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 311.4647780706316,
|
||||
"pooled_p90_ms": 312.62694601900876,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 311.4647780706316,
|
||||
"trial_stddev_of_means_ms": 0.8370818779367518
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 311.46654921424386,
|
||||
"p90_ms": 311.97385501582175,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 311.4168197754589,
|
||||
"p90_ms": 311.92471098620445,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 310.69641002473793,
|
||||
"p90_ms": 311.3162890076637,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 310.6527063394951,
|
||||
"p90_ms": 311.2770210718736,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 312.36881673596946,
|
||||
"p90_ms": 313.099671038799,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 312.3248080969408,
|
||||
"p90_ms": 313.04874701891094,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 312.1343500969265,
|
||||
"pooled_p90_ms": 313.89150698669255,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 312.1343500969265,
|
||||
"trial_stddev_of_means_ms": 0.46642112108308753
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 312.0841491107084,
|
||||
"pooled_p90_ms": 313.84513492230326,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 312.0841491107083,
|
||||
"trial_stddev_of_means_ms": 0.46656234571063265
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 311.59666514998946,
|
||||
"p90_ms": 313.3960220729932,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 311.546319629997,
|
||||
"p90_ms": 313.33198107313365,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 312.4300223219857,
|
||||
"p90_ms": 314.37386700417846,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 312.3801749132588,
|
||||
"p90_ms": 314.32278105057776,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 312.37636281880424,
|
||||
"p90_ms": 313.45780298579484,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 312.32595278886924,
|
||||
"p90_ms": 313.3977819234133,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 312.4610948317168,
|
||||
"pooled_p90_ms": 314.16700792033225,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 312.4610948317168,
|
||||
"trial_stddev_of_means_ms": 0.7149524029740935
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 312.4132300384316,
|
||||
"pooled_p90_ms": 314.1121099470183,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 312.4132300384316,
|
||||
"trial_stddev_of_means_ms": 0.7170007678929775
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 313.2865619732444,
|
||||
"p90_ms": 314.3677569460124,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 313.24100631591773,
|
||||
"p90_ms": 314.32040000800043,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 312.0378761661895,
|
||||
"p90_ms": 313.87826695572585,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 311.9859282132999,
|
||||
"p90_ms": 313.8229319592938,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 312.05884635571664,
|
||||
"p90_ms": 314.0185490483418,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 312.01275558607716,
|
||||
"p90_ms": 313.97431797813624,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 190.31096146856925,
|
||||
"pooled_p90_ms": 191.60501298028976,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 190.31096146856922,
|
||||
"trial_stddev_of_means_ms": 0.5354377240256494
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 190.26420441538883,
|
||||
"pooled_p90_ms": 191.55984604731202,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 190.26420441538883,
|
||||
"trial_stddev_of_means_ms": 0.5389874750392776
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 190.07371114938363,
|
||||
"p90_ms": 191.55416695866734,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 190.0275621526052,
|
||||
"p90_ms": 191.50883494876325,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 190.92403356246794,
|
||||
"p90_ms": 191.83196197263896,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 190.88103114138084,
|
||||
"p90_ms": 191.79061893373728,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 189.93513969385612,
|
||||
"p90_ms": 191.3305459311232,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 189.88401995218044,
|
||||
"p90_ms": 191.23233994469047,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 190.1976155063099,
|
||||
"pooled_p90_ms": 191.3924120599404,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 190.19761550630986,
|
||||
"trial_stddev_of_means_ms": 0.4305067654025827
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 190.15199505472157,
|
||||
"pooled_p90_ms": 191.34706107433885,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 190.1519950547216,
|
||||
"trial_stddev_of_means_ms": 0.4287092219589841
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 190.51239040683853,
|
||||
"p90_ms": 191.31595105864108,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 190.46796200088505,
|
||||
"p90_ms": 191.2707129959017,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 189.70702594537505,
|
||||
"p90_ms": 191.25314499251544,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 189.66398845564444,
|
||||
"p90_ms": 191.21390499640256,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 190.37343016671605,
|
||||
"p90_ms": 191.65610091295093,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 190.32403470763524,
|
||||
"p90_ms": 191.5953279240057,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 188.49150093478283,
|
||||
"pooled_p90_ms": 190.28858200181276,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 188.49150093478283,
|
||||
"trial_stddev_of_means_ms": 1.6533226552993832
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 188.44684911725727,
|
||||
"pooled_p90_ms": 190.2463739970699,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 188.44684911725724,
|
||||
"trial_stddev_of_means_ms": 1.650697495689325
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 189.15035290614583,
|
||||
"p90_ms": 190.26379100978374,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 189.106691004038,
|
||||
"p90_ms": 190.2244269149378,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 189.7138195680382,
|
||||
"p90_ms": 190.8034710213542,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 189.66555941971345,
|
||||
"p90_ms": 190.7602889696136,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 186.61033033016446,
|
||||
"p90_ms": 187.4971800716594,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 186.56829692802035,
|
||||
"p90_ms": 187.45643505826592,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 185.58736601101546,
|
||||
"pooled_p90_ms": 186.94222904741764,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 185.58736601101546,
|
||||
"trial_stddev_of_means_ms": 0.17900602117493175
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 185.54133709128124,
|
||||
"pooled_p90_ms": 186.88921502325684,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 185.54133709128124,
|
||||
"trial_stddev_of_means_ms": 0.17412945433635296
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 185.46277399905068,
|
||||
"p90_ms": 186.92227499559522,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 185.41920935401714,
|
||||
"p90_ms": 186.8766910629347,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 185.79249331095266,
|
||||
"p90_ms": 187.07626103423536,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 185.74072911579827,
|
||||
"p90_ms": 187.01596499886364,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 185.50683072304304,
|
||||
"p90_ms": 186.73917709384114,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 185.46407280402832,
|
||||
"p90_ms": 186.69728806708008,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 113.83082875217904,
|
||||
"pooled_p90_ms": 115.34429900348186,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.83082875217904,
|
||||
"trial_stddev_of_means_ms": 0.9461418127163538
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 113.78785713766635,
|
||||
"pooled_p90_ms": 115.29833707027137,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.78785713766636,
|
||||
"trial_stddev_of_means_ms": 0.9438950942019103
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 114.32171626612616,
|
||||
"p90_ms": 115.71828299202025,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 114.27366546969427,
|
||||
"p90_ms": 115.66144204698503,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns16/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 112.74012975786098,
|
||||
"p90_ms": 113.76826802734286,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 112.70000880352197,
|
||||
"p90_ms": 113.72840602416545,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns16/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 114.43064023254998,
|
||||
"p90_ms": 115.64736906439066,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 114.38989713978283,
|
||||
"p90_ms": 115.60670402832329,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns16/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 113.43910740842456,
|
||||
"pooled_p90_ms": 114.73881592974067,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.43910740842455,
|
||||
"trial_stddev_of_means_ms": 0.15342403653521097
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 113.39516492515813,
|
||||
"pooled_p90_ms": 114.69435307662934,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.39516492515811,
|
||||
"trial_stddev_of_means_ms": 0.15129782609682924
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.3981759986996,
|
||||
"p90_ms": 114.83101104386151,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.35384492909152,
|
||||
"p90_ms": 114.78637403342873,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns32/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.60884601692398,
|
||||
"p90_ms": 115.0080730440095,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.56283011230194,
|
||||
"p90_ms": 114.94788806885481,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns32/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.3103002096501,
|
||||
"p90_ms": 114.50691497884691,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.2688197340809,
|
||||
"p90_ms": 114.4664689199999,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns32/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 113.57268320527,
|
||||
"pooled_p90_ms": 115.08622206747532,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.57268320527,
|
||||
"trial_stddev_of_means_ms": 0.1834744740593822
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 113.52880459490815,
|
||||
"pooled_p90_ms": 115.04431103821844,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.52880459490815,
|
||||
"trial_stddev_of_means_ms": 0.18387162741378107
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.70978420210734,
|
||||
"p90_ms": 115.44408102054149,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.66415427361727,
|
||||
"p90_ms": 115.39141100365669,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns64/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.64400897406998,
|
||||
"p90_ms": 114.92579500190914,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.60279529025698,
|
||||
"p90_ms": 114.8861589608714,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns64/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.36425643963268,
|
||||
"p90_ms": 114.78174000512809,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.31946422085021,
|
||||
"p90_ms": 114.745571045205,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns64/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"pooled_mean_ms": 113.80060324454426,
|
||||
"pooled_p90_ms": 114.74568990524858,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.80060324454426,
|
||||
"trial_stddev_of_means_ms": 0.1793280070989901
|
||||
},
|
||||
"ttft_ms": {
|
||||
"pooled_mean_ms": 113.75673118469592,
|
||||
"pooled_p90_ms": 114.70199096947908,
|
||||
"pooled_samples": 771,
|
||||
"trial_mean_of_means_ms": 113.75673118469592,
|
||||
"trial_stddev_of_means_ms": 0.17717556416711291
|
||||
}
|
||||
},
|
||||
"trials": [
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.60963180313314,
|
||||
"p90_ms": 114.72724401392043,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.56837538357259,
|
||||
"p90_ms": 114.6864800248295,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns8/trial1/results/result.json",
|
||||
"trial": 1
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.82676389363706,
|
||||
"p90_ms": 115.33373803831637,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.78175165850293,
|
||||
"p90_ms": 115.29060895554721,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns8/trial2/results/result.json",
|
||||
"trial": 2
|
||||
},
|
||||
{
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ms": 113.96541403686258,
|
||||
"p90_ms": 114.28226600401103,
|
||||
"samples": 257
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ms": 113.92006651201224,
|
||||
"p90_ms": 114.24182495102286,
|
||||
"samples": 257
|
||||
}
|
||||
},
|
||||
"requests": 257,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns8/trial3/results/result.json",
|
||||
"trial": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"prefill_only": true,
|
||||
"schema": "qwen30-latency-case-real-audit-v1",
|
||||
"trace_manifests": {
|
||||
"tp1": {
|
||||
"description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse",
|
||||
"first_arrival_s": 0.0,
|
||||
"global_offered_request_rate": 1.125,
|
||||
"last_arrival_s": 227.55555555555554,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 1.125,
|
||||
"prefix_caching": false,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp1/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "eb048c65428f2530bdea135e6607405fd7ebaefb103267daaec29839462aa3ab",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp1/public/frontier.csv",
|
||||
"public_csv_sha256": "82c186222779c4a176f18e425d371966012ca7c1c6560f13d2c7036ac4f38d86",
|
||||
"requests": 257,
|
||||
"row_vector_sha256": "9829ec5834913fa49f900f4e07c8e83206726fc5a70d88de1498426ec91c51f3",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 1
|
||||
},
|
||||
"tp2": {
|
||||
"description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse",
|
||||
"first_arrival_s": 0.0,
|
||||
"global_offered_request_rate": 2.25,
|
||||
"last_arrival_s": 113.77777777777777,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 1.125,
|
||||
"prefix_caching": false,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp2/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "aa61aed87875ed792c37efdd62bfeee4753bd3de110a1de92fa32273be13ca12",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp2/public/frontier.csv",
|
||||
"public_csv_sha256": "b32676281c8f5651f47ee3d88a88d433a01a0863d8794751d2d638b988e1b8b2",
|
||||
"requests": 257,
|
||||
"row_vector_sha256": "1c032691cc4624b125e5589ee8466e46a67ddad7fe9e7380d5857bb81633cc3c",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 2
|
||||
},
|
||||
"tp4": {
|
||||
"description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse",
|
||||
"first_arrival_s": 0.0,
|
||||
"global_offered_request_rate": 4.5,
|
||||
"last_arrival_s": 56.888888888888886,
|
||||
"output_tokens": [
|
||||
1
|
||||
],
|
||||
"per_gpu_offered_request_rate": 1.125,
|
||||
"prefix_caching": false,
|
||||
"private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp4/private/real_requests.jsonl",
|
||||
"private_jsonl_sha256": "b82a17cd707c3c54069298b764031a1c18775571aa6e54c55be0746dd1fe94ca",
|
||||
"public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp4/public/frontier.csv",
|
||||
"public_csv_sha256": "67429bf430436a23979a0007707a03f4c67af962b9466cd230294a3c62696931",
|
||||
"requests": 257,
|
||||
"row_vector_sha256": "ab5af2eddc7d07420c6fcdf28b3a014d7241b2b35471ee581c6d7a8e3afb93e6",
|
||||
"schema": "qwen30-latency-case-v1",
|
||||
"tensor_parallel_size": 4
|
||||
}
|
||||
},
|
||||
"traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po",
|
||||
"winners": {
|
||||
"e2e_ms:pooled_mean_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"winner": "tp4_mns32",
|
||||
"winner_value_ms": 113.43910740842456
|
||||
},
|
||||
"e2e_ms:pooled_p90_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"winner": "tp4_mns32",
|
||||
"winner_value_ms": 114.73881592974067
|
||||
},
|
||||
"ttft_ms:pooled_mean_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"winner": "tp4_mns32",
|
||||
"winner_value_ms": 113.39516492515813
|
||||
},
|
||||
"ttft_ms:pooled_p90_ms": {
|
||||
"ranking": [
|
||||
"tp4_mns32",
|
||||
"tp4_mns8",
|
||||
"tp4_mns64",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp2_mns64",
|
||||
"tp2_mns32",
|
||||
"tp2_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns16",
|
||||
"tp1_mns64",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"winner": "tp4_mns32",
|
||||
"winner_value_ms": 114.69435307662934
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
# Qwen30 real latency case audit
|
||||
|
||||
Prefill-only: `True`.
|
||||
|
||||
| Objective | Real winner | Value (ms) |
|
||||
|---|---|---:|
|
||||
| ttft_ms:pooled_mean_ms | tp4_mns32 | 113.40 |
|
||||
| ttft_ms:pooled_p90_ms | tp4_mns32 | 114.69 |
|
||||
| e2e_ms:pooled_mean_ms | tp4_mns32 | 113.44 |
|
||||
| e2e_ms:pooled_p90_ms | tp4_mns32 | 114.74 |
|
||||
@@ -0,0 +1,726 @@
|
||||
{
|
||||
"cells": {
|
||||
"tp1_mns16": {
|
||||
"config": "tp1_mns16",
|
||||
"elapsed_seconds": 351.88990092277527,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 13.842098628638448,
|
||||
"p90_ratio_sim_over_real": 13.072241203568879,
|
||||
"real_mean_ms": 87355.16047904849,
|
||||
"real_p90_ms": 156177.17167094816,
|
||||
"sim_mean_ms": 1209178.7470715286,
|
||||
"sim_p90_ms": 2041585.6585738186
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.816528397308329,
|
||||
"p90_ratio_sim_over_real": 4.062740013130375,
|
||||
"real_mean_ms": 20.58492915882586,
|
||||
"real_p90_ms": 25.316827059621797,
|
||||
"sim_mean_ms": 99.14789585006503,
|
||||
"sim_p90_ms": 102.85568630062728
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 50.0951473194418,
|
||||
"p90_ratio_sim_over_real": 40.15634837465801,
|
||||
"real_mean_ms": 17378.17525871318,
|
||||
"real_p90_ms": 42739.809082006104,
|
||||
"sim_mean_ms": 870562.2497283154,
|
||||
"sim_p90_ms": 1716274.6629634094
|
||||
}
|
||||
},
|
||||
"mns": 16,
|
||||
"offered_request_rate": 0.215,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns16/tp1/result.json",
|
||||
"result_sha256": "d892cbc79c7ac3a2158310d346aea87fdd4a4515ada6465d02087066e412fe89",
|
||||
"tp": 1,
|
||||
"trace_label": "tp1",
|
||||
"trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a"
|
||||
},
|
||||
"tp1_mns32": {
|
||||
"config": "tp1_mns32",
|
||||
"elapsed_seconds": 342.20933651924133,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 8.911637753525204,
|
||||
"p90_ratio_sim_over_real": 7.8084062855710945,
|
||||
"real_mean_ms": 81051.19884917597,
|
||||
"real_p90_ms": 162908.61000702716,
|
||||
"sim_mean_ms": 722298.9236327951,
|
||||
"sim_p90_ms": 1272056.614352521
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 5.03371341949048,
|
||||
"p90_ratio_sim_over_real": 4.17277168119042,
|
||||
"real_mean_ms": 23.75712112165005,
|
||||
"real_p90_ms": 29.99736783063116,
|
||||
"sim_mean_ms": 119.5865393985106,
|
||||
"sim_p90_ms": 125.1721669939102
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 510.8831583325707,
|
||||
"p90_ratio_sim_over_real": 384.77102901825737,
|
||||
"real_mean_ms": 620.4562074374239,
|
||||
"real_p90_ms": 1931.474626995623,
|
||||
"sim_mean_ms": 316980.6268626798,
|
||||
"sim_p90_ms": 743175.4797517607
|
||||
}
|
||||
},
|
||||
"mns": 32,
|
||||
"offered_request_rate": 0.215,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns32/tp1/result.json",
|
||||
"result_sha256": "babdafcbea1bd890e9d2af8043ef104fac262ef2b2ae6f9eca9ed6e3787824b8",
|
||||
"tp": 1,
|
||||
"trace_label": "tp1",
|
||||
"trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a"
|
||||
},
|
||||
"tp1_mns64": {
|
||||
"config": "tp1_mns64",
|
||||
"elapsed_seconds": 347.8728907108307,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 6.680377846313092,
|
||||
"p90_ratio_sim_over_real": 5.889817911531751,
|
||||
"real_mean_ms": 82253.7312629709,
|
||||
"real_p90_ms": 165355.2623600699,
|
||||
"sim_mean_ms": 549486.0041057414,
|
||||
"sim_p90_ms": 973912.3860143718
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 5.715675260466855,
|
||||
"p90_ratio_sim_over_real": 4.93808404053013,
|
||||
"real_mean_ms": 24.114592194742286,
|
||||
"real_p90_ms": 30.355145227612027,
|
||||
"sim_mean_ms": 137.83117802373562,
|
||||
"sim_p90_ms": 149.8962581964453
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 169.41590110493394,
|
||||
"p90_ratio_sim_over_real": 153.2295314740789,
|
||||
"real_mean_ms": 619.7960944975623,
|
||||
"real_p90_ms": 1956.7296199966222,
|
||||
"sim_mean_ms": 105003.31385062331,
|
||||
"sim_p90_ms": 299828.76289353485
|
||||
}
|
||||
},
|
||||
"mns": 64,
|
||||
"offered_request_rate": 0.215,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns64/tp1/result.json",
|
||||
"result_sha256": "a870a95f9f088e0c098025d3d7d4001409b28ab26f8633ff27ea878d2bbd50d2",
|
||||
"tp": 1,
|
||||
"trace_label": "tp1",
|
||||
"trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a"
|
||||
},
|
||||
"tp1_mns8": {
|
||||
"config": "tp1_mns8",
|
||||
"elapsed_seconds": 369.526850938797,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 13.991474955968794,
|
||||
"p90_ratio_sim_over_real": 15.108002258550664,
|
||||
"real_mean_ms": 137996.3827215202,
|
||||
"real_p90_ms": 222136.04307791684,
|
||||
"sim_mean_ms": 1930772.9328624348,
|
||||
"sim_p90_ms": 3356031.8405266753
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 5.3194009645510905,
|
||||
"p90_ratio_sim_over_real": 4.790770710747499,
|
||||
"real_mean_ms": 14.282724020345517,
|
||||
"real_p90_ms": 16.331995667962982,
|
||||
"sim_mean_ms": 75.97553593024297,
|
||||
"sim_p90_ms": 78.2428464941321
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 18.86307572133342,
|
||||
"p90_ratio_sim_over_real": 18.60057703879594,
|
||||
"real_mean_ms": 88479.94003801016,
|
||||
"real_p90_ms": 168185.5451229494,
|
||||
"sim_mean_ms": 1669003.8087560262,
|
||||
"sim_p90_ms": 3128348.188871311
|
||||
}
|
||||
},
|
||||
"mns": 8,
|
||||
"offered_request_rate": 0.215,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns8/tp1/result.json",
|
||||
"result_sha256": "0db2d1f388f20252bea08c982cd53c24c3d33c2d3b36b70bda82c4e501b694e6",
|
||||
"tp": 1,
|
||||
"trace_label": "tp1",
|
||||
"trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a"
|
||||
},
|
||||
"tp2_mns16": {
|
||||
"config": "tp2_mns16",
|
||||
"elapsed_seconds": 282.5357069969177,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 9.98451139764714,
|
||||
"p90_ratio_sim_over_real": 10.19790282414335,
|
||||
"real_mean_ms": 82936.70731938194,
|
||||
"real_p90_ms": 137726.37015290093,
|
||||
"sim_mean_ms": 828082.499513694,
|
||||
"sim_p90_ms": 1404520.1391412807
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.569770850102063,
|
||||
"p90_ratio_sim_over_real": 4.120000340068115,
|
||||
"real_mean_ms": 14.09701977962943,
|
||||
"real_p90_ms": 16.149621762221777,
|
||||
"sim_mean_ms": 64.42015006226278,
|
||||
"sim_p90_ms": 66.53644715232515
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 17.620001287194583,
|
||||
"p90_ratio_sim_over_real": 15.324208422508217,
|
||||
"real_mean_ms": 34442.970551674625,
|
||||
"real_p90_ms": 77922.3588249879,
|
||||
"sim_mean_ms": 606885.185455312,
|
||||
"sim_p90_ms": 1194098.467407587
|
||||
}
|
||||
},
|
||||
"mns": 16,
|
||||
"offered_request_rate": 0.43,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns16/tp2/result.json",
|
||||
"result_sha256": "20b811f38251d62930977529a55c434d62aede91b22ccaeec812a493999edbba",
|
||||
"tp": 2,
|
||||
"trace_label": "tp2",
|
||||
"trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef"
|
||||
},
|
||||
"tp2_mns32": {
|
||||
"config": "tp2_mns32",
|
||||
"elapsed_seconds": 341.36838245391846,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 7.698077489599209,
|
||||
"p90_ratio_sim_over_real": 7.1828008112452935,
|
||||
"real_mean_ms": 62267.50376182025,
|
||||
"real_p90_ms": 117239.33770309668,
|
||||
"sim_mean_ms": 479340.0690424025,
|
||||
"sim_p90_ms": 842106.8099636638
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.155703711688009,
|
||||
"p90_ratio_sim_over_real": 3.6490372704567418,
|
||||
"real_mean_ms": 17.99748422275332,
|
||||
"real_p90_ms": 21.280696114935413,
|
||||
"sim_mean_ms": 74.79221198554237,
|
||||
"sim_p90_ms": 77.6540532646633
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 213.67619737350176,
|
||||
"p90_ratio_sim_over_real": 204.45454462882793,
|
||||
"real_mean_ms": 1050.6337350342683,
|
||||
"real_p90_ms": 2521.7670870479196,
|
||||
"sim_mean_ms": 224495.42133444166,
|
||||
"sim_p90_ms": 515586.7414423483
|
||||
}
|
||||
},
|
||||
"mns": 32,
|
||||
"offered_request_rate": 0.43,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns32/tp2/result.json",
|
||||
"result_sha256": "38b6ff12a60024e98bbb27ddea682439059b5b8481b661bd7b1ac720b3351b7d",
|
||||
"tp": 2,
|
||||
"trace_label": "tp2",
|
||||
"trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef"
|
||||
},
|
||||
"tp2_mns64": {
|
||||
"config": "tp2_mns64",
|
||||
"elapsed_seconds": 342.2184634208679,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 4.990247689497755,
|
||||
"p90_ratio_sim_over_real": 4.806085707327923,
|
||||
"real_mean_ms": 62353.63015175185,
|
||||
"real_p90_ms": 118392.33276096638,
|
||||
"sim_mean_ms": 311160.0587965772,
|
||||
"sim_p90_ms": 569003.698339692
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.529200811393724,
|
||||
"p90_ratio_sim_over_real": 4.064210807294688,
|
||||
"real_mean_ms": 18.239052597893746,
|
||||
"real_p90_ms": 21.560091553416488,
|
||||
"sim_mean_ms": 82.60833182543317,
|
||||
"sim_p90_ms": 87.6247570976582
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 81.23526532158742,
|
||||
"p90_ratio_sim_over_real": 98.0031513549519,
|
||||
"real_mean_ms": 375.75464864569483,
|
||||
"real_p90_ms": 1148.593939957209,
|
||||
"sim_mean_ms": 30524.52857855288,
|
||||
"sim_p90_ms": 112565.82574300688
|
||||
}
|
||||
},
|
||||
"mns": 64,
|
||||
"offered_request_rate": 0.43,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns64/tp2/result.json",
|
||||
"result_sha256": "2293c4ab15356f6f0753a0cec6e4113b4c74f89f7564032a0c1d58ebd97383ee",
|
||||
"tp": 2,
|
||||
"trace_label": "tp2",
|
||||
"trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef"
|
||||
},
|
||||
"tp2_mns8": {
|
||||
"config": "tp2_mns8",
|
||||
"elapsed_seconds": 369.0675718784332,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 10.511662746977503,
|
||||
"p90_ratio_sim_over_real": 10.881956427090019,
|
||||
"real_mean_ms": 128296.94038681783,
|
||||
"real_p90_ms": 215848.26136403717,
|
||||
"sim_mean_ms": 1348614.1688153066,
|
||||
"sim_p90_ms": 2348851.3750265907
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 5.523719280021353,
|
||||
"p90_ratio_sim_over_real": 5.149568696131382,
|
||||
"real_mean_ms": 9.244370614307101,
|
||||
"real_p90_ms": 10.190386958999568,
|
||||
"sim_mean_ms": 51.06330819391097,
|
||||
"sim_p90_ms": 52.476097685529645
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 12.214904033795035,
|
||||
"p90_ratio_sim_over_real": 11.92062856279977,
|
||||
"real_mean_ms": 95920.08463817694,
|
||||
"real_p90_ms": 183878.83744493593,
|
||||
"sim_mean_ms": 1171654.6287688287,
|
||||
"sim_p90_ms": 2191951.3217405193
|
||||
}
|
||||
},
|
||||
"mns": 8,
|
||||
"offered_request_rate": 0.43,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns8/tp2/result.json",
|
||||
"result_sha256": "0d75820000a471d607f5c0eac3571355d6666235e3bb1ae69080fa20fbf36160",
|
||||
"tp": 2,
|
||||
"trace_label": "tp2",
|
||||
"trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef"
|
||||
},
|
||||
"tp4_mns16": {
|
||||
"config": "tp4_mns16",
|
||||
"elapsed_seconds": 353.6330111026764,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 8.790176207252758,
|
||||
"p90_ratio_sim_over_real": 8.919807800227431,
|
||||
"real_mean_ms": 65779.08377038215,
|
||||
"real_p90_ms": 110374.3328540586,
|
||||
"sim_mean_ms": 578209.7370932992,
|
||||
"sim_p90_ms": 984517.8351365307
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.905169954916736,
|
||||
"p90_ratio_sim_over_real": 4.616446999000005,
|
||||
"real_mean_ms": 8.806983272811275,
|
||||
"real_p90_ms": 9.626243685920313,
|
||||
"sim_mean_ms": 43.19974974324813,
|
||||
"sim_p90_ms": 44.439043775509575
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 12.245054441740509,
|
||||
"p90_ratio_sim_over_real": 11.334316081110904,
|
||||
"real_mean_ms": 35042.61012925094,
|
||||
"real_p90_ms": 74281.28085809294,
|
||||
"sim_mean_ms": 429098.66881336516,
|
||||
"sim_p90_ms": 841927.5161553983
|
||||
}
|
||||
},
|
||||
"mns": 16,
|
||||
"offered_request_rate": 0.86,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns16/tp4/result.json",
|
||||
"result_sha256": "c174b3fec57e9bebcd3e41a407c4ea17b6b44929f440336489e529c2ec4e1ffc",
|
||||
"tp": 4,
|
||||
"trace_label": "tp4",
|
||||
"trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69"
|
||||
},
|
||||
"tp4_mns32": {
|
||||
"config": "tp4_mns32",
|
||||
"elapsed_seconds": 342.3954610824585,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 7.0145630517428215,
|
||||
"p90_ratio_sim_over_real": 6.923394844590732,
|
||||
"real_mean_ms": 47944.52887711636,
|
||||
"real_p90_ms": 83825.7212981116,
|
||||
"sim_mean_ms": 336309.9207946372,
|
||||
"sim_p90_ms": 580358.5666794453
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.058731155235696,
|
||||
"p90_ratio_sim_over_real": 3.7164401630936927,
|
||||
"real_mean_ms": 12.181116414313188,
|
||||
"real_p90_ms": 13.75045588274547,
|
||||
"sim_mean_ms": 49.43987669632586,
|
||||
"sim_p90_ms": 51.1027465034832
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 26.481641572689746,
|
||||
"p90_ratio_sim_over_real": 19.75959391638082,
|
||||
"real_mean_ms": 6296.228585964807,
|
||||
"real_p90_ms": 19159.870272967964,
|
||||
"sim_mean_ms": 166734.46867324322,
|
||||
"sim_p90_ms": 378591.2560843835
|
||||
}
|
||||
},
|
||||
"mns": 32,
|
||||
"offered_request_rate": 0.86,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns32/tp4/result.json",
|
||||
"result_sha256": "46efd5bd348ce93a9dc075441ebfd1f2df5ff1646c53610e8f6bdd72105071e4",
|
||||
"tp": 4,
|
||||
"trace_label": "tp4",
|
||||
"trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69"
|
||||
},
|
||||
"tp4_mns64": {
|
||||
"config": "tp4_mns64",
|
||||
"elapsed_seconds": 340.36316752433777,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 4.663503091378936,
|
||||
"p90_ratio_sim_over_real": 4.414200438612224,
|
||||
"real_mean_ms": 44985.13828221082,
|
||||
"real_p90_ms": 83763.61756504048,
|
||||
"sim_mean_ms": 209788.3314451991,
|
||||
"sim_p90_ms": 369749.3973953483
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 4.078641583467541,
|
||||
"p90_ratio_sim_over_real": 3.6320345146531077,
|
||||
"real_mean_ms": 13.178025610291787,
|
||||
"real_p90_ms": 15.389331985205848,
|
||||
"sim_mean_ms": 53.748443242136304,
|
||||
"sim_p90_ms": 55.89458492772267
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 108.51050461378274,
|
||||
"p90_ratio_sim_over_real": 152.34030824222364,
|
||||
"real_mean_ms": 245.9527667526406,
|
||||
"real_p90_ms": 685.5076659703627,
|
||||
"sim_mean_ms": 26688.458831485037,
|
||||
"sim_p90_ms": 104430.44913633233
|
||||
}
|
||||
},
|
||||
"mns": 64,
|
||||
"offered_request_rate": 0.86,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns64/tp4/result.json",
|
||||
"result_sha256": "c6c34b800a646938f507dde0a5e2530f3e94358b9580050845cfd255176dd391",
|
||||
"tp": 4,
|
||||
"trace_label": "tp4",
|
||||
"trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69"
|
||||
},
|
||||
"tp4_mns8": {
|
||||
"config": "tp4_mns8",
|
||||
"elapsed_seconds": 379.0038528442383,
|
||||
"metrics": {
|
||||
"e2e_ms": {
|
||||
"mean_ratio_sim_over_real": 7.5775112842809955,
|
||||
"p90_ratio_sim_over_real": 7.997579501403945,
|
||||
"real_mean_ms": 126405.41922729714,
|
||||
"real_p90_ms": 209092.4072249327,
|
||||
"sim_mean_ms": 957838.490589114,
|
||||
"sim_p90_ms": 1672233.1499213278
|
||||
},
|
||||
"tpot_ms": {
|
||||
"mean_ratio_sim_over_real": 5.242973269896895,
|
||||
"p90_ratio_sim_over_real": 4.960553326136223,
|
||||
"real_mean_ms": 6.757398869596114,
|
||||
"real_p90_ms": 7.242299378971089,
|
||||
"sim_mean_ms": 35.42886164732392,
|
||||
"sim_p90_ms": 35.92581227322934
|
||||
},
|
||||
"ttft_ms": {
|
||||
"mean_ratio_sim_over_real": 8.133065025278983,
|
||||
"p90_ratio_sim_over_real": 8.084332290199812,
|
||||
"real_mean_ms": 102605.87212059843,
|
||||
"real_p90_ms": 193359.7398570273,
|
||||
"sim_mean_ms": 834500.2299322869,
|
||||
"sim_p90_ms": 1563184.3885508017
|
||||
}
|
||||
},
|
||||
"mns": 8,
|
||||
"offered_request_rate": 0.86,
|
||||
"offered_request_rate_per_gpu": 0.215,
|
||||
"request_count": 129,
|
||||
"result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns8/tp4/result.json",
|
||||
"result_sha256": "9361fe342d3b0394d6a9b130965304923a6edf424d0b964368079b0baacf9e75",
|
||||
"tp": 4,
|
||||
"trace_label": "tp4",
|
||||
"trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69"
|
||||
}
|
||||
},
|
||||
"contract": {
|
||||
"configs": [
|
||||
"tp1_mns8",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp2_mns8",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64"
|
||||
],
|
||||
"graph_semantics": "Frontier piecewise; CUDA_EVENT for prefill/mixed and KERNEL_ONLY for captured decode",
|
||||
"real_trials_per_cell": 3,
|
||||
"request_count_per_cell": 129,
|
||||
"trace_policy": "TP-normalized arrivals with full 16-token prefix blocks only"
|
||||
},
|
||||
"real_audit": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/ground-truth/qwen30-tracepd-u0p01-real-audit.json",
|
||||
"real_audit_sha256": "6e2c17bce5b0edb5f6f92aae4cc1b00f9390f692d31d059efe647158b386c8e9",
|
||||
"schema": "frontier-qwen30-piecewise-graph-comparison-v1",
|
||||
"selection": {
|
||||
"e2e_ms:mean": {
|
||||
"concordant_pairs": 61,
|
||||
"discordant_pairs": 5,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.9242424242424242,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp4_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns64",
|
||||
"tp4_mns16",
|
||||
"tp1_mns32",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"concordant_pairs": 55,
|
||||
"discordant_pairs": 11,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.8333333333333334,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns64",
|
||||
"tp4_mns16",
|
||||
"tp1_mns32",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_ms:mean": {
|
||||
"concordant_pairs": 64,
|
||||
"discordant_pairs": 2,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.9696969696969697,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"sim_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp2_mns8",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp1_mns8",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns8",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_ms:p90": {
|
||||
"concordant_pairs": 64,
|
||||
"discordant_pairs": 2,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.9696969696969697,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"sim_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp2_mns8",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp1_mns8",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns8",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"concordant_pairs": 56,
|
||||
"discordant_pairs": 10,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.8484848484848485,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp1_mns16",
|
||||
"tp2_mns16",
|
||||
"tp4_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns8",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"concordant_pairs": 56,
|
||||
"discordant_pairs": 10,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.8484848484848485,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp1_mns16",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns8",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
}
|
||||
},
|
||||
"sim_root": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2"
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
# Frontier piecewise graph-profile vs. real vLLM
|
||||
|
||||
Each cell uses its TP-normalized, 129-request trace. Frontier values are one deterministic simulation; real values pool three fresh-server trials (387 requests/cell).
|
||||
|
||||
| Config | TTFT sim / real mean (ms) | TPOT sim / real mean (ms) | E2E sim / real mean (ms) |
|
||||
|---|---:|---:|---:|
|
||||
| tp1_mns8 | 1669003.8 / 88479.9 | 76.0 / 14.3 | 1930772.9 / 137996.4 |
|
||||
| tp1_mns16 | 870562.2 / 17378.2 | 99.1 / 20.6 | 1209178.7 / 87355.2 |
|
||||
| tp1_mns32 | 316980.6 / 620.5 | 119.6 / 23.8 | 722298.9 / 81051.2 |
|
||||
| tp1_mns64 | 105003.3 / 619.8 | 137.8 / 24.1 | 549486.0 / 82253.7 |
|
||||
| tp2_mns8 | 1171654.6 / 95920.1 | 51.1 / 9.2 | 1348614.2 / 128296.9 |
|
||||
| tp2_mns16 | 606885.2 / 34443.0 | 64.4 / 14.1 | 828082.5 / 82936.7 |
|
||||
| tp2_mns32 | 224495.4 / 1050.6 | 74.8 / 18.0 | 479340.1 / 62267.5 |
|
||||
| tp2_mns64 | 30524.5 / 375.8 | 82.6 / 18.2 | 311160.1 / 62353.6 |
|
||||
| tp4_mns8 | 834500.2 / 102605.9 | 35.4 / 6.8 | 957838.5 / 126405.4 |
|
||||
| tp4_mns16 | 429098.7 / 35042.6 | 43.2 / 8.8 | 578209.7 / 65779.1 |
|
||||
| tp4_mns32 | 166734.5 / 6296.2 | 49.4 / 12.2 | 336309.9 / 47944.5 |
|
||||
| tp4_mns64 | 26688.5 / 246.0 | 53.7 / 13.2 | 209788.3 / 44985.1 |
|
||||
|
||||
## Selection agreement
|
||||
|
||||
| Target | Frontier winner | Real winner | Match | Pairwise agreement |
|
||||
|---|---|---|---:|---:|
|
||||
| ttft_ms:mean | tp4_mns64 | tp4_mns64 | yes | 84.8% (56/66) |
|
||||
| ttft_ms:p90 | tp4_mns64 | tp4_mns64 | yes | 84.8% (56/66) |
|
||||
| tpot_ms:mean | tp4_mns8 | tp4_mns8 | yes | 97.0% (64/66) |
|
||||
| tpot_ms:p90 | tp4_mns8 | tp4_mns8 | yes | 97.0% (64/66) |
|
||||
| e2e_ms:mean | tp4_mns64 | tp4_mns64 | yes | 92.4% (61/66) |
|
||||
| e2e_ms:p90 | tp4_mns64 | tp4_mns64 | yes | 83.3% (55/66) |
|
||||
@@ -0,0 +1,2 @@
|
||||
ANALYSIS_LAUNCH_ECHO sim_root=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2 real_audit=/home/admin/cpfs/wjh/aituner/simulator-tuning-latency-q30-tp-normalized-u0p01-20260717/qwen30-tracepd-u0p01-real-audit.json outputs=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis expected=12 completed cells, rank+latency comparison
|
||||
ANALYSIS_RETRY_ECHO sim_root=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2 real_audit_sha256=6e2c17bce5b0edb5f6f92aae4cc1b00f9390f692d31d059efe647158b386c8e9 output=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis
|
||||
@@ -0,0 +1,230 @@
|
||||
{
|
||||
"e2e_ms:mean": {
|
||||
"concordant_pairs": 61,
|
||||
"discordant_pairs": 5,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.9242424242424242,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp4_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns64",
|
||||
"tp4_mns16",
|
||||
"tp1_mns32",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"e2e_ms:p90": {
|
||||
"concordant_pairs": 55,
|
||||
"discordant_pairs": 11,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.8333333333333334,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp4_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp4_mns8",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns64",
|
||||
"tp4_mns16",
|
||||
"tp1_mns32",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_ms:mean": {
|
||||
"concordant_pairs": 64,
|
||||
"discordant_pairs": 2,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.9696969696969697,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"sim_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp2_mns8",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp1_mns8",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns8",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"tpot_ms:p90": {
|
||||
"concordant_pairs": 64,
|
||||
"discordant_pairs": 2,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.9696969696969697,
|
||||
"real_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp2_mns8",
|
||||
"tp4_mns32",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns32",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"real_winner": "tp4_mns8",
|
||||
"sim_ranking": [
|
||||
"tp4_mns8",
|
||||
"tp4_mns16",
|
||||
"tp4_mns32",
|
||||
"tp2_mns8",
|
||||
"tp4_mns64",
|
||||
"tp2_mns16",
|
||||
"tp2_mns32",
|
||||
"tp1_mns8",
|
||||
"tp2_mns64",
|
||||
"tp1_mns16",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64"
|
||||
],
|
||||
"sim_winner": "tp4_mns8",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_ms:mean": {
|
||||
"concordant_pairs": 56,
|
||||
"discordant_pairs": 10,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.8484848484848485,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp1_mns32",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp1_mns16",
|
||||
"tp2_mns16",
|
||||
"tp4_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns8",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
},
|
||||
"ttft_ms:p90": {
|
||||
"concordant_pairs": 56,
|
||||
"discordant_pairs": 10,
|
||||
"informative_pairs": 66,
|
||||
"pairwise_agreement_fraction": 0.8484848484848485,
|
||||
"real_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns32",
|
||||
"tp1_mns64",
|
||||
"tp2_mns32",
|
||||
"tp4_mns32",
|
||||
"tp1_mns16",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp1_mns8",
|
||||
"tp2_mns8",
|
||||
"tp4_mns8"
|
||||
],
|
||||
"real_winner": "tp4_mns64",
|
||||
"sim_ranking": [
|
||||
"tp4_mns64",
|
||||
"tp2_mns64",
|
||||
"tp1_mns64",
|
||||
"tp4_mns32",
|
||||
"tp2_mns32",
|
||||
"tp1_mns32",
|
||||
"tp4_mns16",
|
||||
"tp2_mns16",
|
||||
"tp4_mns8",
|
||||
"tp1_mns16",
|
||||
"tp2_mns8",
|
||||
"tp1_mns8"
|
||||
],
|
||||
"sim_winner": "tp4_mns64",
|
||||
"tied_pairs": 0,
|
||||
"winner_match": true
|
||||
}
|
||||
}
|
||||
BIN
runs/frontier-split-rootcause-v0/results/decision-pair-axis.png
Normal file
BIN
runs/frontier-split-rootcause-v0/results/decision-pair-axis.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 124 KiB |
7410
runs/frontier-split-rootcause-v0/results/decomposition.json
Normal file
7410
runs/frontier-split-rootcause-v0/results/decomposition.json
Normal file
File diff suppressed because it is too large
Load Diff
301
runs/frontier-split-rootcause-v0/results/decomposition.md
Normal file
301
runs/frontier-split-rootcause-v0/results/decomposition.md
Normal file
@@ -0,0 +1,301 @@
|
||||
# Frontier good/bad split decomposition
|
||||
|
||||
All 70 rows are recomputed from frozen per-config JSON values. Lower is better for every objective. `scale` is the geometric mean of `sim/real`; spreads are shown as `100 ×` log-space width, while `real margin` is the ordinary relative difference `100 × (real_second / real_best - 1)`. H-SCALE compares the log-space spread with `log1p(real margin)`, not directly with the ordinary relative margin. The real top-3 neighborhood is selected independently for every row. Directional `m_s` and `delta_s` are already expressed in log-percentage points.
|
||||
|
||||
- Rows / case surfaces: 70 / 14.
|
||||
- Failures (`regret > 5%`): 23.
|
||||
- Exact winner mismatches: 47.
|
||||
- H-SCALE counterexamples: 40.
|
||||
|
||||
## Unified decomposition
|
||||
|
||||
| case | objective | sim winner | real winner | regret | scale | spread all | spread real top-3 | real margin (relative) | sim-winner real loss | failure | H-SCALE | deciding axis | m_s | delta_s | n_inv_tp | n_inv_mns | n_inv_mixed | fragile success |
|
||||
|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---:|---:|---:|---:|---:|---|
|
||||
| q30 trace-pd | ttft_mean_ms | tp4_mns64 | tp4_mns64 | 0.0% | 45.900x | 414.0% | 73.5% | 52.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 trace-pd | ttft_p90_ms | tp4_mns64 | tp4_mns64 | 0.0% | 43.384x | 386.3% | 136.8% | 67.6% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 trace-pd | tpot_mean_ms | tp4_mns8 | tp4_mns8 | 0.0% | 4.799x | 34.2% | 11.9% | 30.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 trace-pd | tpot_p90_ms | tp4_mns8 | tp4_mns8 | 0.0% | 4.291x | 34.9% | 10.9% | 32.9% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 trace-pd | e2e_mean_ms | tp4_mns64 | tp4_mns64 | 0.0% | 8.263x | 109.9% | 50.1% | 6.6% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 trace-pd | e2e_p90_ms | tp4_mns64 | tp4_mns64 | 0.0% | 8.066x | 123.0% | 70.3% | 0.1% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes |
|
||||
| q30 fixed-pd high | ttft_mean_ms | tp2_mns64 | tp4_mns32 | 58.0% | 80.274x | 664.7% | 229.2% | 0.0% | 58.0% | yes | HOLDS | mixed | 45.7% | -523.8% | 2 | 1 | 2 | no |
|
||||
| q30 fixed-pd high | ttft_p90_ms | tp2_mns64 | tp4_mns64 | 56.2% | 130.663x | 709.3% | 222.7% | 0.1% | 56.2% | yes | HOLDS | tp-axis | 44.6% | -435.3% | 2 | 0 | 0 | no |
|
||||
| q30 fixed-pd high | tpot_mean_ms | tp4_mns8 | tp4_mns8 | 0.0% | 6.313x | 118.1% | 95.7% | 34.4% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-pd high | tpot_p90_ms | tp4_mns8 | tp4_mns8 | 0.0% | 5.885x | 122.3% | 95.1% | 37.2% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-pd high | e2e_mean_ms | tp4_mns64 | tp1_mns32 | 1.7% | 18.867x | 226.5% | 111.5% | 0.7% | 1.7% | no | COUNTEREXAMPLE | mixed | 1.7% | -82.6% | 2 | 1 | 2 | no |
|
||||
| q30 fixed-pd high | e2e_p90_ms | tp2_mns64 | tp1_mns32 | 5.5% | 24.706x | 274.2% | 172.5% | 1.0% | 5.5% | yes | HOLDS | mixed | 5.3% | -101.7% | 1 | 1 | 2 | no |
|
||||
| q30 fixed-po high | ttft_mean_ms | tp4_mns16 | tp4_mns32 | 0.3% | 0.964x | 27.5% | 0.3% | 0.1% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-po high | ttft_p90_ms | tp4_mns16 | tp4_mns32 | 0.5% | 0.956x | 26.9% | 0.3% | 0.0% | 0.5% | no | COUNTEREXAMPLE | mns-axis | 0.5% | -0.5% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-po high | e2e_mean_ms | tp4_mns16 | tp4_mns32 | 0.3% | 0.964x | 27.5% | 0.3% | 0.1% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-po high | e2e_p90_ms | tp4_mns16 | tp4_mns32 | 0.5% | 0.956x | 26.9% | 0.3% | 0.0% | 0.5% | no | COUNTEREXAMPLE | mns-axis | 0.5% | -0.5% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-pd low | ttft_mean_ms | tp4_mns16 | tp4_mns16 | 0.0% | 1.368x | 43.8% | 1.5% | 1.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-pd low | ttft_p90_ms | tp4_mns16 | tp4_mns16 | 0.0% | 1.402x | 48.7% | 2.9% | 1.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-pd low | tpot_mean_ms | tp1_mns16 | tp4_mns32 | 14.0% | 4.952x | 52.7% | 0.2% | 0.0% | 14.0% | yes | HOLDS | mixed | 13.1% | -52.5% | 2 | 0 | 6 | no |
|
||||
| q30 fixed-pd low | tpot_p90_ms | tp1_mns16 | tp4_mns8 | 13.4% | 4.950x | 53.2% | 0.3% | 0.2% | 13.4% | yes | HOLDS | mixed | 12.5% | -52.8% | 2 | 0 | 6 | no |
|
||||
| q30 fixed-pd low | e2e_mean_ms | tp1_mns16 | tp4_mns32 | 26.9% | 4.423x | 59.8% | 0.2% | 0.1% | 26.9% | yes | HOLDS | mixed | 23.8% | -59.6% | 2 | 0 | 6 | no |
|
||||
| q30 fixed-pd low | e2e_p90_ms | tp1_mns16 | tp4_mns8 | 25.7% | 4.421x | 60.0% | 0.2% | 0.1% | 25.7% | yes | HOLDS | mixed | 22.9% | -59.8% | 2 | 0 | 6 | no |
|
||||
| q30 fixed-po low | ttft_mean_ms | tp4_mns16 | tp4_mns8 | 1.5% | 1.218x | 20.9% | 1.1% | 0.3% | 1.5% | no | COUNTEREXAMPLE | mns-axis | 1.5% | -1.5% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-po low | ttft_p90_ms | tp4_mns16 | tp4_mns8 | 3.6% | 1.184x | 16.1% | 2.3% | 0.6% | 3.6% | no | COUNTEREXAMPLE | mns-axis | 3.5% | -3.5% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-po low | e2e_mean_ms | tp4_mns16 | tp4_mns8 | 1.5% | 1.217x | 20.8% | 1.1% | 0.3% | 1.5% | no | COUNTEREXAMPLE | mns-axis | 1.5% | -1.5% | 0 | 0 | 0 | no |
|
||||
| q30 fixed-po low | e2e_p90_ms | tp4_mns16 | tp4_mns8 | 3.6% | 1.183x | 16.0% | 2.4% | 0.7% | 3.6% | no | COUNTEREXAMPLE | mns-axis | 3.5% | -3.5% | 0 | 0 | 0 | no |
|
||||
| q30 trace-po low | ttft_mean_ms | tp4_mns16 | tp4_mns64 | 3.2% | 0.657x | 18.2% | 2.6% | 1.7% | 3.2% | no | COUNTEREXAMPLE | mns-axis | 3.1% | -3.1% | 0 | 3 | 0 | no |
|
||||
| q30 trace-po low | ttft_p90_ms | tp4_mns16 | tp4_mns8 | 0.4% | 0.478x | 17.7% | 0.2% | 0.2% | 0.4% | no | COUNTEREXAMPLE | mns-axis | 0.4% | -0.4% | 0 | 0 | 0 | no |
|
||||
| q30 trace-po low | e2e_mean_ms | tp4_mns16 | tp4_mns64 | 3.2% | 0.657x | 18.2% | 2.6% | 1.7% | 3.2% | no | COUNTEREXAMPLE | mns-axis | 3.1% | -3.1% | 0 | 3 | 0 | no |
|
||||
| q30 trace-po low | e2e_p90_ms | tp4_mns16 | tp4_mns8 | 0.3% | 0.478x | 17.7% | 0.2% | 0.1% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no |
|
||||
| q235 A0 fixed-pd | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 4.2% | 0.991x | 6.1% | 5.0% | 0.4% | 4.2% | no | COUNTEREXAMPLE | tp-axis | 4.1% | -5.0% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.2% | 1.020x | 2.4% | 2.4% | 0.2% | 0.2% | no | COUNTEREXAMPLE | mns-axis | 0.2% | -0.2% | 0 | 0 | 0 | no |
|
||||
| q235 A0 fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 33.0% | 3.025x | 64.0% | 63.9% | 0.7% | 33.0% | yes | HOLDS | tp-axis | 28.5% | -64.0% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 37.2% | 3.062x | 62.9% | 62.8% | 0.3% | 37.2% | yes | HOLDS | tp-axis | 31.6% | -62.8% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 30.7% | 2.913x | 61.5% | 61.5% | 0.7% | 30.7% | yes | HOLDS | tp-axis | 26.8% | -61.5% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 34.6% | 2.955x | 60.5% | 60.4% | 0.3% | 34.6% | yes | HOLDS | tp-axis | 29.7% | -60.4% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-po | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 5.9% | 0.964x | 12.1% | 12.0% | 0.2% | 5.9% | yes | HOLDS | tp-axis | 5.7% | -12.0% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 1.7% | 0.941x | 8.3% | 8.0% | 0.3% | 1.7% | no | COUNTEREXAMPLE | tp-axis | 1.7% | -8.0% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-po | e2e_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 5.9% | 0.964x | 12.1% | 12.0% | 0.2% | 5.9% | yes | HOLDS | tp-axis | 5.7% | -12.0% | 1 | 0 | 1 | no |
|
||||
| q235 A0 fixed-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 1.7% | 0.941x | 8.3% | 8.0% | 0.3% | 1.7% | no | COUNTEREXAMPLE | tp-axis | 1.7% | -8.0% | 1 | 0 | 1 | no |
|
||||
| q235 A0 trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.493x | 101.6% | 101.6% | 2220.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A0 trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.511x | 96.5% | 96.5% | 3455.8% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A0 trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.340x | 61.5% | 43.1% | 16.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A0 trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.281x | 49.4% | 29.8% | 26.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A0 trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.6% | 1.415x | 69.3% | 69.3% | 0.6% | 0.6% | no | COUNTEREXAMPLE | mns-axis | 0.6% | -18.1% | 0 | 1 | 0 | no |
|
||||
| q235 A0 trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 6.2% | 1.473x | 71.5% | 71.5% | 6.2% | 6.2% | yes | HOLDS | mns-axis | 6.0% | -16.7% | 0 | 1 | 0 | no |
|
||||
| q235 A0 trace-po | ttft_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | 7.0% | 0.753x | 36.3% | 34.6% | 1.4% | 7.0% | yes | HOLDS | mixed | 6.8% | -36.3% | 1 | 1 | 1 | no |
|
||||
| q235 A0 trace-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | 21.2% | 0.767x | 31.8% | 31.6% | 0.3% | 21.2% | yes | HOLDS | mixed | 19.2% | -31.6% | 1 | 0 | 1 | no |
|
||||
| q235 A0 trace-po | e2e_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | 7.0% | 0.753x | 36.3% | 34.6% | 1.4% | 7.0% | yes | HOLDS | mixed | 6.8% | -36.3% | 1 | 1 | 1 | no |
|
||||
| q235 A0 trace-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | 21.2% | 0.766x | 31.8% | 31.6% | 0.3% | 21.2% | yes | HOLDS | mixed | 19.2% | -31.6% | 1 | 0 | 1 | no |
|
||||
| q235 A1 fixed-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.805x | 26.8% | 26.8% | 0.4% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes |
|
||||
| q235 A1 fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.2% | 0.837x | 30.5% | 30.5% | 0.2% | 0.2% | no | COUNTEREXAMPLE | mns-axis | 0.2% | -0.2% | 0 | 0 | 0 | no |
|
||||
| q235 A1 fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 33.0% | 2.608x | 60.1% | 60.0% | 0.7% | 33.0% | yes | HOLDS | tp-axis | 28.5% | -60.1% | 1 | 0 | 1 | no |
|
||||
| q235 A1 fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 37.2% | 2.587x | 61.2% | 61.1% | 0.3% | 37.2% | yes | HOLDS | tp-axis | 31.6% | -61.1% | 1 | 0 | 1 | no |
|
||||
| q235 A1 fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 30.7% | 2.508x | 58.3% | 58.2% | 0.7% | 30.7% | yes | HOLDS | tp-axis | 26.8% | -58.3% | 1 | 0 | 1 | no |
|
||||
| q235 A1 fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 34.6% | 2.495x | 59.3% | 59.2% | 0.3% | 34.6% | yes | HOLDS | tp-axis | 29.7% | -59.2% | 1 | 0 | 1 | no |
|
||||
| q235 A1 fixed-po | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.776x | 24.8% | 24.8% | 0.2% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes |
|
||||
| q235 A1 fixed-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.757x | 28.9% | 28.9% | 0.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes |
|
||||
| q235 A1 fixed-po | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.776x | 24.8% | 24.8% | 0.2% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes |
|
||||
| q235 A1 fixed-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.757x | 28.9% | 28.9% | 0.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes |
|
||||
| q235 A1 trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.217x | 154.4% | 154.4% | 2220.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A1 trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.219x | 151.1% | 151.1% | 3455.8% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A1 trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.338x | 46.7% | 28.2% | 16.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A1 trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.221x | 45.3% | 22.3% | 26.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no |
|
||||
| q235 A1 trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.6% | 1.430x | 48.8% | 48.8% | 0.6% | 0.6% | no | COUNTEREXAMPLE | mns-axis | 0.6% | -15.9% | 0 | 1 | 0 | no |
|
||||
| q235 A1 trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 6.2% | 1.492x | 49.7% | 49.7% | 6.2% | 6.2% | yes | HOLDS | mns-axis | 6.0% | -17.4% | 0 | 1 | 0 | no |
|
||||
| q235 A1 trace-po | ttft_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | 1.4% | 0.548x | 32.2% | 32.2% | 1.4% | 1.4% | no | COUNTEREXAMPLE | mns-axis | 1.4% | -1.4% | 0 | 1 | 0 | no |
|
||||
| q235 A1 trace-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.3% | 0.568x | 31.9% | 31.9% | 0.3% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no |
|
||||
| q235 A1 trace-po | e2e_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | 1.4% | 0.548x | 32.2% | 32.2% | 1.4% | 1.4% | no | COUNTEREXAMPLE | mns-axis | 1.4% | -1.4% | 0 | 1 | 0 | no |
|
||||
| q235 A1 trace-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.3% | 0.568x | 31.9% | 31.9% | 0.3% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no |
|
||||
|
||||
## H-SCALE verdict
|
||||
|
||||
H-SCALE is not sufficient as stated; the following rows are counterexamples to the biconditional:
|
||||
|
||||
All 40 observed counterexamples are non-failures with spread > `log1p(margin)`; there are 0 failures with spread <= `log1p(margin)`. Thus the threshold is necessary for the observed failures but not sufficient to predict them.
|
||||
|
||||
| case | objective | failure | spread real top-3 | real margin | counterexample type |
|
||||
|---|---|---|---:|---:|---|
|
||||
| q30 trace-pd | ttft_mean_ms | no | 73.5% | 52.8% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-pd | ttft_p90_ms | no | 136.8% | 67.6% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-pd | e2e_mean_ms | no | 50.1% | 6.6% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-pd | e2e_p90_ms | no | 70.3% | 0.1% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-pd high | tpot_mean_ms | no | 95.7% | 34.4% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-pd high | tpot_p90_ms | no | 95.1% | 37.2% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-pd high | e2e_mean_ms | no | 111.5% | 0.7% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po high | ttft_mean_ms | no | 0.3% | 0.1% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po high | ttft_p90_ms | no | 0.3% | 0.0% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po high | e2e_mean_ms | no | 0.3% | 0.1% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po high | e2e_p90_ms | no | 0.3% | 0.0% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-pd low | ttft_mean_ms | no | 1.5% | 1.3% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-pd low | ttft_p90_ms | no | 2.9% | 1.8% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po low | ttft_mean_ms | no | 1.1% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po low | ttft_p90_ms | no | 2.3% | 0.6% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po low | e2e_mean_ms | no | 1.1% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q30 fixed-po low | e2e_p90_ms | no | 2.4% | 0.7% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-po low | ttft_mean_ms | no | 2.6% | 1.7% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-po low | ttft_p90_ms | no | 0.2% | 0.2% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-po low | e2e_mean_ms | no | 2.6% | 1.7% | non-failure with spread > log1p(margin) |
|
||||
| q30 trace-po low | e2e_p90_ms | no | 0.2% | 0.1% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 fixed-pd | ttft_mean_ms | no | 5.0% | 0.4% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 fixed-pd | ttft_p90_ms | no | 2.4% | 0.2% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 fixed-po | ttft_p90_ms | no | 8.0% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 fixed-po | e2e_p90_ms | no | 8.0% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 trace-pd | tpot_mean_ms | no | 43.1% | 16.8% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 trace-pd | tpot_p90_ms | no | 29.8% | 26.3% | non-failure with spread > log1p(margin) |
|
||||
| q235 A0 trace-pd | e2e_mean_ms | no | 69.3% | 0.6% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 fixed-pd | ttft_mean_ms | no | 26.8% | 0.4% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 fixed-pd | ttft_p90_ms | no | 30.5% | 0.2% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 fixed-po | ttft_mean_ms | no | 24.8% | 0.2% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 fixed-po | ttft_p90_ms | no | 28.9% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 fixed-po | e2e_mean_ms | no | 24.8% | 0.2% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 fixed-po | e2e_p90_ms | no | 28.9% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 trace-pd | tpot_mean_ms | no | 28.2% | 16.8% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 trace-pd | e2e_mean_ms | no | 48.8% | 0.6% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 trace-po | ttft_mean_ms | no | 32.2% | 1.4% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 trace-po | ttft_p90_ms | no | 31.9% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 trace-po | e2e_mean_ms | no | 32.2% | 1.4% | non-failure with spread > log1p(margin) |
|
||||
| q235 A1 trace-po | e2e_p90_ms | no | 31.9% | 0.3% | non-failure with spread > log1p(margin) |
|
||||
|
||||
## S0b directional conclusions
|
||||
|
||||
Directional pair statistics remove the S0 range statistic's ambiguity by measuring every config against the real winner. Strict inversions use the frozen-value test `real_c > real_w and sim_c < sim_w`; simulator ties are not counted as inversions.
|
||||
|
||||
### (a) Which config axis decides each material failure?
|
||||
|
||||
| case | objective | real winner → sim winner | deciding axis | m_s | delta_s | -delta_s | regret |
|
||||
|---|---|---|---|---:|---:|---:|---:|
|
||||
| q30 fixed-pd high | ttft_mean_ms | tp4_mns32 → tp2_mns64 | mixed | 45.7% | -523.8% | 523.8% | 58.0% |
|
||||
| q30 fixed-pd high | ttft_p90_ms | tp4_mns64 → tp2_mns64 | tp-axis | 44.6% | -435.3% | 435.3% | 56.2% |
|
||||
| q30 fixed-pd high | e2e_p90_ms | tp1_mns32 → tp2_mns64 | mixed | 5.3% | -101.7% | 101.7% | 5.5% |
|
||||
| q30 fixed-pd low | tpot_mean_ms | tp4_mns32 → tp1_mns16 | mixed | 13.1% | -52.5% | 52.5% | 14.0% |
|
||||
| q30 fixed-pd low | tpot_p90_ms | tp4_mns8 → tp1_mns16 | mixed | 12.5% | -52.8% | 52.8% | 13.4% |
|
||||
| q30 fixed-pd low | e2e_mean_ms | tp4_mns32 → tp1_mns16 | mixed | 23.8% | -59.6% | 59.6% | 26.9% |
|
||||
| q30 fixed-pd low | e2e_p90_ms | tp4_mns8 → tp1_mns16 | mixed | 22.9% | -59.8% | 59.8% | 25.7% |
|
||||
| q235 A0 fixed-pd | tpot_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 28.5% | -64.0% | 64.0% | 33.0% |
|
||||
| q235 A0 fixed-pd | tpot_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 31.6% | -62.8% | 62.8% | 37.2% |
|
||||
| q235 A0 fixed-pd | e2e_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 26.8% | -61.5% | 61.5% | 30.7% |
|
||||
| q235 A0 fixed-pd | e2e_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 29.7% | -60.4% | 60.4% | 34.6% |
|
||||
| q235 A0 fixed-po | ttft_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns128 | tp-axis | 5.7% | -12.0% | 12.0% | 5.9% |
|
||||
| q235 A0 fixed-po | e2e_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns128 | tp-axis | 5.7% | -12.0% | 12.0% | 5.9% |
|
||||
| q235 A0 trace-pd | e2e_p90_ms | tp8_ep8_mns64 → tp8_ep8_mns128 | mns-axis | 6.0% | -16.7% | 16.7% | 6.2% |
|
||||
| q235 A0 trace-po | ttft_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns64 | mixed | 6.8% | -36.3% | 36.3% | 7.0% |
|
||||
| q235 A0 trace-po | ttft_p90_ms | tp8_ep8_mns64 → tp4_ep1_mns128 | mixed | 19.2% | -31.6% | 31.6% | 21.2% |
|
||||
| q235 A0 trace-po | e2e_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns64 | mixed | 6.8% | -36.3% | 36.3% | 7.0% |
|
||||
| q235 A0 trace-po | e2e_p90_ms | tp8_ep8_mns64 → tp4_ep1_mns128 | mixed | 19.2% | -31.6% | 31.6% | 21.2% |
|
||||
| q235 A1 fixed-pd | tpot_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 28.5% | -60.1% | 60.1% | 33.0% |
|
||||
| q235 A1 fixed-pd | tpot_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 31.6% | -61.1% | 61.1% | 37.2% |
|
||||
| q235 A1 fixed-pd | e2e_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 26.8% | -58.3% | 58.3% | 30.7% |
|
||||
| q235 A1 fixed-pd | e2e_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 29.7% | -59.2% | 59.2% | 34.6% |
|
||||
| q235 A1 trace-pd | e2e_p90_ms | tp8_ep8_mns64 → tp8_ep8_mns128 | mns-axis | 6.0% | -17.4% | 17.4% | 6.2% |
|
||||
|
||||
### (b) Where do TP-axis and MNS-axis inversions and boundary ties concentrate?
|
||||
|
||||
`n_inv_*` counts sum strict `(real winner, c)` pair inversions across all objectives in each case; a row can contain more than one inverted pair. `n_tie_*` counts cover exact simulator ties only for the simulator-selected winner versus the real winner. Ties remain decision-boundary winner-label mismatches, not strict reversals.
|
||||
|
||||
| case | n_inv_tp | n_inv_mns | n_inv_mixed | n_tie_tp | n_tie_mns | n_tie_mixed |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| q30 trace-pd | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| q30 fixed-pd high | 7 | 3 | 6 | 0 | 0 | 0 |
|
||||
| q30 fixed-po high | 0 | 0 | 0 | 0 | 4 | 0 |
|
||||
| q30 fixed-pd low | 8 | 0 | 24 | 0 | 0 | 0 |
|
||||
| q30 fixed-po low | 0 | 0 | 0 | 0 | 4 | 0 |
|
||||
| q30 trace-po low | 0 | 6 | 0 | 0 | 2 | 0 |
|
||||
| q235 A0 fixed-pd | 5 | 0 | 5 | 0 | 1 | 0 |
|
||||
| q235 A0 fixed-po | 4 | 0 | 4 | 0 | 0 | 0 |
|
||||
| q235 A0 trace-pd | 0 | 2 | 0 | 0 | 0 | 0 |
|
||||
| q235 A0 trace-po | 4 | 2 | 4 | 0 | 0 | 0 |
|
||||
| q235 A1 fixed-pd | 4 | 0 | 4 | 0 | 1 | 0 |
|
||||
| q235 A1 fixed-po | 0 | 0 | 0 | 0 | 0 | 0 |
|
||||
| q235 A1 trace-pd | 0 | 2 | 0 | 0 | 0 | 0 |
|
||||
| q235 A1 trace-po | 0 | 2 | 0 | 0 | 2 | 0 |
|
||||
| **total** | **32** | **17** | **47** | **0** | **14** | **0** |
|
||||
|
||||
- Among strict inversions, TP-axis cases are most concentrated in: q30 fixed-pd low (8), q30 fixed-pd high (7), q235 A0 fixed-pd (5), q235 A0 fixed-po (4), q235 A0 trace-po (4).
|
||||
- Among strict inversions, MNS-axis cases are most concentrated in: q30 trace-po low (6), q30 fixed-pd high (3), q235 A0 trace-pd (2), q235 A0 trace-po (2), q235 A1 trace-pd (2).
|
||||
- The eight Q235 A0/A1 Fixed-PD failures are all tp-axis. This does not conflict with the A0/MNS64 state-drift diagnosis, but does not constitute additional state-drift evidence for A1/MNS128.
|
||||
- Q30 admission evidence does not map one-to-one to an MNS-axis deciding pair: high-pressure Fixed-PD TTFT mean is **mixed** and p90 is **tp-axis**. This is an axis/mechanism mismatch, not a direct causal contradiction: changing TP changes offered arrival pressure, while the diagnosed nonlinear event is crossing the MNS admission cap. It rules out the simpler claim that an MNS-threshold mechanism must appear as an mns-axis winner pair.
|
||||
- All 14 exact simulator winner ties are mns-axis. Including these boundary errors gives MNS **17 + 14 = 31**, comparable to TP's **32** strict inversions.
|
||||
|
||||
The exact ties below show that Frontier is completely insensitive to MNS changes within the corresponding case/objective cell:
|
||||
|
||||
| case | objective | real winner ↔ simulator-selected tied winner (MNS pair) |
|
||||
|---|---|---|
|
||||
| q30 fixed-po high | ttft_mean_ms | tp4_mns32 ↔ tp4_mns16 |
|
||||
| q30 fixed-po high | ttft_p90_ms | tp4_mns32 ↔ tp4_mns16 |
|
||||
| q30 fixed-po high | e2e_mean_ms | tp4_mns32 ↔ tp4_mns16 |
|
||||
| q30 fixed-po high | e2e_p90_ms | tp4_mns32 ↔ tp4_mns16 |
|
||||
| q30 fixed-po low | ttft_mean_ms | tp4_mns8 ↔ tp4_mns16 |
|
||||
| q30 fixed-po low | ttft_p90_ms | tp4_mns8 ↔ tp4_mns16 |
|
||||
| q30 fixed-po low | e2e_mean_ms | tp4_mns8 ↔ tp4_mns16 |
|
||||
| q30 fixed-po low | e2e_p90_ms | tp4_mns8 ↔ tp4_mns16 |
|
||||
| q30 trace-po low | ttft_p90_ms | tp4_mns8 ↔ tp4_mns16 |
|
||||
| q30 trace-po low | e2e_p90_ms | tp4_mns8 ↔ tp4_mns16 |
|
||||
| q235 A0 fixed-pd | ttft_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 |
|
||||
| q235 A1 fixed-pd | ttft_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 |
|
||||
| q235 A1 trace-po | ttft_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 |
|
||||
| q235 A1 trace-po | e2e_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 |
|
||||
|
||||
### (c) How many exact-winner successes are margin-robust?
|
||||
|
||||
Among exact winner matches, **17** are margin-robust (`margin >= 1%`) and **6** are fragile (`margin < 1%`). Separately, 14 winner-label mismatches come from exact simulator ties and sit on the decision boundary rather than being strict inversions.
|
||||
|
||||
| fragile-success case | objective | real margin |
|
||||
|---|---|---:|
|
||||
| q30 trace-pd | e2e_p90_ms | 0.1% |
|
||||
| q235 A1 fixed-pd | ttft_mean_ms | 0.4% |
|
||||
| q235 A1 fixed-po | ttft_mean_ms | 0.2% |
|
||||
| q235 A1 fixed-po | ttft_p90_ms | 0.3% |
|
||||
| q235 A1 fixed-po | e2e_mean_ms | 0.2% |
|
||||
| q235 A1 fixed-po | e2e_p90_ms | 0.3% |
|
||||
|
||||
## Cross-checks
|
||||
|
||||
| check | observed | frozen expectation | result |
|
||||
|---|---:|---:|---|
|
||||
| q30 high-pressure fixed-pd TTFT mean regret | 58.0% | 58.0% | PASS |
|
||||
| q235 A0 fixed-pd TPOT mean regret | 33.0% | 33.0% | PASS |
|
||||
| q235 A1 fixed-pd TPOT mean regret | 33.0% | 33.0% | PASS |
|
||||
| q30 trace-pd all-objective regret | 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0% | 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0% | PASS |
|
||||
| q235 Trace-PO TTFT p90 A0 to A1 regret | 21.2%, 0.3% | 21.2%, 0.3% | PASS |
|
||||
| q235 Fixed-PD TPOT/E2E regrets unchanged from A0 to A1 | 33.0%, 37.2%, 30.7%, 34.6%, 33.0%, 37.2%, 30.7%, 34.6% | 33.0%, 37.2%, 30.7%, 34.6%, 33.0%, 37.2%, 30.7%, 34.6% | PASS |
|
||||
|
||||
## Data gaps
|
||||
|
||||
No per-config sim/real pairing gaps were found in the requested surfaces.
|
||||
|
||||
## 异常
|
||||
|
||||
- 无。重算 winner/regret 与 frozen JSON 一致，硬性交叉核对均通过。
|
||||
|
||||
## Figure
|
||||
|
||||
`margin-vs-residual.png` uses x = `100 × log1p(real best-vs-second relative margin)` and y = `100 ×` real-top-3 log-ratio spread. Both axes are logarithmic because the real margins span six orders of magnitude. Green means exact winner match; red means winner mismatch. The diagonal is the H-SCALE threshold, while failure itself is defined by regret, not by exact winner match.
|
||||
|
||||
`decision-pair-axis.png` is a mechanism census, not a hypothesis test: for a strict winner reversal, `-delta_s > m_s` is algebraically equivalent to the mismatch. The frozen surfaces also contain deterministic winner-label mismatches caused by exact simulator ties; those points lie on `-delta_s = m_s` and are not counted as strict inversions. Color identifies the winner-deciding config axis and marker area is proportional to regret.
|
||||
|
||||
## A0 vs A1 对照
|
||||
|
||||
A1 repairs the Q235 Trace-PO p90 ranking error but leaves the four large Fixed-PD TPOT/E2E regrets unchanged.
|
||||
|
||||
| case / objective | A0 regret | A1 regret |
|
||||
|---|---:|---:|
|
||||
| trace-po / ttft_p90_ms | 21.2% | 0.3% |
|
||||
| fixed-pd / tpot_mean_ms | 33.0% | 33.0% |
|
||||
| fixed-pd / tpot_p90_ms | 37.2% | 37.2% |
|
||||
| fixed-pd / e2e_mean_ms | 30.7% | 30.7% |
|
||||
| fixed-pd / e2e_p90_ms | 34.6% | 34.6% |
|
||||
|
||||
Thus Trace-PO TTFT p90 changes **21.2% → 0.3%**, whereas Fixed-PD remains **33.0% / 37.2% / 30.7% / 34.6%** in both A0 and A1.
|
||||
|
||||
## 已知分量归因摘要
|
||||
|
||||
### Q30 high-pressure Fixed-PD TTFT: first-scheduling wait vs execution-side TTFT
|
||||
|
||||
`queue-free TTFT` is the execution-side TTFT after removing first-scheduling wait; in the frozen diagnostic, simulated TTFT is their sum (up to floating-point rounding).
|
||||
|
||||
| config | sim TPOT / real | sim slots / MNS | real slots upper / MNS | first-scheduling wait | queue-free TTFT | sim TTFT | real TTFT | real waiting max |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
||||
| tp2_mns64 | 8.09x | 58.0/64 | 7.5/64 | 38.2 ms | 191.0 ms | 229.3 ms | 194.6 ms | 0 |
|
||||
| tp4_mns16 | 3.83x | 54.5/16 | 14.6/16 | 64906.3 ms | 134.7 ms | 65041.0 ms | 124.5 ms | 0 |
|
||||
| tp4_mns32 | 4.67x | 65.9/32 | 14.5/32 | 27180.7 ms | 135.6 ms | 27316.3 ms | 123.2 ms | 0 |
|
||||
| tp4_mns64 | 5.61x | 79.8/64 | 14.7/64 | 6367.3 ms | 136.7 ms | 6504.0 ms | 123.2 ms | 0 |
|
||||
|
||||
For TP4/MNS32 minus TP2/MNS64, the observed simulator contrast is **+27087.0 ms**; removing first-scheduling wait changes it to **-55.4 ms**, aligned with the real contrast **-71.4 ms**. This localizes the Q30 high-pressure TTFT reversal to admission queueing, while the upstream service-time overprediction remains unattributed in the frozen data.
|
||||
|
||||
### Q235 Fixed-PD: own-composition vs exact-state contrast
|
||||
|
||||
Positive TP8−TP4 means TP8 is slower and therefore agrees with the real TP4 winner.
|
||||
|
||||
| config | sim own decode batch mean | real exact decode batch mean | real token-weighted iteration | exact-state matched sim | coverage |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| tp4_ep1_mns64 | 13.537 | 3.890 | 18.4710 ms | 34.2575 ms | 100.0% |
|
||||
| tp8_ep8_mns64 | 18.938 | 10.921 | 27.5887 ms | 45.1603 ms | 100.0% |
|
||||
|
||||
| component | own-composition TP8−TP4 | exact-state TP8−TP4 |
|
||||
|---|---:|---:|
|
||||
| total | -20.0663 ms | +10.9028 ms |
|
||||
| moe_compute | -32.4544 ms | -1.9774 ms |
|
||||
| attention | -7.4619 ms | -2.1304 ms |
|
||||
| tp_dp_communication | -6.5711 ms | -6.4718 ms |
|
||||
| ep_communication | +6.2354 ms | +6.5355 ms |
|
||||
| moe_routing | -0.2862 ms | +0.1384 ms |
|
||||
| dense_mlp_compute | -0.1231 ms | -0.0308 ms |
|
||||
| pipeline_communication | +0.0000 ms | +0.0000 ms |
|
||||
| runtime_overhead | +0.0000 ms | +0.0000 ms |
|
||||
|
||||
The total contrast flips from **-20.0663 ms** on Frontier's own composition to **+10.9028 ms** at exact real state, a **+30.9690 ms** state-composition shift. The exact real iteration contrast is **+9.1177 ms** and the frozen real TPOT contrast is **+6.9500 ms**. This supports closed-loop state/composition drift for Q235 Fixed-PD; it does not by itself attribute every remaining conditional execution residual.
|
||||
BIN
runs/frontier-split-rootcause-v0/results/margin-vs-residual.png
Normal file
BIN
runs/frontier-split-rootcause-v0/results/margin-vs-residual.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 218 KiB |
Reference in New Issue
Block a user