diff --git a/.research/experiments/frontier-split-rootcause-s0-20260720.md b/.research/experiments/frontier-split-rootcause-s0-20260720.md new file mode 100644 index 0000000..10db9e3 --- /dev/null +++ b/.research/experiments/frontier-split-rootcause-s0-20260720.md @@ -0,0 +1,60 @@ +# 实验 S0:good/bad case 分裂的统一分解(margin vs differential residual) + +> **状态:** 已完成(2026-07-20;含 S0b 方向化修正与一轮 strict review 修复) +> +> 用户指令:核心要务是分析为什么部分 case 下 Frontier work、部分不 work 的 system 根因;本 card 是该诊断 campaign 的第一个 slice,仅使用 frozen artifacts,零 GPU 成本。人工 review 由用户的直接指令("只有做好这个分析我们才能推进下一步")满足。 + +## Claim 与决策 + +- **Parent claim:** ongoing.md H2——误差机制是 action-conditioned residual;本实验把它细化为"分裂从哪来"。 +- **现象(已冻结):** 同一 best-effort Frontier 栈上,Q30/Q235 的 Trace-PD 与多数 PO 面 selection 近优,而 Fixed-PD 面 14–58% regret;失败 objective 随负载档切换(Q30 低压 TPOT/E2E 全反、高压 TTFT 56–58%);A1 collective profile 修复了 Q235 Trace-PO p90 21.2%→0.3% 但对 Fixed-PD 33% 完全无效(本 card frozen-inputs/q235-ablation-a1)。 +- **Competing hypotheses:** + - **H-SCALE:** 分裂完全由「config-differential residual vs 真机 decision margin」的关系解释:good case 的 sim/real 比值跨 config 近似均匀(乘性偏移,argmin 不变),bad case 的比值跨 config 分散且超过 margin。workload shape 本身不需要出现在解释里。 + - **H-THRESH:** 绝对 service-time 高估近似均匀,但与离散机制(MNS admission cap、MoE token-bucket、graph bucket)交互后被转换为 config-differential 误差;fixed uniform workload 把所有请求同步到同一 state 轨迹,使阈值交叉对整个 cell 相干生效;trace 的长度/到达异质性把阈值效应摊平。 + - **H-STATE:** 失败由 simulator 闭环 batch state 分布漂移主导(Q235:sim decode batch 13.5 vs real 3.9 + B4→B5 profile cliff 正反馈);即使打破 workload 同步性,闭环漂移仍可翻转排序。 + - 三者关系:H-SCALE 是现象层(必要条件),H-THRESH/H-STATE 是 differential residual 的两种产生机制,可共存但可判别(见事前预测)。 +- **事前预测:** + - H-SCALE 成立 ⟺ 对每个 case×objective,failure 恰好发生在「top 邻域 log-ratio spread > log1p(真机相对 margin)」处(两侧同为 log-space 尺度),无反例。 + - H-THRESH 独有:bad case 的 differential 误差集中于阈值语义分量(first-scheduling wait、bucket 跳变段),且 sim-only 反事实(去阈值/加 jitter)恢复排序——Q30 高压 TTFT 的 admission 反事实已支持一例。 + - H-STATE 独有:差异化误差在去掉阈值分量后仍在 execution 项内(Q235 Fixed-PD 的 own-composition −20.07 vs exact-state +10.90 已支持一例)。 +- **判定规则:** S0 只裁决 H-SCALE 与「分量定位」(queue vs execution);H-THRESH/H-STATE 的干预判别属 S1+(sim-only 反事实)与 GPU 实验(需另行 review)。若 H-SCALE 出现反例(good case 有 spread>margin 仍选对,或 bad case spread5%;H-SCALE 检验=failure ⟺ 邻域 spread>log1p(margin)(review 修正:初版直接以 ln 差比较普通 relative margin,尺度不一致;修正后 70 行 verdict 不变)。 +- **分量定位:** q30-admission-diag 提供 TTFT=first-scheduling wait+prefill execution 分解;q235-state-diag 提供 own-composition vs exact-state contrast;把这些已知分量证据合并进统一表。 +- **交叉核对:** 重算的 regret 必须与各 frozen comparison.md 表一致(抽查 58.0%、33.0%、0.0%);A0 vs A1 的 Q235 对比必须复现 trace-po p90 21.2%→0.3%、fixed-pd 四项不变。 + +## 预期产物与 review + +- runs/frontier-split-rootcause-v0/analyze_split_decomposition.py(只读 frozen-inputs,确定性输出) +- runs/frontier-split-rootcause-v0/results/decomposition.{json,md}:统一表,每行 case×objective,列出 winner、regret、scale、spread(全/邻域)、margin、H-SCALE verdict、已知分量归因 +- runs/frontier-split-rootcause-v0/results/margin-vs-residual.png:x=真机 margin,y=邻域 differential residual,点色=selection 对错;H-SCALE 成立则对错点被对角线分离 +- 人工验收:编排者亲自重跑脚本、抽查交叉核对数字、亲自查看渲染图 + +## 复现信息 + +- **Code:** AITuner branch feature/sim,自 HEAD 18c0b25 起;worker/reviewer job-id 见下方「Review 与 provenance 补记」;脚本与产物随本 card 同一 commit 入库(含 frozen-inputs 本地拷贝)。 +- **Environment:** 本地 workstation,CPU-only,python3+matplotlib;不访问远端。 +- **已知 deviation:** frozen-inputs 是 cpfs 原件的本地拷贝(scp,2026-07-20);q30-expansion-lo 的低压 Fixed-PD 面已被高压面取代为 primary,本分析将两档并列为独立观测,不混合。 + +## 结果 + +- **观察事实:** + - 70 行(14 个 case surface)全部算出,无数据缺口;四组硬性交叉核对通过;连续运行产物 SHA 一致。 + - **H-SCALE 判为必要非充分**:23 个 material failure(regret>5%)全部满足「top-3 邻域 log spread > log1p(margin)」,无一例失败发生在 residual 小于 margin 处;但另有 40/70 行同样满足该条件却均非 material failure(其中仅 16 行 exact winner match,其余 24 行是小 regret 的 winner 错位——14 个 MNS 精确 tie 与 10 个 strict reversal)——无方向 spread 不携带决策信息。 + - **S0b 方向化后的机制普查**:23 个 material failure 的 winner-deciding pair 分布为 tp-axis 11、mixed 10、mns-axis 2(Q235 A0/A1 Fixed-PD 的 8 个 TPOT/E2E failure 全为 tp-axis;Q30 Fixed-PD 高低压为 tp/mixed);仅有的 2 个 mns-axis failure 是 Q235 A0/A1 Trace-PD E2E p90(regret 6.2%,勉强越过 5% 门槛)。trace 面严格反序中 tp-axis 为 0(q30 trace-pd 三轴全 0)。 + - **A1 对照的轴分解**:serving-matched collective profile 把 Q235 两个 PO 面的 tp-axis 反序从 4/4 清零(trace-po p90 regret 21.2%→0.3%),但 Fixed-PD 仅 5→4、四项 regret 一位小数不动——prefill 路径的 TP-differential 误差源=collective profile(可修),decode 耦合的 TP-differential 误差另有来源。 + - **MNS 不敏感缺陷**:14 个 winner-label mismatch 是 simulator 逐位相等的 tie,全部 mns-axis(如 q30 fixed-po 的 MNS16↔32、q235 fixed-pd 的 MNS64↔128);tie 计入后 MNS 边界误差 31 与 TP 严格反序 32 相当,但 MNS 侧 regret 小。 + - **成功的鲁棒性**:23 个 exact-winner success 中 17 个 margin-robust(margin≥1%),6 个 fragile(含 q30 trace-pd E2E p90 的 0.1% margin 与 q235 A1 fixed-po 四项)。 + - **面级 scale 对照**:prefill-only 面 geomean scale 0.96–1.37×(绝对预测基本准确),含 decode 的面 4.3–130×——绝对误差灾难集中于 decode。 +- **异常:** 无数据异常。strict review(FAIL:3 Major/1 Minor)指出 H-SCALE 尺度混用(log spread vs relative margin)、tie 轴普查缺失、card 状态过期、Q235 一致性表述过强;全部修复,修复后 70 行 verdict 逐行不变。 +- **含义:** 分裂的现象层解释是「margin 保护 + config-differential 误差」。机制层上,21/23 个 material failure 由 TP/mixed pair 决定,且所有大 regret(≥13%)failure 都发生在含 decode 的面上:其中 Q235 Fixed-PD 有 state-drift 直接证据、Q30 高压 TTFT 有 admission 反事实证据,而 **Q30 低压 TPOT/E2E 反转的机制尚未诊断**(S1 目标)。「decode 耦合的 TP-differential 误差是主要载体」是当前最强归纳,不是对全部 failure 的已证机制归因;2 个 mns-axis 边缘 failure(6.2%)在该归纳之外。trace 面成功伴随「TP 反序为零 + TP margin 宽」,但「误差小」与「margin 宽」谁是主因仍未判——这正是 H-THRESH vs H-STATE 的判别缺口。轴标签与机制不一一对应(Q30 admission 是 MNS 阈值机制但 deciding pair 为 tp/mixed,因 TP 改变到达压力)。 +- **Claim update:** H2(action-conditioned residual)supported 且被细化:residual 的决策相关分量集中在 TP 轴、由 decode 状态耦合产生;H-SCALE 降级为必要条件;H-THRESH/H-STATE 保持 competing,待 S1 判别。 +- **下一步:** S1(sim-only 反事实:Q30 低压 Fixed-PD TPOT 反转的分量定位——这是唯一无机制解释的 material failure;fixed workload jitter 判别 H-THRESH vs H-STATE);GPU 判别实验(加压 Trace-PD、jittered Fixed-PD 真机面,dash1–4)另行出 card 供 review。 + +## Review 与 provenance 补记 + +- S0 worker:codex `task-mrsn1s9c-z6ha0w`;S0b:`task-mrsnjzn6-k18x4n`(resume);strict reviewer(fresh 只读):`task-mrsocmlb-386btc`(verdict FAIL);修复轮:`task-mrsop06n-pwxo0b`(fresh writable)。编排者独立验收:脚本重跑、SHA 比对、两图目视检查。 +- 产物 SHA(修复后):decomposition.json `81ea56b2…`、decomposition.md `4d19af54…`、margin-vs-residual.png `df2110c4…`、decision-pair-axis.png `2e787301…`。 diff --git a/.research/ongoing.md b/.research/ongoing.md index c97e5ce..dcc6521 100644 --- a/.research/ongoing.md +++ b/.research/ongoing.md @@ -3,14 +3,16 @@ > 2026-07-17|写给未参与项目的读者,可直接作为 presentation 讲稿。历史过程与复现信息见 `../runs/*/` 各 experiment card、`../docs/` 各 campaign 文档。 > > **2026-07-19 update:** Qwen235 Fixed-PD 的错误排序在 exact real state composition 下已经翻正,主因是 simulator closed-loop batch state,而不是 collective。Qwen30 Fixed-PD 的 56--58% TTFT regret 也已定位:Frontier 将 decode service time 高估 4--8×,使 TP4 的 modeled concurrency 越过 MNS admission cap并产生虚假排队;去掉该等待后 Frontier 与真机都判定 TP4 topology 更快。详见 [`experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md`](experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md)。 +> +> **2026-07-20 update:** 对全部 14 个 frozen case surface(70 个 case×objective)做了统一的 margin-vs-residual 分解与方向化机制普查([`experiments/frontier-split-rootcause-s0-20260720.md`](experiments/frontier-split-rootcause-s0-20260720.md))。三个要点:(1) 「residual 超过 margin」是失败的必要条件但远非充分——good/bad 分裂不能用无方向误差量解释;(2) 23 个 material failure 的 winner-deciding pair 中 21 个落在 TP 轴或 mixed(其余 2 个是 6.2% regret 的边缘 mns-axis case),trace 面的 TP 反序为零;A1 measured collective 把 Qwen235 两个 prefill-only 面的 TP 反序清零(trace-PO p90 regret 21.2%→0.3%)却对 Fixed-PD 完全无效——prefill 路径的 TP 差异化误差源是 collective profile(可修),decode 耦合的 TP 差异化误差是当前所有 material failure 的载体;(3) 「Fixed-PD 失败因为高压」被否证:失败 Fixed-PD 的真机 in-flight(14.05)低于全对的 Trace-PD(38.69),且低压 Fixed-PD 同样失败、失败 objective 随负载切换。另有次要缺陷:14 个 winner 错位来自 simulator 对 MNS 逐位不敏感的精确 tie。 ## 一眼看懂 - **Topic / problem:** LLM serving 的自动、低成本配置调优(AITuner)。当前主线问题:用 simulator 给部署配置(并行度、批量上限等)排序,什么时候可信?需要补多少真机证据?算上这些成本还划算吗? - **Central claim:** simulator 要能帮助配置调优,必须先满足 scheduler transition 的 liveness/coverage,再满足「配置相关残差小于真机 decision margin」;前者决定 capacity 是否有定义,后者决定排序是否正确。(ID: C0) - **当前结论:** 早先 35 个 trace stall 不是 Frontier scheduler liveness failure:adapter 为不满 16-token 的 prefix block 错误生成了 cache identity,Frontier 又没有 fail-fast。修正为完整 block、使用真实 graph buckets/KV blocks 和 `piecewise`/`KERNEL_ONLY` profile 后,Qwen30 Trace-PD 的全部 12 个 cell 完成 129/129 request,Frontier 对 TTFT/TPOT/E2E 的 6 个 argmin 均与三次 fresh-server 真机一致;但绝对 latency 仍高估 4--511×。这只证明一个 MoE Trace-PD surface 的 selection fidelity,不能外推到 prefill-only、fixed workload 或 235B。 -- **最大 uncertainty / risk:** 这个正确选择是否依赖 Trace-PD 的 queue/margin 偶然性;Fixed-PD、Trace-P、Fixed-P 与大型 FP8 MoE 是否仍保留正确 config ranking。Q235 还缺少同栈 profile/runtime contract,不能复用旧 vLLM 0.10.2 数据。 -- **下一项 critical action:** 完成 Qwen30 其余三项 fixed/trace × PD/P surface。Fixed case 已先启动无请求 vLLM runtime-state preflight;Trace-P 的 graph-aligned Frontier 12-cell CPU surface 正在运行。Q235 先做 vLLM 0.20 TP4/TP8 compatibility gate,随后才允许新 profile。 +- **最大 uncertainty / risk:** trace 面成功的原因未判——是 decode 耦合的 TP 差异化误差在 trace 状态分布下真的变小,还是只是被宽 TP margin 掩盖。这决定「sim 剪枝可信域」的边界怎么画,也决定加压 trace 是否会失败。 +- **下一项 critical action:** 判别两个失败机制假设(离散阈值转换 vs 闭环 state 漂移):先做零 GPU 的 sim-only jitter 反事实与 Q30 低压 Fixed-PD TPOT 反转分量定位,再按结果决定 dash1–4 上的加压 Trace-PD / jittered Fixed-PD 真机判别面。 - **停止条件:** T1 出 verdict 且成本账本建立后:pass 且摊销论证成立 → 转向「sim 剪枝 + 真机终选」的 hybrid 机制设计;fail → 转入失败机制归因;两条路都无 insight 增量 → 收敛写作。 ## 核心概念 @@ -34,9 +36,10 @@ - **Supporting:** 235B prefill-only regret=0;235B fixed-shape mixed 的 top set 全中;30B 加 per-TP 校准后 regret 0.76%(但这是外部端到端 scale 给出的上界,不是原生 profile 保真度)。 - **Counterevidence:** 修正 prefix trace contract 后的 TP2/MNS16 `none`-graph run 完成但 p50 TPOT 约 96 ms,真机为约 14 ms;然而该比较尚未对齐 real vLLM 的 `FULL_AND_PIECEWISE` graph path。 - **下一项 discriminative experiment:** 补齐 `KERNEL_ONLY` graph family并以 `piecewise` 重跑相同 trace;若 full surface 仍错,graph omission 不再是可用解释。 - - **Hypothesis(机制,active):** 误差机制是 action-conditioned residual——执行状态的转移(并行拓扑、kernel family、graph mode、batch 组成)使按算子 profile 的组合预测跨配置不可复合;残差大于 margin 时排序失败。(ID: H2;supported) + - **Hypothesis(机制,active):** 误差机制是 action-conditioned residual——执行状态的转移(并行拓扑、kernel family、graph mode、batch 组成)使按算子 profile 的组合预测跨配置不可复合;残差大于 margin 时排序失败。(ID: H2;supported,已细化) - **Supporting:** 三个 TP 档的端到端校准系数为 0.72/0.47/0.35,残差确实随配置剧烈变化;235B 的批量上限交互预测错误但被 2× margin 容忍;30B prefill-only 在低负载近似对齐、饱和后按 TP 反向放大,最终 τ-b=−1。 - - **下一步:** 按 collective 通信 → batch 组成 → 调度器逐步轨迹的单变量顺序定位组合性失效点(对应 experiment card 中 A1/A2/A3 消融阶梯)。 + - **细化(2026-07-20 统一普查):** 决策相关的残差分量集中在 TP 轴且由 decode 状态耦合产生——prefill-only 面的绝对 scale 仅 0.96–1.37× 且 measured collective 即可清除其 TP 反序,而含 decode 的面 scale 4.3–130×、全部 material failure 都由 TP/mixed pair 决定。「residual>margin」只是必要条件;失败还需要残差对准 winner-deciding pair。 + - **下一步:** 判别两个 competing 机制假设——离散阈值转换(admission cap/profile cliff 把均匀高估转成差异化误差;打破 fixed workload 的同步性应恢复排序)vs 闭环 state 漂移(打破同步性也救不了)。最便宜路径:sim-only jitter 反事实 + Q30 低压 Fixed-PD TPOT 反转的分量定位(唯一无机制解释的 material failure)。 - **Subclaim:** 成本论证只有在摊销前提下成立。(ID: C3) - **Hypothesis(active):** 每个 model×硬件×runtime 的一次性对齐成本,摊销到大配置面、频繁重调(引擎版本 churn 的频率证据见 claim map)或禁止在线实验的场景后,低于重复真机调优。(ID: H3;untested——分母已实测,分子未入账) - **下一步:** 建 cost ledger(见「下一步」)。 @@ -52,7 +55,7 @@ ## Key evidence(最多 3 条) - **E1(否证「prefill-only 是充分 easy condition」,支持 H2):** 30B BF16、去掉 decode/prefix/混合 batch 后,真机最优是 TP4(8 vs 7 req/s/GPU),simulator 却把 TP4 排最差(6 vs 8):top set 无交集,regret 12.5%,τ-b=−1。产物:`../runs/frontier-phase-factorial-v0/results/final/`(dash0,12.07 H20-GPUh)。 -- **E2(margin 可掩盖残差,支持 C0/H2):** 235B FP8 的 prefill-only 与 fixed-shape mixed 都选对 top set,但 simulator 漏掉并发上限(max-num-seqs)×批 token 上限(max-num-batched-tokens)的交互棋盘格,并把 10/34 个真机 anchor 误判为不可行;真机头部 margin 有 2×,残差被容忍。产物:`../runs/frontier-multicase-sufficiency-v0/best_effort/fixed_cohort_evidence/`、`../runs/frontier-multicase-sufficiency-v1/results/t0-final/`。 +- **E2(统一机制普查:material failure 全部由 decode 耦合的 TP 差异化误差决定,支持 H2 细化):** 对 14 个 frozen surface、70 个 case×objective 的方向化分解显示:23 个 material failure 中 21 个由 TP/mixed pair 决定(仅 2 个 6.2% 边缘 mns-axis case)、trace 面 TP 反序为零;measured collective(A1)把 Qwen235 两个 prefill-only 面的 TP 反序清零(trace-PO p90 regret 21.2%→0.3%)但对 Fixed-PD 的 33% 无效;「residual>margin」仅为失败的必要条件。产物:[`../runs/frontier-split-rootcause-v0/results/`](../runs/frontier-split-rootcause-v0/results/decomposition.md)(实验 card:[`experiments/frontier-split-rootcause-s0-20260720.md`](experiments/frontier-split-rootcause-s0-20260720.md))。 - **E3(selection 与 calibration 分开):** 用完整 block projection、`piecewise` 和 graph-compatible KERNEL_ONLY profile 后,Qwen30 Trace-PD 的 12/12 sim cells 完成,6 个 mean/p90 latency objective 的 argmin 都与真机一致;但 sim/real latency ratio 仍为 4--511×。产物:[`../runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md`](../runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md)。 ## 下一步(最多 3 项) diff --git a/runs/frontier-split-rootcause-v0/analyze_split_decomposition.py b/runs/frontier-split-rootcause-v0/analyze_split_decomposition.py new file mode 100644 index 0000000..4e3d178 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/analyze_split_decomposition.py @@ -0,0 +1,1444 @@ +#!/usr/bin/env python3 +"""Analyze frozen good/bad Frontier selection cases by margin and residual.""" + +from __future__ import annotations + +import argparse +import json +import math +import os +import re +from dataclasses import dataclass +from pathlib import Path +from statistics import fmean +from typing import Any + + +HERE = Path(__file__).resolve().parent +DEFAULT_INPUT_ROOT = HERE / "frozen-inputs" +DEFAULT_RESULTS_ROOT = HERE / "results" + +OBJECTIVE_ORDER = { + "ttft_mean_ms": 0, + "ttft_p90_ms": 1, + "tpot_mean_ms": 2, + "tpot_p90_ms": 3, + "e2e_mean_ms": 4, + "e2e_p90_ms": 5, +} + +CONFIG_PATTERN = re.compile( + r"^tp(?P[0-9]+)(?:_ep(?P[0-9]+))?_mns(?P[0-9]+)$" +) +INVERSION_AXES = ("tp-axis", "mns-axis", "mixed") + + +@dataclass(frozen=True) +class Surface: + case: str + objective: str + values: dict[str, tuple[float, float]] + expected_configs: tuple[str, ...] + expected_sim_winner: str + expected_real_winner: str + expected_regret: float | None + sources: tuple[str, ...] + + +def load_json(path: Path) -> dict[str, Any]: + with path.open() as handle: + value = json.load(handle) + if not isinstance(value, dict): + raise ValueError(f"expected JSON object: {path}") + return value + + +def atomic_write(path: Path, text: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_suffix(path.suffix + ".tmp") + temporary.write_text(text) + os.replace(temporary, path) + + +def positive_float(value: Any, context: str) -> float: + if not isinstance(value, (int, float)) or isinstance(value, bool): + raise ValueError(f"non-numeric value for {context}: {value!r}") + result = float(value) + if not math.isfinite(result) or result <= 0: + raise ValueError(f"non-positive or non-finite value for {context}: {value!r}") + return result + + +def normalized_objective(metric: str, statistic: str) -> str: + return f"{metric.removesuffix('_ms')}_{statistic}_ms" + + +def objective_key(objective: str) -> tuple[int, str]: + return OBJECTIVE_ORDER.get(objective, len(OBJECTIVE_ORDER)), objective + + +def config_dimensions(config: str) -> tuple[tuple[int, int | None], int]: + match = CONFIG_PATTERN.fullmatch(config) + if match is None: + raise ValueError(f"unrecognized config name: {config}") + ep = int(match.group("ep")) if match.group("ep") is not None else None + return (int(match.group("tp")), ep), int(match.group("mns")) + + +def pair_axis(left: str, right: str) -> str: + if left == right: + return "none" + left_plan, left_mns = config_dimensions(left) + right_plan, right_mns = config_dimensions(right) + plan_differs = left_plan != right_plan + mns_differs = left_mns != right_mns + if plan_differs and not mns_differs: + return "tp-axis" + if mns_differs and not plan_differs: + return "mns-axis" + if plan_differs and mns_differs: + return "mixed" + raise ValueError(f"distinct configs have identical dimensions: {left}, {right}") + + +def load_q30_trace_pd(input_root: Path) -> list[Surface]: + relative = Path("q30-trace-pd/comparison.json") + data = load_json(input_root / relative) + if data.get("schema") != "frontier-qwen30-piecewise-graph-comparison-v1": + raise ValueError(f"unexpected schema: {relative}") + configs = tuple(sorted(data["cells"])) + surfaces = [] + for source_objective, selection in sorted( + data["selection"].items(), + key=lambda item: objective_key( + normalized_objective(*item[0].split(":")) + ), + ): + metric, statistic = source_objective.split(":") + objective = normalized_objective(metric, statistic) + values = {} + for config in configs: + cell = data["cells"][config]["metrics"][metric] + values[config] = ( + positive_float(cell[f"sim_{statistic}_ms"], f"{relative}:{config}:sim"), + positive_float(cell[f"real_{statistic}_ms"], f"{relative}:{config}:real"), + ) + surfaces.append( + Surface( + case="q30 trace-pd", + objective=objective, + values=values, + expected_configs=configs, + expected_sim_winner=selection["sim_winner"], + expected_real_winner=selection["real_winner"], + expected_regret=None, + sources=(str(relative),), + ) + ) + return surfaces + + +def load_q30_case( + input_root: Path, + case: str, + comparison_relative: Path, + audit_relative: Path, +) -> list[Surface]: + comparison = load_json(input_root / comparison_relative) + audit = load_json(input_root / audit_relative) + if comparison.get("schema") != "qwen30-latency-case-frontier-real-comparison-v1": + raise ValueError(f"unexpected schema: {comparison_relative}") + if audit.get("schema") != "qwen30-latency-case-real-audit-v1": + raise ValueError(f"unexpected schema: {audit_relative}") + configs = tuple(sorted(set(comparison["cells"]) | set(audit["configs"]))) + surfaces = [] + for source_objective, selection in sorted( + comparison["selection"].items(), + key=lambda item: objective_key( + normalized_objective(*item[0].split(":")) + ), + ): + metric, statistic = source_objective.split(":") + objective = normalized_objective(metric, statistic) + values = {} + for config in configs: + if config not in comparison["cells"] or config not in audit["configs"]: + continue + sim = comparison["cells"][config]["metrics"].get(metric, {}).get(statistic) + real = ( + audit["configs"][config]["metrics"] + .get(metric, {}) + .get(f"pooled_{statistic}_ms") + ) + if sim is None or real is None: + continue + values[config] = ( + positive_float(sim, f"{comparison_relative}:{config}:sim"), + positive_float(real, f"{audit_relative}:{config}:real"), + ) + surfaces.append( + Surface( + case=case, + objective=objective, + values=values, + expected_configs=configs, + expected_sim_winner=selection["sim_winner"], + expected_real_winner=selection["real_winner"], + expected_regret=selection.get("selected_config_real_regret"), + sources=(str(comparison_relative), str(audit_relative)), + ) + ) + return surfaces + + +def load_q235_campaign( + input_root: Path, campaign: str, relative: Path +) -> list[Surface]: + data = load_json(input_root / relative) + if data.get("schema") != "qwen235-v020-simulator-real-comparison-v1": + raise ValueError(f"unexpected schema: {relative}") + surfaces = [] + for case_name in ("fixed-pd", "fixed-po", "trace-pd", "trace-po"): + case = data["cases"][case_name] + configs = tuple(sorted(set(case["sim"]) | set(case["real"]))) + for objective, selection in sorted( + case["comparison"].items(), key=lambda item: objective_key(item[0]) + ): + values = {} + for config in configs: + if config not in case["sim"] or config not in case["real"]: + continue + sim = case["sim"][config].get(objective) + real = case["real"][config].get(objective) + if sim is None or real is None: + continue + values[config] = ( + positive_float(sim, f"{relative}:{case_name}:{config}:sim"), + positive_float(real, f"{relative}:{case_name}:{config}:real"), + ) + surfaces.append( + Surface( + case=f"q235 {campaign} {case_name}", + objective=objective, + values=values, + expected_configs=configs, + expected_sim_winner=selection["sim_winner"], + expected_real_winner=selection["real_winner"], + expected_regret=selection.get("selected_real_regret"), + sources=(str(relative),), + ) + ) + return surfaces + + +def load_surfaces(input_root: Path) -> list[Surface]: + surfaces = load_q30_trace_pd(input_root) + q30_specs = ( + ( + "q30 fixed-pd high", + Path("q30-fixed-hi/fixed-pd/comparison.json"), + Path("q30-fixed-hi/fixed-pd/real-audit.json"), + ), + ( + "q30 fixed-po high", + Path("q30-fixed-hi/fixed-po/comparison.json"), + Path("q30-fixed-hi/fixed-po/real-audit.json"), + ), + ( + "q30 fixed-pd low", + Path("q30-expansion-lo/fixed-pd-comparison.json"), + Path("q30-expansion-lo/fixed-pd-real-audit.json"), + ), + ( + "q30 fixed-po low", + Path("q30-expansion-lo/fixed-po-comparison.json"), + Path("q30-expansion-lo/fixed-po-real-audit.json"), + ), + ( + "q30 trace-po low", + Path("q30-expansion-lo/trace-po-comparison.json"), + Path("q30-expansion-lo/trace-po-real-audit.json"), + ), + ) + for case, comparison, audit in q30_specs: + surfaces.extend(load_q30_case(input_root, case, comparison, audit)) + surfaces.extend( + load_q235_campaign( + input_root, "A0", Path("q235-fourcase-a0/comparison.json") + ) + ) + surfaces.extend( + load_q235_campaign( + input_root, "A1", Path("q235-ablation-a1/comparison.json") + ) + ) + return surfaces + + +def analyze_surface(surface: Surface) -> tuple[dict[str, Any], list[str]]: + anomalies = [] + paired_configs = tuple(sorted(surface.values)) + missing_configs = sorted(set(surface.expected_configs) - set(paired_configs)) + if missing_configs: + anomalies.append( + f"{surface.case} / {surface.objective}: missing paired sim+real data for " + + ", ".join(missing_configs) + ) + if len(paired_configs) < 2: + return ( + { + "case": surface.case, + "objective": surface.objective, + "sim_winner": None, + "real_winner": None, + "regret": None, + "config_uniform_scale": None, + "log_ratio_spread_all": None, + "log_ratio_spread_real_top3": None, + "real_relative_margin_best_vs_second": None, + "sim_winner_real_relative_loss": None, + "failure": None, + "h_scale_condition_spread_gt_log_margin": None, + "h_scale_verdict": "N/A", + "winner_match": None, + "winner_deciding_axis": None, + "winner_pair_m_s_percent": None, + "winner_pair_delta_s_percent": None, + "winner_pair_strict_reversal": None, + "winner_pair_relation": "N/A", + "n_inv_tp": None, + "n_inv_mns": None, + "n_inv_mixed": None, + "n_tie_tp": None, + "n_tie_mns": None, + "n_tie_mixed": None, + "fragile_success": None, + "pairwise_vs_real_winner": [], + "paired_config_count": len(paired_configs), + "expected_config_count": len(surface.expected_configs), + "real_top3_configs": [], + "missing_configs": missing_configs, + "sources": list(surface.sources), + }, + anomalies, + ) + + sim_ranking = sorted(paired_configs, key=lambda config: (surface.values[config][0], config)) + real_ranking = sorted(paired_configs, key=lambda config: (surface.values[config][1], config)) + sim_winner = sim_ranking[0] + real_winner = real_ranking[0] + real_best = surface.values[real_winner][1] + regret = surface.values[sim_winner][1] / real_best - 1.0 + margin = surface.values[real_ranking[1]][1] / real_best - 1.0 + log_ratios = { + config: math.log(surface.values[config][0] / surface.values[config][1]) + for config in paired_configs + } + spread_all = max(log_ratios.values()) - min(log_ratios.values()) + top3 = tuple(real_ranking[:3]) + top3_ratios = [log_ratios[config] for config in top3] + spread_top3 = max(top3_ratios) - min(top3_ratios) + scale = math.exp(fmean(log_ratios.values())) + failure = regret > 0.05 + residual_exceeds_log_margin = spread_top3 > math.log1p(margin) + h_scale_holds = failure == residual_exceeds_log_margin + + pairwise = [] + inversion_counts = {axis: 0 for axis in INVERSION_AXES} + sim_at_real_winner = surface.values[real_winner][0] + for config in paired_configs: + sim_value, real_value = surface.values[config] + real_pair_log = math.log(real_value / real_best) + sim_pair_log = math.log(sim_value / sim_at_real_winner) + pair_margin_percent = 100.0 * real_pair_log + delta_percent = 100.0 * (sim_pair_log - real_pair_log) + axis = pair_axis(real_winner, config) + inversion = real_value > real_best and sim_value < sim_at_real_winner + delta_inversion = ( + pair_margin_percent > 0 + and delta_percent < -pair_margin_percent + ) + if inversion != delta_inversion: + anomalies.append( + f"{surface.case} / {surface.objective} / {config}: direct inversion " + f"test disagrees with delta < -margin" + ) + if inversion: + inversion_counts[axis] += 1 + pairwise.append( + { + "config": config, + "axis_vs_real_winner": axis, + "real_pair_margin_percent": pair_margin_percent, + "signed_pairwise_delta_percent": delta_percent, + "inversion": inversion, + } + ) + + winner_pair = next(item for item in pairwise if item["config"] == sim_winner) + winner_match = sim_winner == real_winner + if winner_match: + winner_pair_relation = "match" + elif winner_pair["inversion"]: + winner_pair_relation = "reversal" + elif surface.values[sim_winner][0] == sim_at_real_winner: + winner_pair_relation = "sim-tie" + else: + winner_pair_relation = "non-reversal" + tie_counts = {axis: 0 for axis in INVERSION_AXES} + if winner_pair_relation == "sim-tie": + tie_counts[winner_pair["axis_vs_real_winner"]] += 1 + + if sim_winner != surface.expected_sim_winner: + anomalies.append( + f"{surface.case} / {surface.objective}: recomputed sim winner " + f"{sim_winner} != frozen JSON {surface.expected_sim_winner}" + ) + if real_winner != surface.expected_real_winner: + anomalies.append( + f"{surface.case} / {surface.objective}: recomputed real winner " + f"{real_winner} != frozen JSON {surface.expected_real_winner}" + ) + if surface.expected_regret is not None and not math.isclose( + regret, float(surface.expected_regret), rel_tol=1e-12, abs_tol=1e-12 + ): + anomalies.append( + f"{surface.case} / {surface.objective}: recomputed regret " + f"{regret:.12g} != frozen JSON {float(surface.expected_regret):.12g}" + ) + + return ( + { + "case": surface.case, + "objective": surface.objective, + "sim_winner": sim_winner, + "real_winner": real_winner, + "regret": regret, + "config_uniform_scale": scale, + "log_ratio_spread_all": spread_all, + "log_ratio_spread_real_top3": spread_top3, + "real_relative_margin_best_vs_second": margin, + "sim_winner_real_relative_loss": regret, + "failure": failure, + "h_scale_condition_spread_gt_log_margin": residual_exceeds_log_margin, + "h_scale_verdict": "HOLDS" if h_scale_holds else "COUNTEREXAMPLE", + "winner_match": winner_match, + "winner_deciding_axis": pair_axis(real_winner, sim_winner), + "winner_pair_m_s_percent": winner_pair["real_pair_margin_percent"], + "winner_pair_delta_s_percent": winner_pair["signed_pairwise_delta_percent"], + "winner_pair_strict_reversal": winner_pair["inversion"], + "winner_pair_relation": winner_pair_relation, + "n_inv_tp": inversion_counts["tp-axis"], + "n_inv_mns": inversion_counts["mns-axis"], + "n_inv_mixed": inversion_counts["mixed"], + "n_tie_tp": tie_counts["tp-axis"], + "n_tie_mns": tie_counts["mns-axis"], + "n_tie_mixed": tie_counts["mixed"], + "fragile_success": winner_match and margin < 0.01, + "pairwise_vs_real_winner": pairwise, + "paired_config_count": len(paired_configs), + "expected_config_count": len(surface.expected_configs), + "real_top3_configs": list(top3), + "missing_configs": missing_configs, + "sources": list(surface.sources), + }, + anomalies, + ) + + +def row_index(rows: list[dict[str, Any]]) -> dict[tuple[str, str], dict[str, Any]]: + return {(row["case"], row["objective"]): row for row in rows} + + +def cross_checks(rows: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], list[str]]: + by_key = row_index(rows) + checks = [] + anomalies = [] + + def check(name: str, observed: float, expected_percent: float) -> None: + observed_percent = observed * 100.0 + passed = round(observed_percent, 1) == expected_percent + checks.append( + { + "name": name, + "observed_percent": observed_percent, + "expected_percent": expected_percent, + "passed": passed, + } + ) + if not passed: + anomalies.append( + f"hard cross-check failed: {name}: observed {observed_percent:.6f}% " + f"vs expected {expected_percent:.1f}%" + ) + + check( + "q30 high-pressure fixed-pd TTFT mean regret", + by_key[("q30 fixed-pd high", "ttft_mean_ms")]["regret"], + 58.0, + ) + check( + "q235 A0 fixed-pd TPOT mean regret", + by_key[("q235 A0 fixed-pd", "tpot_mean_ms")]["regret"], + 33.0, + ) + check( + "q235 A1 fixed-pd TPOT mean regret", + by_key[("q235 A1 fixed-pd", "tpot_mean_ms")]["regret"], + 33.0, + ) + trace_pd_rows = [row for row in rows if row["case"] == "q30 trace-pd"] + trace_passed = len(trace_pd_rows) == 6 and all( + round(row["regret"] * 100.0, 1) == 0.0 for row in trace_pd_rows + ) + checks.append( + { + "name": "q30 trace-pd all-objective regret", + "observed_percent": [row["regret"] * 100.0 for row in trace_pd_rows], + "expected_percent": [0.0] * 6, + "passed": trace_passed, + } + ) + if not trace_passed: + anomalies.append( + "hard cross-check failed: q30 trace-pd does not have six objectives all at 0.0% regret" + ) + + def check_sequence( + name: str, observed: list[float], expected_percent: list[float] + ) -> None: + observed_percent = [value * 100.0 for value in observed] + passed = [round(value, 1) for value in observed_percent] == expected_percent + checks.append( + { + "name": name, + "observed_percent": observed_percent, + "expected_percent": expected_percent, + "passed": passed, + } + ) + if not passed: + anomalies.append( + f"hard cross-check failed: {name}: observed " + f"{[round(value, 6) for value in observed_percent]} vs expected " + f"{expected_percent}" + ) + + check_sequence( + "q235 Trace-PO TTFT p90 A0 to A1 regret", + [ + by_key[("q235 A0 trace-po", "ttft_p90_ms")]["regret"], + by_key[("q235 A1 trace-po", "ttft_p90_ms")]["regret"], + ], + [21.2, 0.3], + ) + fixed_pd_objectives = ( + "tpot_mean_ms", + "tpot_p90_ms", + "e2e_mean_ms", + "e2e_p90_ms", + ) + fixed_pd_expected = [33.0, 37.2, 30.7, 34.6] + check_sequence( + "q235 Fixed-PD TPOT/E2E regrets unchanged from A0 to A1", + [ + by_key[(f"q235 {campaign} fixed-pd", objective)]["regret"] + for campaign in ("A0", "A1") + for objective in fixed_pd_objectives + ], + fixed_pd_expected + fixed_pd_expected, + ) + return checks, anomalies + + +def known_attribution(input_root: Path) -> dict[str, Any]: + q30_relative = Path("q30-admission-diag/result.json") + q235_relative = Path("q235-state-diag/state-diagnosis-exact.json") + q30 = load_json(input_root / q30_relative) + q235 = load_json(input_root / q235_relative) + if q30.get("schema") != "qwen30-fixed-pd-ttft-admission-diagnosis-v1": + raise ValueError(f"unexpected schema: {q30_relative}") + if q235.get("schema") != "qwen235-fixed-pd-state-diagnosis-v1": + raise ValueError(f"unexpected schema: {q235_relative}") + + q30_cells = [] + for config, cell in sorted(q30["cells"].items()): + q30_cells.append( + { + "config": config, + "sim_tpot_over_real": cell["tpot_overprediction_ratio"], + "sim_required_slots": cell["simulator"]["required_slots"], + "mns": cell["mns"], + "real_required_slots_upper_bound": cell["real"]["required_slots_upper_bound"], + "sim_first_scheduling_wait_ms": cell["simulator"]["first_scheduling_delay_ms"], + "sim_queue_free_ttft_ms": cell["simulator"]["queue_free_ttft_ms"], + "sim_ttft_ms": cell["simulator"]["ttft_ms"], + "real_ttft_ms": cell["real"]["ttft_ms"], + "real_waiting_max": cell["real"]["periodic_queue"]["waiting_max"], + } + ) + + q235_configs = [] + for config in sorted(q235["simulator"]): + q235_configs.append( + { + "config": config, + "sim_own_decode_batch_mean": q235["simulator"][config]["decode_batch_size"]["mean"], + "real_exact_decode_batch_mean": q235["real_iteration_state"][config]["decode_batch_size"]["mean"], + "real_token_weighted_iteration_ms": q235["real_iteration_state"][config]["decode_token_weighted_iteration_elapsed_ms"], + "exact_state_matched_sim_ms": q235["exact_state_matched"]["configs"][config]["components_ms"]["total"], + "exact_state_coverage": q235["exact_state_matched"]["configs"][config]["coverage"], + } + ) + + component_order = ( + "total", + "moe_compute", + "attention", + "tp_dp_communication", + "ep_communication", + "moe_routing", + "dense_mlp_compute", + "pipeline_communication", + "runtime_overhead", + ) + q235_components = [] + own = q235["simulator_internal_all_step_tp8_minus_tp4_ms"] + exact = q235["exact_state_matched"]["tp8_minus_tp4_ms"] + for component in component_order: + q235_components.append( + { + "component": component, + "own_composition_tp8_minus_tp4_ms": own[component], + "exact_state_tp8_minus_tp4_ms": exact[component], + } + ) + return { + "q30_admission": { + "source": str(q30_relative), + "verdict": q30["verdict"], + "cells": q30_cells, + "contrasts": q30["contrasts"], + }, + "q235_state": { + "source": str(q235_relative), + "verdict": q235["verdict"], + "configs": q235_configs, + "components": q235_components, + "contrast_decomposition": q235["contrast_decomposition"], + "reference": q235["reference"], + }, + } + + +def directional_summary(rows: list[dict[str, Any]]) -> dict[str, Any]: + case_order = list(dict.fromkeys(row["case"] for row in rows)) + by_case = [] + for case in case_order: + case_rows = [row for row in rows if row["case"] == case] + by_case.append( + { + "case": case, + "n_inv_tp": sum(row["n_inv_tp"] for row in case_rows), + "n_inv_mns": sum(row["n_inv_mns"] for row in case_rows), + "n_inv_mixed": sum(row["n_inv_mixed"] for row in case_rows), + "n_tie_tp": sum(row["n_tie_tp"] for row in case_rows), + "n_tie_mns": sum(row["n_tie_mns"] for row in case_rows), + "n_tie_mixed": sum(row["n_tie_mixed"] for row in case_rows), + } + ) + fragile = [ + { + "case": row["case"], + "objective": row["objective"], + "real_relative_margin_percent": 100.0 + * row["real_relative_margin_best_vs_second"], + } + for row in rows + if row["fragile_success"] + ] + material_failures = [ + { + "case": row["case"], + "objective": row["objective"], + "winner_deciding_axis": row["winner_deciding_axis"], + "winner_pair_m_s_percent": row["winner_pair_m_s_percent"], + "winner_pair_delta_s_percent": row["winner_pair_delta_s_percent"], + "regret": row["regret"], + } + for row in rows + if row["failure"] + ] + boundary_winner_ties = [ + { + "case": row["case"], + "objective": row["objective"], + "real_winner": row["real_winner"], + "sim_winner": row["sim_winner"], + "winner_deciding_axis": row["winner_deciding_axis"], + } + for row in rows + if row["winner_pair_relation"] == "sim-tie" + ] + return { + "axis_inversions_total": { + "tp-axis": sum(row["n_inv_tp"] for row in rows), + "mns-axis": sum(row["n_inv_mns"] for row in rows), + "mixed": sum(row["n_inv_mixed"] for row in rows), + }, + "axis_inversions_by_case": by_case, + "axis_ties_total": { + "tp-axis": sum(row["n_tie_tp"] for row in rows), + "mns-axis": sum(row["n_tie_mns"] for row in rows), + "mixed": sum(row["n_tie_mixed"] for row in rows), + }, + "strict_winner_reversal_count": sum( + row["winner_pair_relation"] == "reversal" for row in rows + ), + "sim_tie_winner_mismatch_count": sum( + row["winner_pair_relation"] == "sim-tie" for row in rows + ), + "boundary_winner_ties": boundary_winner_ties, + "material_failures": material_failures, + "margin_robust_success_count": sum( + row["winner_match"] and not row["fragile_success"] for row in rows + ), + "fragile_success_count": len(fragile), + "fragile_successes": fragile, + } + + +def analyze(input_root: Path) -> dict[str, Any]: + surfaces = load_surfaces(input_root) + rows = [] + anomalies = [] + for surface in surfaces: + row, row_anomalies = analyze_surface(surface) + rows.append(row) + anomalies.extend(row_anomalies) + checks, check_anomalies = cross_checks(rows) + anomalies.extend(check_anomalies) + counterexamples = [ + {"case": row["case"], "objective": row["objective"]} + for row in rows + if row["h_scale_verdict"] == "COUNTEREXAMPLE" + ] + data_gaps = [ + { + "case": row["case"], + "objective": row["objective"], + "missing_configs": row["missing_configs"], + } + for row in rows + if row["missing_configs"] or row["h_scale_verdict"] == "N/A" + ] + directional = directional_summary(rows) + return { + "schema": "frontier-split-rootcause-decomposition-v2", + "definitions": { + "objective_direction": "all objectives are minimized", + "config_uniform_scale": "geomean_c(sim_c / real_c) over paired configs", + "log_ratio_spread_all": "max_c(log(sim_c / real_c)) - min_c(log(sim_c / real_c))", + "log_ratio_spread_real_top3": "log-ratio spread over the three configs with lowest real objective values", + "real_relative_margin_best_vs_second": "real_second / real_best - 1", + "regret": "real_sim_winner / real_best - 1, recomputed from per-config values", + "failure": "regret > 0.05", + "h_scale_verdict": "HOLDS iff failure equals (top-3 log-ratio spread > log1p(real relative margin))", + "signed_pairwise_delta_percent": "100 * (log(sim_c / sim_w) - log(real_c / real_w)); negative favors c relative to real", + "real_pair_margin_percent": "100 * log(real_c / real_w)", + "pair_inversion": "real_c > real_w and sim_c < sim_w, equivalently delta_c < -m_c", + "winner_pair_tie": "sim_s == sim_w with real_s > real_w; a decision-boundary winner-label mismatch, not a strict inversion", + "winner_deciding_axis": "axis difference between real winner w and sim winner s; none for a winner match", + "fragile_success": "winner match and real best-vs-second relative margin < 0.01", + }, + "summary": { + "row_count": len(rows), + "case_count": len({row["case"] for row in rows}), + "failure_count": sum(row["failure"] is True for row in rows), + "winner_mismatch_count": sum(row["winner_match"] is False for row in rows), + "h_scale_counterexample_count": len(counterexamples), + "h_scale_counterexamples": counterexamples, + "data_gap_count": len(data_gaps), + }, + "directional_summary": directional, + "rows": rows, + "cross_checks": checks, + "data_gaps": data_gaps, + "anomalies": anomalies, + "known_attribution": known_attribution(input_root), + } + + +def percent(value: float | None, digits: int = 1) -> str: + if value is None: + return "N/A" + return f"{value * 100.0:.{digits}f}%" + + +def multiple(value: float | None) -> str: + if value is None: + return "N/A" + return f"{value:.3f}x" + + +def percent_points(value: float | None, signed: bool = False) -> str: + if value is None: + return "N/A" + sign = "+" if signed else "" + return f"{value:{sign}.1f}%" + + +def markdown(result: dict[str, Any]) -> str: + summary = result["summary"] + lines = [ + "# Frontier good/bad split decomposition", + "", + "All 70 rows are recomputed from frozen per-config JSON values. Lower is better for every objective. " + "`scale` is the geometric mean of `sim/real`; spreads are shown as `100 ×` log-space width, while `real margin` is the ordinary relative difference `100 × (real_second / real_best - 1)`. " + "H-SCALE compares the log-space spread with `log1p(real margin)`, not directly with the ordinary relative margin. " + "The real top-3 neighborhood is selected independently for every row. Directional `m_s` and `delta_s` are already expressed in log-percentage points.", + "", + f"- Rows / case surfaces: {summary['row_count']} / {summary['case_count']}.", + f"- Failures (`regret > 5%`): {summary['failure_count']}.", + f"- Exact winner mismatches: {summary['winner_mismatch_count']}.", + f"- H-SCALE counterexamples: {summary['h_scale_counterexample_count']}.", + "", + "## Unified decomposition", + "", + "| case | objective | sim winner | real winner | regret | scale | spread all | spread real top-3 | real margin (relative) | sim-winner real loss | failure | H-SCALE | deciding axis | m_s | delta_s | n_inv_tp | n_inv_mns | n_inv_mixed | fragile success |", + "|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---:|---:|---:|---:|---:|---|", + ] + for row in result["rows"]: + lines.append( + "| {case} | {objective} | {sim} | {real} | {regret} | {scale} | " + "{spread_all} | {spread_top3} | {margin} | {loss} | {failure} | {verdict} | " + "{axis} | {pair_margin} | {delta} | {n_tp} | {n_mns} | {n_mixed} | {fragile} |".format( + case=row["case"], + objective=row["objective"], + sim=row["sim_winner"] or "N/A", + real=row["real_winner"] or "N/A", + regret=percent(row["regret"]), + scale=multiple(row["config_uniform_scale"]), + spread_all=percent(row["log_ratio_spread_all"]), + spread_top3=percent(row["log_ratio_spread_real_top3"]), + margin=percent(row["real_relative_margin_best_vs_second"]), + loss=percent(row["sim_winner_real_relative_loss"]), + failure=( + "yes" if row["failure"] is True else "no" if row["failure"] is False else "N/A" + ), + verdict=row["h_scale_verdict"], + axis=row["winner_deciding_axis"] or "N/A", + pair_margin=percent_points(row["winner_pair_m_s_percent"]), + delta=percent_points(row["winner_pair_delta_s_percent"], signed=True), + n_tp=row["n_inv_tp"] if row["n_inv_tp"] is not None else "N/A", + n_mns=row["n_inv_mns"] if row["n_inv_mns"] is not None else "N/A", + n_mixed=( + row["n_inv_mixed"] if row["n_inv_mixed"] is not None else "N/A" + ), + fragile=( + "yes" + if row["fragile_success"] is True + else "no" + if row["fragile_success"] is False + else "N/A" + ), + ) + ) + + lines.extend( + [ + "", + "## H-SCALE verdict", + "", + ] + ) + if summary["h_scale_counterexample_count"] == 0: + lines.append( + "H-SCALE holds on every available row: failure occurs exactly when the real-top-3 differential residual exceeds `log1p` of the real best-vs-second relative margin." + ) + else: + failure_counterexamples = sum( + row["failure"] is True and row["h_scale_verdict"] == "COUNTEREXAMPLE" + for row in result["rows"] + ) + nonfailure_counterexamples = summary["h_scale_counterexample_count"] - failure_counterexamples + lines.extend( + [ + "H-SCALE is not sufficient as stated; the following rows are counterexamples to the biconditional:", + "", + f"All {nonfailure_counterexamples} observed counterexamples are non-failures with spread > `log1p(margin)`; " + f"there are {failure_counterexamples} failures with spread <= `log1p(margin)`. Thus the threshold is necessary for the observed failures but not sufficient to predict them.", + "", + "| case | objective | failure | spread real top-3 | real margin | counterexample type |", + "|---|---|---|---:|---:|---|", + ] + ) + for row in result["rows"]: + if row["h_scale_verdict"] != "COUNTEREXAMPLE": + continue + kind = ( + "failure with spread <= log1p(margin)" + if row["failure"] + else "non-failure with spread > log1p(margin)" + ) + lines.append( + f"| {row['case']} | {row['objective']} | " + f"{'yes' if row['failure'] else 'no'} | " + f"{percent(row['log_ratio_spread_real_top3'])} | " + f"{percent(row['real_relative_margin_best_vs_second'])} | {kind} |" + ) + + directional = result["directional_summary"] + lines.extend( + [ + "", + "## S0b directional conclusions", + "", + "Directional pair statistics remove the S0 range statistic's ambiguity by measuring every config against the real winner. " + "Strict inversions use the frozen-value test `real_c > real_w and sim_c < sim_w`; simulator ties are not counted as inversions.", + "", + "### (a) Which config axis decides each material failure?", + "", + "| case | objective | real winner → sim winner | deciding axis | m_s | delta_s | -delta_s | regret |", + "|---|---|---|---|---:|---:|---:|---:|", + ] + ) + for row in result["rows"]: + if not row["failure"]: + continue + lines.append( + f"| {row['case']} | {row['objective']} | {row['real_winner']} → " + f"{row['sim_winner']} | {row['winner_deciding_axis']} | " + f"{percent_points(row['winner_pair_m_s_percent'])} | " + f"{percent_points(row['winner_pair_delta_s_percent'], signed=True)} | " + f"{percent_points(-row['winner_pair_delta_s_percent'])} | " + f"{percent(row['regret'])} |" + ) + + lines.extend( + [ + "", + "### (b) Where do TP-axis and MNS-axis inversions and boundary ties concentrate?", + "", + "`n_inv_*` counts sum strict `(real winner, c)` pair inversions across all objectives in each case; a row can contain more than one inverted pair. " + "`n_tie_*` counts cover exact simulator ties only for the simulator-selected winner versus the real winner. Ties remain decision-boundary winner-label mismatches, not strict reversals.", + "", + "| case | n_inv_tp | n_inv_mns | n_inv_mixed | n_tie_tp | n_tie_mns | n_tie_mixed |", + "|---|---:|---:|---:|---:|---:|---:|", + ] + ) + for item in directional["axis_inversions_by_case"]: + lines.append( + f"| {item['case']} | {item['n_inv_tp']} | {item['n_inv_mns']} | " + f"{item['n_inv_mixed']} | {item['n_tie_tp']} | {item['n_tie_mns']} | " + f"{item['n_tie_mixed']} |" + ) + totals = directional["axis_inversions_total"] + tie_totals = directional["axis_ties_total"] + lines.append( + f"| **total** | **{totals['tp-axis']}** | **{totals['mns-axis']}** | " + f"**{totals['mixed']}** | **{tie_totals['tp-axis']}** | " + f"**{tie_totals['mns-axis']}** | **{tie_totals['mixed']}** |" + ) + + def concentrated(axis_key: str) -> str: + ranked = sorted( + ( + (item[axis_key], item["case"]) + for item in directional["axis_inversions_by_case"] + if item[axis_key] > 0 + ), + key=lambda item: (-item[0], item[1]), + ) + return ", ".join(f"{case} ({count})" for count, case in ranked[:5]) + + by_key = row_index(result["rows"]) + q30_ttft_axes = [ + by_key[("q30 fixed-pd high", objective)]["winner_deciding_axis"] + for objective in ("ttft_mean_ms", "ttft_p90_ms") + ] + q235_fixed_pd_failures = [ + row + for row in result["rows"] + if row["case"] in ("q235 A0 fixed-pd", "q235 A1 fixed-pd") + and row["failure"] + ] + q235_fixed_pd_axes = { + row["winner_deciding_axis"] for row in q235_fixed_pd_failures + } + q235_consistency = ( + "The eight Q235 A0/A1 Fixed-PD failures are all tp-axis. This does not conflict with the A0/MNS64 state-drift diagnosis, but does not constitute additional state-drift evidence for A1/MNS128." + if len(q235_fixed_pd_failures) == 8 and q235_fixed_pd_axes == {"tp-axis"} + else "The Q235 A0/A1 Fixed-PD failures do not match the expected eight tp-axis cases; no state-drift compatibility claim is made." + ) + lines.extend( + [ + "", + f"- Among strict inversions, TP-axis cases are most concentrated in: {concentrated('n_inv_tp')}.", + f"- Among strict inversions, MNS-axis cases are most concentrated in: {concentrated('n_inv_mns')}.", + f"- {q235_consistency}", + f"- Q30 admission evidence does not map one-to-one to an MNS-axis deciding pair: high-pressure Fixed-PD TTFT mean is **{q30_ttft_axes[0]}** and p90 is **{q30_ttft_axes[1]}**. " + "This is an axis/mechanism mismatch, not a direct causal contradiction: changing TP changes offered arrival pressure, while the diagnosed nonlinear event is crossing the MNS admission cap. It rules out the simpler claim that an MNS-threshold mechanism must appear as an mns-axis winner pair.", + f"- All {directional['sim_tie_winner_mismatch_count']} exact simulator winner ties are mns-axis. Including these boundary errors gives MNS **{totals['mns-axis']} + {tie_totals['mns-axis']} = {totals['mns-axis'] + tie_totals['mns-axis']}**, comparable to TP's **{totals['tp-axis']}** strict inversions.", + "", + "The exact ties below show that Frontier is completely insensitive to MNS changes within the corresponding case/objective cell:", + "", + "| case | objective | real winner ↔ simulator-selected tied winner (MNS pair) |", + "|---|---|---|", + ] + ) + for item in directional["boundary_winner_ties"]: + lines.append( + f"| {item['case']} | {item['objective']} | {item['real_winner']} ↔ " + f"{item['sim_winner']} |" + ) + + lines.extend( + [ + "", + "### (c) How many exact-winner successes are margin-robust?", + "", + f"Among exact winner matches, **{directional['margin_robust_success_count']}** are margin-robust (`margin >= 1%`) and " + f"**{directional['fragile_success_count']}** are fragile (`margin < 1%`). " + f"Separately, {directional['sim_tie_winner_mismatch_count']} winner-label mismatches come from exact simulator ties and sit on the decision boundary rather than being strict inversions.", + "", + "| fragile-success case | objective | real margin |", + "|---|---|---:|", + ] + ) + for item in directional["fragile_successes"]: + lines.append( + f"| {item['case']} | {item['objective']} | " + f"{item['real_relative_margin_percent']:.1f}% |" + ) + + lines.extend( + [ + "", + "## Cross-checks", + "", + "| check | observed | frozen expectation | result |", + "|---|---:|---:|---|", + ] + ) + for check in result["cross_checks"]: + observed = check["observed_percent"] + expected = check["expected_percent"] + if isinstance(observed, list): + observed_text = ", ".join(f"{value:.1f}%" for value in observed) + expected_text = ", ".join(f"{value:.1f}%" for value in expected) + else: + observed_text = f"{observed:.1f}%" + expected_text = f"{expected:.1f}%" + lines.append( + f"| {check['name']} | {observed_text} | {expected_text} | " + f"{'PASS' if check['passed'] else 'FAIL'} |" + ) + + lines.extend(["", "## Data gaps", ""]) + if result["data_gaps"]: + for gap in result["data_gaps"]: + missing = ", ".join(gap["missing_configs"]) or "insufficient paired configs" + lines.append(f"- {gap['case']} / {gap['objective']}: N/A because {missing}.") + else: + lines.append("No per-config sim/real pairing gaps were found in the requested surfaces.") + + lines.extend(["", "## 异常", ""]) + if result["anomalies"]: + lines.extend(f"- {item}" for item in result["anomalies"]) + else: + lines.append("- 无。重算 winner/regret 与 frozen JSON 一致,硬性交叉核对均通过。") + + lines.extend( + [ + "", + "## Figure", + "", + "`margin-vs-residual.png` uses x = `100 × log1p(real best-vs-second relative margin)` and y = `100 ×` real-top-3 log-ratio spread. " + "Both axes are logarithmic because the real margins span six orders of magnitude. Green means exact winner match; red means winner mismatch. " + "The diagonal is the H-SCALE threshold, while failure itself is defined by regret, not by exact winner match.", + "", + "`decision-pair-axis.png` is a mechanism census, not a hypothesis test: for a strict winner reversal, `-delta_s > m_s` is algebraically equivalent to the mismatch. " + "The frozen surfaces also contain deterministic winner-label mismatches caused by exact simulator ties; those points lie on `-delta_s = m_s` and are not counted as strict inversions. " + "Color identifies the winner-deciding config axis and marker area is proportional to regret.", + "", + "## A0 vs A1 对照", + "", + "A1 repairs the Q235 Trace-PO p90 ranking error but leaves the four large Fixed-PD TPOT/E2E regrets unchanged.", + "", + "| case / objective | A0 regret | A1 regret |", + "|---|---:|---:|", + ] + ) + by_key = row_index(result["rows"]) + a0_a1_rows = ( + ("trace-po / ttft_p90_ms", "trace-po", "ttft_p90_ms"), + ("fixed-pd / tpot_mean_ms", "fixed-pd", "tpot_mean_ms"), + ("fixed-pd / tpot_p90_ms", "fixed-pd", "tpot_p90_ms"), + ("fixed-pd / e2e_mean_ms", "fixed-pd", "e2e_mean_ms"), + ("fixed-pd / e2e_p90_ms", "fixed-pd", "e2e_p90_ms"), + ) + for label, case, objective in a0_a1_rows: + a0 = by_key[(f"q235 A0 {case}", objective)]["regret"] + a1 = by_key[(f"q235 A1 {case}", objective)]["regret"] + lines.append(f"| {label} | {percent(a0)} | {percent(a1)} |") + lines.extend( + [ + "", + "Thus Trace-PO TTFT p90 changes **21.2% → 0.3%**, whereas Fixed-PD remains **33.0% / 37.2% / 30.7% / 34.6%** in both A0 and A1.", + "", + "## 已知分量归因摘要", + "", + "### Q30 high-pressure Fixed-PD TTFT: first-scheduling wait vs execution-side TTFT", + "", + "`queue-free TTFT` is the execution-side TTFT after removing first-scheduling wait; in the frozen diagnostic, simulated TTFT is their sum (up to floating-point rounding).", + "", + "| config | sim TPOT / real | sim slots / MNS | real slots upper / MNS | first-scheduling wait | queue-free TTFT | sim TTFT | real TTFT | real waiting max |", + "|---|---:|---:|---:|---:|---:|---:|---:|---:|", + ] + ) + q30 = result["known_attribution"]["q30_admission"] + for cell in q30["cells"]: + lines.append( + f"| {cell['config']} | {cell['sim_tpot_over_real']:.2f}x | " + f"{cell['sim_required_slots']:.1f}/{cell['mns']} | " + f"{cell['real_required_slots_upper_bound']:.1f}/{cell['mns']} | " + f"{cell['sim_first_scheduling_wait_ms']:.1f} ms | " + f"{cell['sim_queue_free_ttft_ms']:.1f} ms | " + f"{cell['sim_ttft_ms']:.1f} ms | {cell['real_ttft_ms']:.1f} ms | " + f"{cell['real_waiting_max']} |" + ) + contrasts = q30["contrasts"] + lines.extend( + [ + "", + "For TP4/MNS32 minus TP2/MNS64, the observed simulator contrast is " + f"**{contrasts['observed_sim_ttft_tp4_minus_tp2_ms']:+.1f} ms**; removing first-scheduling wait changes it to " + f"**{contrasts['queue_free_sim_ttft_tp4_minus_tp2_ms']:+.1f} ms**, aligned with the real contrast " + f"**{contrasts['real_ttft_tp4_minus_tp2_ms']:+.1f} ms**. This localizes the Q30 high-pressure TTFT reversal to admission queueing, while the upstream service-time overprediction remains unattributed in the frozen data.", + "", + "### Q235 Fixed-PD: own-composition vs exact-state contrast", + "", + "Positive TP8−TP4 means TP8 is slower and therefore agrees with the real TP4 winner.", + "", + "| config | sim own decode batch mean | real exact decode batch mean | real token-weighted iteration | exact-state matched sim | coverage |", + "|---|---:|---:|---:|---:|---:|", + ] + ) + q235 = result["known_attribution"]["q235_state"] + for config in q235["configs"]: + lines.append( + f"| {config['config']} | {config['sim_own_decode_batch_mean']:.3f} | " + f"{config['real_exact_decode_batch_mean']:.3f} | " + f"{config['real_token_weighted_iteration_ms']:.4f} ms | " + f"{config['exact_state_matched_sim_ms']:.4f} ms | " + f"{config['exact_state_coverage'] * 100.0:.1f}% |" + ) + lines.extend( + [ + "", + "| component | own-composition TP8−TP4 | exact-state TP8−TP4 |", + "|---|---:|---:|", + ] + ) + for component in q235["components"]: + lines.append( + f"| {component['component']} | " + f"{component['own_composition_tp8_minus_tp4_ms']:+.4f} ms | " + f"{component['exact_state_tp8_minus_tp4_ms']:+.4f} ms |" + ) + decomposition = q235["contrast_decomposition"] + reference = q235["reference"] + lines.extend( + [ + "", + f"The total contrast flips from **{decomposition['simulator_internal_tp8_minus_tp4_ms']:+.4f} ms** on Frontier's own composition to " + f"**{decomposition['exact_state_matched_tp8_minus_tp4_ms']:+.4f} ms** at exact real state, a " + f"**{decomposition['state_composition_shift_ms']:+.4f} ms** state-composition shift. " + f"The exact real iteration contrast is **{reference['exact_iteration_tp8_minus_tp4_ms']:+.4f} ms** and the frozen real TPOT contrast is " + f"**{reference['observed_real_tp8_minus_tp4_ms']:+.4f} ms**. This supports closed-loop state/composition drift for Q235 Fixed-PD; it does not by itself attribute every remaining conditional execution residual.", + "", + ] + ) + return "\n".join(lines) + + +def render_plot(rows: list[dict[str, Any]], output: Path) -> None: + os.environ.setdefault( + "MPLCONFIGDIR", "/tmp/aituner-frontier-split-rootcause-matplotlib" + ) + import matplotlib + + matplotlib.use("Agg") + import matplotlib.pyplot as plt + + usable = [ + row + for row in rows + if row["real_relative_margin_best_vs_second"] is not None + and row["log_ratio_spread_real_top3"] is not None + ] + if not usable: + raise ValueError("no complete rows available for margin-vs-residual plot") + + plt.rcParams.update( + { + "font.family": "DejaVu Sans", + "font.size": 8, + "axes.titlesize": 10, + "axes.labelsize": 9, + } + ) + fig, ax = plt.subplots(figsize=(13.0, 9.0), dpi=180) + colors = {True: "#2a9d8f", False: "#d1495b"} + labels = {True: "winner match", False: "winner mismatch"} + for match in (True, False): + selected = [row for row in usable if row["winner_match"] is match] + ax.scatter( + [ + math.log1p(row["real_relative_margin_best_vs_second"]) * 100.0 + for row in selected + ], + [row["log_ratio_spread_real_top3"] * 100.0 for row in selected], + s=28, + color=colors[match], + edgecolor="white", + linewidth=0.4, + alpha=0.88, + label=labels[match], + zorder=3, + ) + + for index, row in enumerate(usable): + x = math.log1p(row["real_relative_margin_best_vs_second"]) * 100.0 + y = row["log_ratio_spread_real_top3"] * 100.0 + dx = 3 if index % 2 == 0 else -3 + dy = 3 if (index // 2) % 2 == 0 else -5 + ax.annotate( + row["case"], + (x, y), + xytext=(dx, dy), + textcoords="offset points", + ha="left" if dx > 0 else "right", + va="bottom" if dy > 0 else "top", + fontsize=4.5, + color="#303030", + alpha=0.82, + clip_on=True, + zorder=4, + ) + + max_value = max( + max( + math.log1p(row["real_relative_margin_best_vs_second"]) + for row in usable + ), + max(row["log_ratio_spread_real_top3"] for row in usable), + ) * 100.0 + min_value = min( + min( + math.log1p(row["real_relative_margin_best_vs_second"]) + for row in usable + ), + min(row["log_ratio_spread_real_top3"] for row in usable), + ) * 100.0 + lower = min_value / 5.0 + upper = max_value * 2.0 + ax.plot( + [lower, upper], + [lower, upper], + linestyle="--", + color="#555555", + linewidth=1.0, + label="spread = log1p(margin)", + zorder=2, + ) + ax.set_xscale("log") + ax.set_yscale("log") + ax.set_xlim(left=lower, right=upper) + ax.set_ylim(bottom=lower, top=upper) + ax.set_xlabel("Real best-vs-second log margin: 100 × log1p(relative margin)") + ax.set_ylabel("Top-3 differential residual: 100 × log-ratio spread") + ax.set_title("Frontier selection: real margin vs config-differential residual") + ax.grid(True, color="#d8d8d8", linewidth=0.5, alpha=0.7, zorder=1) + ax.legend(frameon=False, loc="upper left") + fig.tight_layout() + output.parent.mkdir(parents=True, exist_ok=True) + temporary = output.with_suffix(output.suffix + ".tmp") + fig.savefig( + temporary, + format="png", + dpi=180, + metadata={"Software": "analyze_split_decomposition.py"}, + ) + plt.close(fig) + os.replace(temporary, output) + + +def render_decision_pair_axis_plot( + rows: list[dict[str, Any]], output: Path +) -> None: + os.environ.setdefault( + "MPLCONFIGDIR", "/tmp/aituner-frontier-split-rootcause-matplotlib" + ) + import matplotlib + + matplotlib.use("Agg") + import matplotlib.pyplot as plt + from matplotlib.lines import Line2D + + usable = [ + row + for row in rows + if row["winner_match"] is False + and row["winner_pair_m_s_percent"] > 0 + and -row["winner_pair_delta_s_percent"] > 0 + ] + if not usable: + raise ValueError("no winner mismatches available for decision-pair-axis plot") + + plt.rcParams.update( + { + "font.family": "DejaVu Sans", + "font.size": 8, + "axes.titlesize": 10, + "axes.labelsize": 9, + } + ) + colors = { + "tp-axis": "#3572b0", + "mns-axis": "#e07a1f", + "mixed": "#7b4ab5", + } + fig, ax = plt.subplots(figsize=(11.0, 8.0), dpi=180) + for axis in INVERSION_AXES: + selected = [row for row in usable if row["winner_deciding_axis"] == axis] + ax.scatter( + [row["winner_pair_m_s_percent"] for row in selected], + [-row["winner_pair_delta_s_percent"] for row in selected], + s=[1000.0 * row["regret"] for row in selected], + color=colors[axis], + edgecolor="white", + linewidth=0.6, + alpha=0.82, + label=axis, + zorder=3, + ) + + values = [row["winner_pair_m_s_percent"] for row in usable] + [ + -row["winner_pair_delta_s_percent"] for row in usable + ] + lower = min(values) / 1.8 + upper = max(values) * 1.8 + ax.plot( + [lower, upper], + [lower, upper], + linestyle="--", + color="#555555", + linewidth=1.0, + label="reversal boundary", + zorder=2, + ) + ax.set_xscale("log") + ax.set_yscale("log") + ax.set_xlim(lower, upper) + ax.set_ylim(lower, upper) + ax.set_xlabel("Winner-pair real log margin $m_s$ (%)") + ax.set_ylabel("Simulator preference strength $-\\delta_s$ (%)") + ax.set_title("Winner-deciding pair census by config axis") + ax.grid(True, color="#d8d8d8", linewidth=0.5, alpha=0.7, zorder=1) + axis_handles = [ + Line2D( + [0], + [0], + marker="o", + linestyle="none", + markerfacecolor=colors[axis], + markeredgecolor="white", + markersize=7, + label=axis, + ) + for axis in INVERSION_AXES + ] + axis_handles.append( + Line2D( + [0], + [0], + linestyle="--", + color="#555555", + linewidth=1.0, + label="reversal boundary", + ) + ) + axis_legend = ax.legend( + handles=axis_handles, frameon=False, loc="upper left", title="deciding axis" + ) + ax.add_artist(axis_legend) + size_handles = [ + ax.scatter( + [], + [], + s=10.0 * regret_percent, + facecolor="none", + edgecolor="#555555", + linewidth=0.7, + label=f"{regret_percent}%", + ) + for regret_percent in (5, 20, 50) + ] + ax.legend( + handles=size_handles, + frameon=False, + loc="lower right", + title="regret (marker area)", + ) + fig.tight_layout() + output.parent.mkdir(parents=True, exist_ok=True) + temporary = output.with_suffix(output.suffix + ".tmp") + fig.savefig( + temporary, + format="png", + dpi=180, + metadata={"Software": "analyze_split_decomposition.py"}, + ) + plt.close(fig) + os.replace(temporary, output) + + +def parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser() + parser.add_argument("--input-root", type=Path, default=DEFAULT_INPUT_ROOT) + parser.add_argument("--results-root", type=Path, default=DEFAULT_RESULTS_ROOT) + return parser.parse_args() + + +def main() -> None: + args = parse_args() + result = analyze(args.input_root.resolve()) + results_root = args.results_root.resolve() + atomic_write( + results_root / "decomposition.json", + json.dumps(result, indent=2, sort_keys=True) + "\n", + ) + atomic_write(results_root / "decomposition.md", markdown(result)) + render_plot(result["rows"], results_root / "margin-vs-residual.png") + render_decision_pair_axis_plot( + result["rows"], results_root / "decision-pair-axis.png" + ) + print( + json.dumps( + { + "status": "PASS" if not result["anomalies"] else "ANOMALY", + "rows": result["summary"]["row_count"], + "h_scale_counterexamples": result["summary"]["h_scale_counterexample_count"], + "anomalies": len(result["anomalies"]), + }, + sort_keys=True, + ) + ) + if result["anomalies"]: + raise SystemExit("analysis anomalies found; see results/decomposition.md") + + +if __name__ == "__main__": + main() diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/aituner.commit b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/aituner.commit new file mode 100644 index 0000000..159620e --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/aituner.commit @@ -0,0 +1 @@ +10567da5238db57c29a45e09ebd40343a7d1a543 diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/artifacts.sha256 b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/artifacts.sha256 new file mode 100644 index 0000000..d7d710c --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/artifacts.sha256 @@ -0,0 +1,180 @@ +bfbfb84d0310b312c3f3f0047219d5b615850d8b5b465282d6207ebe7b5ba987 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/analysis/comparison.json +aa0295fcd1a0465483ca3655aae94de543acf5b39bde2fcfff6ff40a7fb27f2a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/analysis/comparison.md +f2c2a94627194431497c751e1ed0e3768f1f7971c3667e1f70563f1d7c5118f3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/controller.log +30bdc9da5c8e638d6278b3ae3f812613a1f7ef296439e34d28dcb72ae30b0d1b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/profiles/qwen235-serving-allreduce.csv +87a92ee3ba6955b95cf0fcdbf5dd8675a8ec85bf4a0dfd61ae12a75c32679fac /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/profiles/qwen235-serving-allreduce.manifest.json +13a6cd30307a6dcdd12dc460949ce1e618cbd167cd976ad1b71ad6477869b55d /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/provenance/aituner.commit +e3f7b627aa262187e7e02b703595f32fb1ced508724484ff515a68fe037d4f03 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/provenance/frontier.commit +cd3d7ea44155f2d3e801926b35134d9735777256d72ba3f6a8f0cff2770ba09a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/provenance/input.sha256 +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/138b2eb7_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/479e1bac_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/5ae40466_model_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/6e9cd646_model_lock.file +a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_138b2eb7_predictions.csv +f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_138b2eb7_predictions.pkl +628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_479e1bac_predictions.csv +7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_479e1bac_predictions.pkl +7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_5ae40466.pkl +125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/cc-cache/all_reduce_6e9cd646.pkl +c56228d3654b51ea42c73f0883d58c67c19b1024c14f1677c63cb882e4dc3334 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/frontier_surface.json +cf54e197cfcb6b480ccbd658f73dad0a90597bd58399d1c07f25e3987ca02227 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/command.json +7796df845a96bdb9603baf372118466e27200fc10ee83b26b469d4e2c089308f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv +01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv +ab0ee72a7e2fbb3bf87c1653a9b1805b7805fd0f9ae3676ce69a0d9b112a4545 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json +84a609818002bcfb648a236a65b1d40dcd3c964360cb488dfb7549eddc546533 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/stderr.log +469ad61c0082534fa069a3ddc38510b4dc2180d016518834d3ac69f31c3ef7e5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns128/eval/stdout.log +080edcceed5f26160cd073a33c87aa8399cb0fe7cc9227bacb442aa1f90c5a30 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/command.json +db4e6b4632fc16a368407e458d26fd7a0adc9ae930755ac7784b899da60ee9fc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv +01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv +ab0ee72a7e2fbb3bf87c1653a9b1805b7805fd0f9ae3676ce69a0d9b112a4545 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json +eb987dc5b2222740cef7577350190033cccee41946638aa5e9a5fd20c6be8b45 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/stderr.log +62312ef79e6302bf7be0c6ba276211065950b46eb4b4aef0a031358a804e17ce /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp4_ep1_mns64/eval/stdout.log +89b4fc5eb8ccd1d5dc58b778352acaec5d496dded0181990fd3bce30c557ab48 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/command.json +bcc034a01b617f26aa4d7d760082fc1baa1e62b1b63ba50e11c9e234124d2c37 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv +51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv +758424010e61780cbcbb320d68caa22c2e35c067654f9c4ef98217ff90e07b30 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json +463057bcb54084ab86eb35fc4f33ad62d1b3ebd05b6b04a1510ce8df6e686807 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/stderr.log +0a0433b5c338632e35516e34b262cf8b714cea73488ba9e3459f4881a0fe327e /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns128/eval/stdout.log +2a69949a89e4dcedbede612b8beab834d63eb8683d97975bc779764304b1a627 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/command.json +88def2ba5bd00b375e0d907976bdd45c36faf53ad7980d8a1eb7e46057ccc8ec /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv +51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv +758424010e61780cbcbb320d68caa22c2e35c067654f9c4ef98217ff90e07b30 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json +6ed18c855e41284d27a8aeca93799ba55745e7527344671edfed449693b29758 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/stderr.log +30bc99671a0bedac259b8be1d980a2981a48439e2d04a742d7baa4cc82638b39 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd/runs/tp8_ep8_mns64/eval/stdout.log +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/138b2eb7_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/479e1bac_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/5ae40466_model_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/6e9cd646_model_lock.file +a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_138b2eb7_predictions.csv +f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_138b2eb7_predictions.pkl +628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_479e1bac_predictions.csv +7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_479e1bac_predictions.pkl +7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_5ae40466.pkl +125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/cc-cache/all_reduce_6e9cd646.pkl +e83e951102f893a83c7ef4ebbae6e6f998a1a43a8d9f7a6ed165f5db96609980 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/frontier_surface.json +e3f2699e0b9e182ab440acc690ddd8aaf0c954c5a83b7e1f444b3d494fcb2379 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/command.json +4a28f5aa74e7f9909545b1d1bc00a66098d85f7fff6b72fbb98d11d4a3f786c0 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv +0030204dc1047a02d601f7c1d57eee9fc2234080afd4e499c016b902a8da7573 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv +d5b01e9e1689106d8174b1246da2c0c7a35ab7b83b1f8f2f3919753ac320e1e3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json +599bb360437de9ebb16c1b29958c27b2c7f8e95eb047f9b8f26cb5709003755a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/stderr.log +386b21c7695ae4159fb60f5c8772fd91512319f1511769e9b675955b7f26a223 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns128/eval/stdout.log +3db0c671d763a7db9565b151464b96060455bafe7f065b35719e27c7d45b12b5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/command.json +dfe34eef677ee24edd43ef01008aa30ed02fd7f251876149a55b9436130f179b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv +0030204dc1047a02d601f7c1d57eee9fc2234080afd4e499c016b902a8da7573 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv +d5b01e9e1689106d8174b1246da2c0c7a35ab7b83b1f8f2f3919753ac320e1e3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json +9cceef591ceae33b03dddd558387bf4e1853c2a3e74913dd7839fb47dacfcb9c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/stderr.log +1d5494b859450660d3f06050a91af27072ec4d15f4fe4bfb159f32f2e6769b89 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp4_ep1_mns64/eval/stdout.log +439fe52e1d3c10c9c5cb4db20308e62da26a44866085d44768244d82ff0d22d1 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/command.json +9c1211fc6032dfa395933dbd2977115c5494a37b814c335dd2ed1ce70716c7ce /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv +d3ebd523ddf95f021d98c043d290670b20d8c0ffc2fdbca27fc5740ae46981f5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv +264cd59b0b0b9797f5054f8527e13412a9c9b5fd3a4608d3f81b228a6c08d0bc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json +80adf7553fe587b542d0c43acf6850412199503a0eaa17ebbb04ac2870326059 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/stderr.log +3b86967c56b8dbd8db2edbbbbae367864ef41c703ee51fc4636811ca43f5c1ac /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns128/eval/stdout.log +0ee66ca4a8363be762616e69329287c5a74993f36283ed5a470a4b012ec9a000 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/command.json +98b3db100b934f7b3473fa340f1c368f510f55071f7f79d458264a6d3aac5209 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv +d3ebd523ddf95f021d98c043d290670b20d8c0ffc2fdbca27fc5740ae46981f5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv +264cd59b0b0b9797f5054f8527e13412a9c9b5fd3a4608d3f81b228a6c08d0bc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json +6b2fad699b75bf96907cf6408da1cb5c34ba7429da791df8f5c55ab24a949240 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/stderr.log +05475a151dca4d87d060c2714d7bd8837964d77b1e4ba335f1363b38a74e3a01 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-po/runs/tp8_ep8_mns64/eval/stdout.log +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/138b2eb7_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/479e1bac_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/5ae40466_model_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/6e9cd646_model_lock.file +a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_138b2eb7_predictions.csv +f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_138b2eb7_predictions.pkl +628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_479e1bac_predictions.csv +7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_479e1bac_predictions.pkl +7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_5ae40466.pkl +125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/cc-cache/all_reduce_6e9cd646.pkl +42420125d3a8ce4aed8dd1c859d39ff840b4e77a52ad5752de73d00a0aed5c3b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/frontier_surface.json +a222ec73d311c5d4e57b0ed3ed2054cb83ec31ff78031eb2b8bde6031472f9ca /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/command.json +9ff45df909e4f94da9435e85d893c0582fbb57ecc28ab15dddadce26604c0f1b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv +12984c1c3ac1ab1d8fb35e7bcbacb147d827cdba43e934fdc757ee8ea7baf6fb /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv +343388af242d66c2e8e46a4462dd23862d6437725c80961588fa6cdc328acdec /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json +b64186eaaa0f2fa566d181b40ed42cd9303b8131ce6fd7c4ae3d77855533a645 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/stderr.log +ece0b457d6c6fb30280d003e209912b918d06abc5e0781296ce29c6855542ed5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns128/eval/stdout.log +837819c6bcb470bc6067a3e5b24713166e4e3b1164090793690874597ea60e03 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/command.json +8db5d447bfc3506e4cf76a511a5a4760c567e2303240f878212aad44ff6f7cdc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv +d244788351f45f3473a8a2705b232e1a951485b5a24e09029331f42eb47bbafc /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv +5795a7d8f48ea0cde14f5c724d5a77d0903398a75151a95c39ad99c2ba2bfb68 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json +5baba11faf64f5c9e4ebbd123aa745c46d73c3f0f81cdd0a16249d212020d443 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/stderr.log +4d70795d321dff067aca5e9ead098726df129d556ceca8eb437d63cefa0c900c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp4_ep1_mns64/eval/stdout.log +40b8d7eb4a946406f37532bb308514bc4982088d872374d078635ce6ef764001 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/command.json +658539186152881399746213af4e3b041b5589cf3a8559346a109d444149767b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv +00e4ed87800ce6054cf4818a4bac9b5db72c5e6dff52ff57d3b7b2489e97ffda /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv +8947f894df28ac9f9cec7be09e79767a685f2ef96835e495d78eadcf5092633a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json +04134b8170da075e36d33766ea9d520dd4db57abcf188116e05aaee1d125f8f3 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/stderr.log +bf6e1295c814c455d1c0b9f02ba0669636b58475efe8873907c6f53677e75fce /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns128/eval/stdout.log +8719be2b20a1d0b35fc5bf23f398801737725527e6ca502a6f343566741517ca /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/command.json +5280da88ab96a0da6e54dc6dcf17d783822b8a49387a696e79830dfd7e81d2d2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv +20adfc333507bbe98f607b1075b0f790e4651ad3185f96906009f6177839de96 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv +1b8a2c5be9bb1c5ab551768a9bbe1c95a6c3f93c9f2b1cb6e1a53a86d3ffb239 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json +683afcccfbcfe232313965d985217bbc8acce32426bc31df749c6b8177cee8a9 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/stderr.log +02b34698d18bb18b46e4792b6208c6277249399e80e5f32c42a112f2fdc4a0e2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-pd/runs/tp8_ep8_mns64/eval/stdout.log +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/138b2eb7_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/479e1bac_prediction_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/5ae40466_model_lock.file +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/6e9cd646_model_lock.file +a6ad892511f6689578fca7b77a971dcfa16b11b2af54a2aaa90b98fa29643776 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_138b2eb7_predictions.csv +f38672b7f93f91f2cf83996389fca9f659fe211df0853044dfa9cafa75b296c2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_138b2eb7_predictions.pkl +628f44a6a1c0f7c2ea121df3ef3a388e997581a998b97933d81be2577b0333f8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_479e1bac_predictions.csv +7dcc465e6b8e39b157bb7a2838b8385ef98f49eb34cffa9832afc2ab8dc6a785 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_479e1bac_predictions.pkl +7fe2a18a04257bb047feb3a5abe09a299e24e454f567ec54b213ba5722be5f5f /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_5ae40466.pkl +125f0eda94a6a39cf3d47ec3c99a9d57180de075ec49d85348eecbd19010a5aa /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/cc-cache/all_reduce_6e9cd646.pkl +aafb5ba279ba8d0f693e12778851c563de4702ffba51e908324ad0e613468d3c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/frontier_surface.json +1ae2c8f734d6bde7740d06953f21c69af00b6307bcb8fc15b3a6a9415e6d29d8 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/command.json +b8f251e1ae5034352f35d61cbfdcad77c48ff978452b43092102ad785dc59930 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/op_precision_metadata.csv +c164e64b1f0b2eb009185b07247d7ce3abbe4baf377ad62ef071a4767395b704 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/request_metrics.csv +3b65bd4a043ab4c395b988fa94172123e851638b1f250ca8761f8725d1cf8cf5 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns128_eval/system_metrics.json +2b61dc88137c817f96e7dcd45c65f451b8031647e56c6f8e554c9b91aee11982 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/stderr.log +0fb063af9bf0cc706e42f12ac1883a309bb4e7ca41924dac8b9ae8d9d580b223 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns128/eval/stdout.log +08e6f686731ff6ceda8577c088ade825182777cff2528e207b444f2fe575f7ef /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/command.json +2f870b68c0820e241e43ebe435d301c88e2abf68bf9e29ff22c08baf39aeb574 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/op_precision_metadata.csv +9349cb9f1ab478343b1b8d079cc175b50328b7c0bffb37018b3a7dc3e208c915 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/request_metrics.csv +141593e073d0247455c73de04da5d82e918e3bf4128c225153a3e85faf63420c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp4_ep1_mns64_eval/system_metrics.json +c6faee6e21ba387608c5387e1a05eae3158a6edda0c530c38d4d8664b5e34a3c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/stderr.log +cd8ee4fb98de672f9b7550dc39e0bf3e80a44c41c65f80c2f5192a9083d1b0cb /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp4_ep1_mns64/eval/stdout.log +961af5f501fcc9606d0ba69924aa50b06e8030dc4239091216780e036702c95d /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/command.json +d9330a115bd2bf15f84986c44fdb35aecf67b8f8138807ed7c2256f36e1a6543 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/op_precision_metadata.csv +6c6f40eba2795ef41f027c7412134694a91baa484933afc9b300803412d376cf /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/request_metrics.csv +d7a7c8fe3fdcfb5d33589cf4ac1706e9fc011c95302180d6d7a1b874f6fd19f4 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns128_eval/system_metrics.json +8aed33f0fc90d82c63f0a3c17dc0ef37ffd555046a4155f6333fca8ca4250df0 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/stderr.log +756930da43bef6ec443ebe6d6360ad15069a3a43cc0c0c6ef07d2795b83cfdb2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns128/eval/stdout.log +3dad31841fddfff0dc4c8dec00f78f9ffc75de82fc94eec50a354ee64902aa6d /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/command.json +b0437765d7f3c27102ea54833ff86d9c14c4f2b76185e770cf1276a53edb5669 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/config.json +9d2e51495987994f2ef7f8308141f893d96a96b1a6268b29e6799643265bcb02 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/op_precision_metadata.csv +015d5b1b48dbe1bb0de1a8a3265a1292e75af1f58d5ac076486e0b85fb133968 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/request_metrics.csv +c0cb828fcaf067076c82a68eeb4c180e5e38d6f0f43dfcf666b0a4515b78a70c /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/metrics/qwen3_235b_a22b/online_serving/qwen235_v020_tp8_ep8_mns64_eval/system_metrics.json +8e4de34f2714e2b0cc1b7a7cc7e657662c3f99231adcea922dca73503a1d9ca2 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/result.json +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/stderr.log +336c646cf9d88d4059f6a4584dab312a2e55401a808627fa8011280c7eb39c3a /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/trace-po/runs/tp8_ep8_mns64/eval/stdout.log diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/frontier.commit b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/frontier.commit new file mode 100644 index 0000000..896d98d --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/frontier.commit @@ -0,0 +1 @@ +6e8e0d845bceff11b0b62cb29df3a1a93411fdd4 diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/input.sha256 b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/input.sha256 new file mode 100644 index 0000000..167bf99 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1-prov/input.sha256 @@ -0,0 +1,4 @@ +8eadf8b39becfcbe103c6328ca616902689e9694cc6cb6c1f08609f0e758ec41 /home/admin/cpfs/wjh/aituner/aituner-q235-collective-20260719/runs/frontier-fidelity-envelope-v1/run_qwen235_collective_profile_ablation.sh +30bdc9da5c8e638d6278b3ae3f812613a1f7ef296439e34d28dcb72ae30b0d1b /home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/profiles/qwen235-serving-allreduce.csv +5aa8657c5271abec6fca78a9f8fa2cfc0696f24c61033a850d7289f53f48632e /home/admin/cpfs/wjh/aituner/qwen235-v020-profiles-20260719-r2/frozen/manifest.json +99befe893683e3a638ea28048d00fa6c7b3d6a209921fe19ad51e78dfbc39459 /home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/provenance/runtime-contract.json diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.json new file mode 100644 index 0000000..57dc08b --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.json @@ -0,0 +1,799 @@ +{ + "cases": { + "fixed-pd": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.3067524317507717, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.3456375801249614, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "tpot_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.33025510615656906, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "tpot_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.37228890038064755, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "ttft_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + }, + "ttft_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns64", + "reversals": [], + "selected_real_regret": 0.002088257708717345, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 5729.31677040774, + "e2e_p90_ms": 5871.668232022785, + "request_samples": 387, + "tpot_mean_ms": 21.044200828709027, + "tpot_p90_ms": 21.59612646244247, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 363.00995954954925, + "ttft_p90_ms": 369.09779999405146 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 5769.497427325912, + "e2e_p90_ms": 5888.945628073998, + "request_samples": 387, + "tpot_mean_ms": 21.187139407074238, + "tpot_p90_ms": 21.653656356547977, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 366.7432916322386, + "ttft_p90_ms": 369.99153497163206 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 7486.7986220007915, + "e2e_p90_ms": 7901.13743103575, + "request_samples": 387, + "tpot_mean_ms": 27.994155607374484, + "tpot_p90_ms": 29.63612463562658, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 348.25438050341427, + "ttft_p90_ms": 365.8816539682448 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 7484.121415326708, + "e2e_p90_ms": 7908.651847974397, + "request_samples": 387, + "tpot_mean_ms": 27.978544004773987, + "tpot_p90_ms": 29.650389521346227, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 349.5577253122725, + "ttft_p90_ms": 365.11919100303203 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 19261.684938205322, + "e2e_p50_ms": 19681.421068277567, + "e2e_p90_ms": 19712.678973231064, + "e2e_p95_ms": 19717.18006692948, + "tpot_mean_ms": 74.22292738142716, + "tpot_p50_ms": 75.8976524911963, + "tpot_p90_ms": 75.8976524912086, + "tpot_p95_ms": 75.89765249120967, + "ttft_mean_ms": 334.8384559413977, + "ttft_p50_ms": 334.75193988430976, + "ttft_p90_ms": 359.7779859328796, + "ttft_p95_ms": 363.2786816744229 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 19261.684938205322, + "e2e_p50_ms": 19681.421068277567, + "e2e_p90_ms": 19712.678973231064, + "e2e_p95_ms": 19717.18006692948, + "tpot_mean_ms": 74.22292738142716, + "tpot_p50_ms": 75.8976524911963, + "tpot_p90_ms": 75.8976524912086, + "tpot_p95_ms": 75.89765249120967, + "ttft_mean_ms": 334.8384559413977, + "ttft_p50_ms": 334.75193988430976, + "ttft_p90_ms": 359.7779859328796, + "ttft_p95_ms": 363.2786816744229 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 14056.464119438264, + "e2e_p50_ms": 14650.653983203298, + "e2e_p90_ms": 14674.997135818061, + "e2e_p95_ms": 14677.859760866291, + "tpot_mean_ms": 54.15667269956665, + "tpot_p50_ms": 56.52350191837838, + "tpot_p90_ms": 56.52350850045728, + "tpot_p95_ms": 56.523546100039816, + "ttft_mean_ms": 246.51258104876885, + "ttft_p50_ms": 246.7781280760306, + "ttft_p90_ms": 263.0087113349084, + "ttft_p95_ms": 264.77576248352364 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 14056.464119438264, + "e2e_p50_ms": 14650.653983203298, + "e2e_p90_ms": 14674.997135818061, + "e2e_p95_ms": 14677.859760866291, + "tpot_mean_ms": 54.15667269956665, + "tpot_p50_ms": 56.52350191837838, + "tpot_p90_ms": 56.52350850045728, + "tpot_p95_ms": 56.523546100039816, + "ttft_mean_ms": 246.51258104876885, + "ttft_p50_ms": 246.7781280760306, + "ttft_p90_ms": 263.0087113349084, + "ttft_p95_ms": 264.77576248352364 + } + } + }, + "fixed-po": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + }, + "e2e_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + }, + "ttft_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + }, + "ttft_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 345.40159726161653, + "e2e_p90_ms": 346.7378600034863, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 345.35322551680514, + "ttft_p90_ms": 346.6896569589153 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 345.8011649264437, + "e2e_p90_ms": 347.9283060878515, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 345.75379653310387, + "ttft_p90_ms": 347.88418107200414 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 326.2389304512842, + "e2e_p90_ms": 341.0220619989559, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 326.19059710668336, + "ttft_p90_ms": 340.98013502079993 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 326.89427194042366, + "e2e_p90_ms": 341.91650594584644, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 326.846390089937, + "ttft_p90_ms": 341.87235799618065 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 303.2842141052072, + "e2e_p50_ms": 303.28421410520434, + "e2e_p90_ms": 303.28421410521855, + "e2e_p95_ms": 303.28421410521855, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 303.2842141052072, + "ttft_p50_ms": 303.28421410520434, + "ttft_p90_ms": 303.28421410521855, + "ttft_p95_ms": 303.28421410521855 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 303.2842141052072, + "e2e_p50_ms": 303.28421410520434, + "e2e_p90_ms": 303.28421410521855, + "e2e_p95_ms": 303.28421410521855, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 303.2842141052072, + "ttft_p50_ms": 303.28421410520434, + "ttft_p90_ms": 303.28421410521855, + "ttft_p95_ms": 303.28421410521855 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 224.0004422359394, + "e2e_p50_ms": 224.00044223590854, + "e2e_p90_ms": 224.00044223607017, + "e2e_p95_ms": 224.00044223607017, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 224.0004422359394, + "ttft_p50_ms": 224.00044223590854, + "ttft_p90_ms": 224.00044223607017, + "ttft_p95_ms": 224.00044223607017 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 224.0004422359394, + "e2e_p50_ms": 224.00044223590854, + "e2e_p90_ms": 224.00044223607017, + "e2e_p95_ms": 224.00044223607017, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 224.0004422359394, + "ttft_p50_ms": 224.00044223590854, + "ttft_p90_ms": 224.00044223607017, + "ttft_p95_ms": 224.00044223607017 + } + } + }, + "trace-pd": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.8333333333333334, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.006154214824712234, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.8333333333333334, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.062164651568153984, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "tpot_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "winner_match": true + }, + "tpot_p90_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "winner_match": true + }, + "ttft_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + }, + "ttft_p90_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 203026.22048668287, + "e2e_p90_ms": 344183.7428959552, + "request_samples": 387, + "tpot_mean_ms": 58.09653520572589, + "tpot_p90_ms": 75.82137218925735, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 26487.547554662306, + "ttft_p90_ms": 93044.83354603872 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 198593.25371425177, + "e2e_p90_ms": 332386.17597706616, + "request_samples": 387, + "tpot_mean_ms": 53.60856802543236, + "tpot_p90_ms": 64.87020396877685, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 31557.98602907994, + "ttft_p90_ms": 112666.82129900437 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 182457.35910624202, + "e2e_p90_ms": 318000.89644105174, + "request_samples": 387, + "tpot_mean_ms": 61.88514020253078, + "tpot_p90_ms": 78.6175951982538, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 1141.539354895564, + "ttft_p90_ms": 2616.6939180111513 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 181341.34550937495, + "e2e_p90_ms": 299389.4552709535, + "request_samples": 387, + "tpot_mean_ms": 45.89371812653489, + "tpot_p90_ms": 51.37046630467056, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 37215.26084531421, + "ttft_p90_ms": 132333.42359995004 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 350485.31604335306, + "e2e_p50_ms": 329886.2419457557, + "e2e_p90_ms": 618683.6003279199, + "e2e_p95_ms": 719028.7082294676, + "tpot_mean_ms": 92.88846066050185, + "tpot_p50_ms": 91.14780565610876, + "tpot_p90_ms": 104.48481783284998, + "tpot_p95_ms": 117.7639165127043, + "ttft_mean_ms": 51997.2508135414, + "ttft_p50_ms": 915.3872825012286, + "ttft_p90_ms": 175964.46056759413, + "ttft_p95_ms": 254194.45368846075 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 350205.0884431818, + "e2e_p50_ms": 345577.391630324, + "e2e_p90_ms": 612408.2764512596, + "e2e_p95_ms": 709831.5675796239, + "tpot_mean_ms": 86.77595832694777, + "tpot_p50_ms": 87.01966249041537, + "tpot_p90_ms": 95.27552612769624, + "tpot_p95_ms": 98.22236811591964, + "ttft_mean_ms": 64276.51238724817, + "ttft_p50_ms": 2913.567965463905, + "ttft_p90_ms": 231348.6370051864, + "ttft_p95_ms": 250924.94176651328 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 197537.45276613487, + "e2e_p50_ms": 184538.43666142275, + "e2e_p90_ms": 356508.87615749374, + "e2e_p95_ms": 426676.71753314673, + "tpot_mean_ms": 62.78094439594437, + "tpot_p50_ms": 61.98322714624318, + "tpot_p90_ms": 73.38270919472093, + "tpot_p95_ms": 79.82017533221833, + "ttft_mean_ms": 496.4562724801789, + "ttft_p50_ms": 258.18551545088656, + "ttft_p90_ms": 1185.2339118913733, + "ttft_p95_ms": 1563.6528594879735 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 230232.00122833299, + "e2e_p50_ms": 225231.54242404614, + "e2e_p90_ms": 399325.5400221139, + "e2e_p95_ms": 451116.17307457025, + "tpot_mean_ms": 56.01681518539994, + "tpot_p50_ms": 56.94928232211729, + "tpot_p90_ms": 60.38839798597023, + "tpot_p95_ms": 62.67163079918898, + "ttft_mean_ms": 46908.98176860714, + "ttft_p50_ms": 1623.170360660838, + "ttft_p90_ms": 166051.82069777363, + "ttft_p95_ms": 181013.0283352888 + } + } + }, + "trace-po": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.0137482658615673, + "sim_winner": "tp8_ep8_mns64", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns64", + "reversals": [], + "selected_real_regret": 0.0025194410024500513, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "ttft_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.013750984200462835, + "sim_winner": "tp8_ep8_mns64", + "winner_match": false + }, + "ttft_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns64", + "reversals": [], + "selected_real_regret": 0.002607797240047338, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 1083.6389664896267, + "e2e_p90_ms": 3156.966591021046, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 1083.561832460368, + "ttft_p90_ms": 3156.9307829486206 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 1102.0341126990652, + "e2e_p90_ms": 3162.989347940311, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 1101.9558890756384, + "ttft_p90_ms": 3162.9534389358014 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 1029.581229927413, + "e2e_p90_ms": 2612.0267439400777, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 1029.4865175674956, + "ttft_p90_ms": 2611.9744409807026 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 1043.7361864025345, + "e2e_p90_ms": 2605.4624350508675, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 1043.6429704051557, + "ttft_p90_ms": 2605.180658050813 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 698.2455408721717, + "e2e_p50_ms": 208.8217362068434, + "e2e_p90_ms": 2102.42229741975, + "e2e_p95_ms": 2868.6579248910534, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 698.2455408721717, + "ttft_p50_ms": 208.8217362068434, + "ttft_p90_ms": 2102.42229741975, + "ttft_p95_ms": 2868.6579248910534 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 698.0132356513792, + "e2e_p50_ms": 208.8217362068434, + "e2e_p90_ms": 2102.42229741975, + "e2e_p95_ms": 2868.6579248910534, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 698.0132356513792, + "ttft_p50_ms": 208.8217362068434, + "ttft_p90_ms": 2102.42229741975, + "ttft_p95_ms": 2868.6579248910534 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 487.42679595918156, + "e2e_p50_ms": 253.39288448911512, + "e2e_p90_ms": 1264.1466431938486, + "e2e_p95_ms": 1656.1837032209396, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 487.42679595918156, + "ttft_p50_ms": 253.39288448911512, + "ttft_p90_ms": 1264.1466431938486, + "ttft_p95_ms": 1656.1837032209396 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 487.3699604726311, + "e2e_p50_ms": 253.41330144894124, + "e2e_p90_ms": 1264.1466431938486, + "e2e_p95_ms": 1656.1837032209396, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 487.3699604726311, + "ttft_p50_ms": 253.41330144894124, + "ttft_p90_ms": 1264.1466431938486, + "ttft_p95_ms": 1656.1837032209396 + } + } + } + }, + "real_root": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1", + "schema": "qwen235-v020-simulator-real-comparison-v1", + "sim_root": "/home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.md new file mode 100644 index 0000000..fc3325a --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.md @@ -0,0 +1,24 @@ +# Qwen235 vLLM 0.20 Frontier vs real + +| case | metric | Frontier winner | real winner | match | regret | pair agreement | +|---|---|---|---|---:|---:|---:| +| fixed-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% | +| fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.2% | 100.0% | +| fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 33.0% | 0.0% | +| fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 37.2% | 0.0% | +| fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 30.7% | 0.0% | +| fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 34.6% | 0.0% | +| fixed-po | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% | +| fixed-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% | +| fixed-po | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% | +| fixed-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 100.0% | +| trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% | +| trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% | +| trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% | +| trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% | +| trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.6% | 83.3% | +| trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 6.2% | 83.3% | +| trace-po | ttft_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | no | 1.4% | 66.7% | +| trace-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.3% | 100.0% | +| trace-po | e2e_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | no | 1.4% | 66.7% | +| trace-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.3% | 100.0% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.json new file mode 100644 index 0000000..fb5ede7 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.json @@ -0,0 +1,952 @@ +{ + "cases": { + "fixed-pd": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.3067524317507717, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.3456375801249614, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "tpot_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.33025510615656906, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "tpot_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp4_ep1_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.37228890038064755, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "ttft_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.04237011756982145, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + }, + "ttft_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 1.0, + "real_winner": "tp8_ep8_mns64", + "reversals": [], + "selected_real_regret": 0.002088257708717345, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 5729.31677040774, + "e2e_p90_ms": 5871.668232022785, + "request_samples": 387, + "tpot_mean_ms": 21.044200828709027, + "tpot_p90_ms": 21.59612646244247, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 363.00995954954925, + "ttft_p90_ms": 369.09779999405146 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 5769.497427325912, + "e2e_p90_ms": 5888.945628073998, + "request_samples": 387, + "tpot_mean_ms": 21.187139407074238, + "tpot_p90_ms": 21.653656356547977, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 366.7432916322386, + "ttft_p90_ms": 369.99153497163206 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 7486.7986220007915, + "e2e_p90_ms": 7901.13743103575, + "request_samples": 387, + "tpot_mean_ms": 27.994155607374484, + "tpot_p90_ms": 29.63612463562658, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 348.25438050341427, + "ttft_p90_ms": 365.8816539682448 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 7484.121415326708, + "e2e_p90_ms": 7908.651847974397, + "request_samples": 387, + "tpot_mean_ms": 27.978544004773987, + "tpot_p90_ms": 29.650389521346227, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 349.5577253122725, + "ttft_p90_ms": 365.11919100303203 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 22734.095381397616, + "e2e_p50_ms": 23441.595611495926, + "e2e_p90_ms": 23484.306701188103, + "e2e_p95_ms": 23489.511452611594, + "tpot_mean_ms": 87.77349881741111, + "tpot_p50_ms": 90.6067822139126, + "tpot_p90_ms": 90.60678221391389, + "tpot_p95_ms": 90.60678221391389, + "ttft_mean_ms": 351.8531829577845, + "ttft_p50_ms": 351.44748294051453, + "ttft_p90_ms": 381.14669406792245, + "ttft_p95_ms": 385.3185272085611 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 22734.095381397616, + "e2e_p50_ms": 23441.595611495926, + "e2e_p90_ms": 23484.306701188103, + "e2e_p95_ms": 23489.511452611594, + "tpot_mean_ms": 87.77349881741111, + "tpot_p50_ms": 90.6067822139126, + "tpot_p90_ms": 90.60678221391389, + "tpot_p95_ms": 90.60678221391389, + "ttft_mean_ms": 351.8531829577845, + "ttft_p50_ms": 351.44748294051453, + "ttft_p90_ms": 381.14669406792245, + "ttft_p95_ms": 385.3185272085611 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 16059.899811799207, + "e2e_p50_ms": 17256.45308650381, + "e2e_p90_ms": 17280.553823012277, + "e2e_p95_ms": 17282.746891876577, + "tpot_mean_ms": 61.58781555229474, + "tpot_p50_ms": 66.32324900760649, + "tpot_p90_ms": 66.3246446898248, + "tpot_p95_ms": 66.3246446898248, + "ttft_mean_ms": 355.00684596404665, + "ttft_p50_ms": 355.16695480338, + "ttft_p90_ms": 368.8582878708644, + "ttft_p95_ms": 370.503089520021 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 16059.899811799207, + "e2e_p50_ms": 17256.45308650381, + "e2e_p90_ms": 17280.553823012277, + "e2e_p95_ms": 17282.746891876577, + "tpot_mean_ms": 61.58781555229474, + "tpot_p50_ms": 66.32324900760649, + "tpot_p90_ms": 66.3246446898248, + "tpot_p95_ms": 66.3246446898248, + "ttft_mean_ms": 355.00684596404665, + "ttft_p50_ms": 355.16695480338, + "ttft_p90_ms": 368.8582878708644, + "ttft_p95_ms": 370.503089520021 + } + } + }, + "fixed-po": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.05873813644442971, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.016760786592592503, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + }, + "ttft_mean_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.058746722254088946, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + }, + "ttft_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.016744441542810407, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 345.40159726161653, + "e2e_p90_ms": 346.7378600034863, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 345.35322551680514, + "ttft_p90_ms": 346.6896569589153 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 345.8011649264437, + "e2e_p90_ms": 347.9283060878515, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 345.75379653310387, + "ttft_p90_ms": 347.88418107200414 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 326.2389304512842, + "e2e_p90_ms": 341.0220619989559, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 326.19059710668336, + "ttft_p90_ms": 340.98013502079993 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 326.89427194042366, + "e2e_p90_ms": 341.91650594584644, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-po/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 326.846390089937, + "ttft_p90_ms": 341.87235799618065 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 313.84871569626137, + "e2e_p50_ms": 313.848715696261, + "e2e_p90_ms": 313.8487156962668, + "e2e_p95_ms": 313.8487156962668, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 313.84871569626137, + "ttft_p50_ms": 313.848715696261, + "ttft_p90_ms": 313.8487156962668, + "ttft_p95_ms": 313.8487156962668 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 313.84871569626137, + "e2e_p50_ms": 313.848715696261, + "e2e_p90_ms": 313.8487156962668, + "e2e_p95_ms": 313.8487156962668, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 313.84871569626137, + "ttft_p50_ms": 313.848715696261, + "ttft_p90_ms": 313.8487156962668, + "ttft_p95_ms": 313.8487156962668 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 334.2963928223074, + "e2e_p50_ms": 334.2963928222744, + "e2e_p90_ms": 334.2963928224325, + "e2e_p95_ms": 334.2963928224325, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 334.2963928223074, + "ttft_p50_ms": 334.2963928222744, + "ttft_p90_ms": 334.2963928224325, + "ttft_p95_ms": 334.2963928224325 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 334.2963928223074, + "e2e_p50_ms": 334.2963928222744, + "e2e_p90_ms": 334.2963928224325, + "e2e_p95_ms": 334.2963928224325, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 334.2963928223074, + "ttft_p50_ms": 334.2963928222744, + "ttft_p90_ms": 334.2963928224325, + "ttft_p95_ms": 334.2963928224325 + } + } + }, + "trace-pd": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.8333333333333334, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.006154214824712234, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.062164651568153984, + "sim_winner": "tp8_ep8_mns128", + "winner_match": false + }, + "tpot_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "winner_match": true + }, + "tpot_p90_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "winner_match": true + }, + "ttft_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + }, + "ttft_p90_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.6666666666666666, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ] + ], + "selected_real_regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "winner_match": true + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 203026.22048668287, + "e2e_p90_ms": 344183.7428959552, + "request_samples": 387, + "tpot_mean_ms": 58.09653520572589, + "tpot_p90_ms": 75.82137218925735, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 26487.547554662306, + "ttft_p90_ms": 93044.83354603872 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 198593.25371425177, + "e2e_p90_ms": 332386.17597706616, + "request_samples": 387, + "tpot_mean_ms": 53.60856802543236, + "tpot_p90_ms": 64.87020396877685, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 31557.98602907994, + "ttft_p90_ms": 112666.82129900437 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 182457.35910624202, + "e2e_p90_ms": 318000.89644105174, + "request_samples": 387, + "tpot_mean_ms": 61.88514020253078, + "tpot_p90_ms": 78.6175951982538, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 1141.539354895564, + "ttft_p90_ms": 2616.6939180111513 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 181341.34550937495, + "e2e_p90_ms": 299389.4552709535, + "request_samples": 387, + "tpot_mean_ms": 45.89371812653489, + "tpot_p90_ms": 51.37046630467056, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-pd/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 37215.26084531421, + "ttft_p90_ms": 132333.42359995004 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 385157.8895230749, + "e2e_p50_ms": 367024.8243163692, + "e2e_p90_ms": 673951.1520207045, + "e2e_p95_ms": 785718.9717121953, + "tpot_mean_ms": 100.38179934319204, + "tpot_p50_ms": 100.42433178975269, + "tpot_p90_ms": 113.67517808832463, + "tpot_p95_ms": 121.63558651348922, + "ttft_mean_ms": 59067.296381594686, + "ttft_p50_ms": 908.4160127742393, + "ttft_p90_ms": 200789.2407497713, + "ttft_p95_ms": 289881.2920798696 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 380604.1804371099, + "e2e_p50_ms": 378158.20614125655, + "e2e_p90_ms": 678448.1689084907, + "e2e_p95_ms": 763969.1713963022, + "tpot_mean_ms": 93.54579486627527, + "tpot_p50_ms": 92.86650455014843, + "tpot_p90_ms": 102.66261969117096, + "tpot_p95_ms": 112.87824516729198, + "ttft_mean_ms": 72259.33174260205, + "ttft_p50_ms": 2897.238841988653, + "ttft_p90_ms": 265395.14114254655, + "ttft_p95_ms": 273163.7753733101 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 174939.1805624549, + "e2e_p50_ms": 166723.75338099696, + "e2e_p90_ms": 317439.7347771481, + "e2e_p95_ms": 371471.8509246631, + "tpot_mean_ms": 58.37204729277776, + "tpot_p50_ms": 54.29394782230334, + "tpot_p90_ms": 75.91440592454555, + "tpot_p95_ms": 84.58610544605006, + "ttft_mean_ms": 945.9090659083846, + "ttft_p50_ms": 669.9533543429972, + "ttft_p90_ms": 2348.607298432925, + "ttft_p95_ms": 2781.46390848989 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 208352.4497977196, + "e2e_p50_ms": 207573.22184026084, + "e2e_p90_ms": 353236.8601573669, + "e2e_p95_ms": 402705.2991150841, + "tpot_mean_ms": 52.06723645713691, + "tpot_p50_ms": 51.64653531173072, + "tpot_p90_ms": 60.374786839964116, + "tpot_p95_ms": 62.94158800335912, + "ttft_mean_ms": 43679.663681651284, + "ttft_p50_ms": 2772.0384805371, + "ttft_p90_ms": 151027.16878398336, + "ttft_p95_ms": 165014.86227423933 + } + } + }, + "trace-po": { + "comparison": { + "e2e_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ], + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.07037121566091509, + "sim_winner": "tp4_ep1_mns64", + "winner_match": false + }, + "e2e_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.21167227304868486, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + }, + "ttft_mean_ms": { + "informative_pairs": 6, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns128", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ], + [ + "tp8_ep8_mns64", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.07039370625209918, + "sim_winner": "tp4_ep1_mns64", + "winner_match": false + }, + "ttft_p90_ms": { + "informative_pairs": 4, + "pair_direction_agreement": 0.0, + "real_winner": "tp8_ep8_mns64", + "reversals": [ + [ + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns64" + ], + [ + "tp4_ep1_mns128", + "tp8_ep8_mns128" + ] + ], + "selected_real_regret": 0.2117895828808376, + "sim_winner": "tp4_ep1_mns128", + "winner_match": false + } + }, + "real": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 1083.6389664896267, + "e2e_p90_ms": 3156.966591021046, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 1083.561832460368, + "ttft_p90_ms": 3156.9307829486206 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 1102.0341126990652, + "e2e_p90_ms": 3162.989347940311, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp4_ep1_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 1101.9558890756384, + "ttft_p90_ms": 3162.9534389358014 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 1029.581229927413, + "e2e_p90_ms": 2612.0267439400777, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns128/trial3/results/result.json" + ], + "ttft_mean_ms": 1029.4865175674956, + "ttft_p90_ms": 2611.9744409807026 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 1043.7361864025345, + "e2e_p90_ms": 2605.4624350508675, + "request_samples": 387, + "tpot_mean_ms": null, + "tpot_p90_ms": null, + "trials": [ + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial1/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial2/results/result.json", + "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/trace-po/real/tp8_ep8_mns64/trial3/results/result.json" + ], + "ttft_mean_ms": 1043.6429704051557, + "ttft_p90_ms": 2605.180658050813 + } + }, + "sim": { + "tp4_ep1_mns128": { + "e2e_mean_ms": 691.7873605232029, + "e2e_p50_ms": 211.81272326305088, + "e2e_p90_ms": 2068.417017674378, + "e2e_p95_ms": 2843.8890088954663, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 691.7873605232029, + "ttft_p50_ms": 211.81272326305088, + "ttft_p90_ms": 2068.417017674378, + "ttft_p95_ms": 2843.8890088954663 + }, + "tp4_ep1_mns64": { + "e2e_mean_ms": 691.5722223601863, + "e2e_p50_ms": 210.89579132517144, + "e2e_p90_ms": 2068.417017674378, + "e2e_p95_ms": 2843.8890088954663, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 691.5722223601863, + "ttft_p50_ms": 210.89579132517144, + "ttft_p90_ms": 2068.417017674378, + "ttft_p95_ms": 2843.8890088954663 + }, + "tp8_ep8_mns128": { + "e2e_mean_ms": 928.7998403823643, + "e2e_p50_ms": 593.0899264387222, + "e2e_p90_ms": 2341.4108333476474, + "e2e_p95_ms": 2898.493085937801, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 928.7998403823643, + "ttft_p50_ms": 593.0899264387222, + "ttft_p90_ms": 2341.4108333476474, + "ttft_p95_ms": 2898.493085937801 + }, + "tp8_ep8_mns64": { + "e2e_mean_ms": 928.7182564423873, + "e2e_p50_ms": 593.0899264387222, + "e2e_p90_ms": 2341.4108333476474, + "e2e_p95_ms": 2898.493085937801, + "tpot_mean_ms": null, + "tpot_p50_ms": null, + "tpot_p90_ms": null, + "tpot_p95_ms": null, + "ttft_mean_ms": 928.7182564423873, + "ttft_p50_ms": 593.0899264387222, + "ttft_p90_ms": 2341.4108333476474, + "ttft_p95_ms": 2898.493085937801 + } + } + } + }, + "schema": "qwen235-v020-simulator-real-comparison-v1" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.md new file mode 100644 index 0000000..1e05500 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.md @@ -0,0 +1,24 @@ +# Qwen235 vLLM 0.20 Frontier vs real + +| case | metric | Frontier winner | real winner | match | regret | pair agreement | +|---|---|---|---|---:|---:|---:| +| fixed-pd | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 4.2% | 0.0% | +| fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.2% | 100.0% | +| fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 33.0% | 0.0% | +| fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 37.2% | 0.0% | +| fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 30.7% | 0.0% | +| fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 34.6% | 0.0% | +| fixed-po | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 5.9% | 0.0% | +| fixed-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 1.7% | 0.0% | +| fixed-po | e2e_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 5.9% | 0.0% | +| fixed-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 1.7% | 0.0% | +| trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% | +| trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% | +| trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% | +| trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% | +| trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.6% | 83.3% | +| trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 6.2% | 66.7% | +| trace-po | ttft_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | no | 7.0% | 0.0% | +| trace-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | no | 21.2% | 0.0% | +| trace-po | e2e_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | no | 7.0% | 0.0% | +| trace-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | no | 21.2% | 0.0% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-exact.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-exact.json new file mode 100644 index 0000000..0a6061f --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-exact.json @@ -0,0 +1,1587 @@ +{ + "contrast_decomposition": { + "exact_state_matched_tp8_minus_tp4_ms": 10.902788634037428, + "residual_vs_exact_iteration_ms": 1.7850640558340523, + "residual_vs_frozen_real_tpot_ms": 3.9527886340374287, + "simulator_internal_tp8_minus_tp4_ms": -20.06625468177714, + "state_composition_shift_ms": 30.969043315814567 + }, + "exact_state_matched": { + "configs": { + "tp4_ep1_mns64": { + "components_ms": { + "attention": 7.861372580683772, + "dense_mlp_compute": 0.20457670875208997, + "ep_communication": 0.0, + "moe_compute": 11.818199308522054, + "moe_routing": 1.3660523028556315, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 34.25746674314369, + "tp_dp_communication": 13.007265841334672 + }, + "coverage": 1.0, + "supported_decode_tokens": 32895, + "total_decode_tokens": 32895, + "unsupported": {} + }, + "tp8_ep8_mns64": { + "components_ms": { + "attention": 5.730952171784399, + "dense_mlp_compute": 0.17382455581232167, + "ep_communication": 6.535508227248577, + "moe_compute": 9.840752983018692, + "moe_routing": 1.5044582710663996, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 45.160255377181116, + "tp_dp_communication": 6.535508227248577 + }, + "coverage": 1.0, + "supported_decode_tokens": 32895, + "total_decode_tokens": 32895, + "unsupported": {} + } + }, + "method": "decode-token-weighted exact (context_tokens, generation_requests) composition from vLLM iteration details; no interpolation", + "tp8_minus_tp4_ms": { + "attention": -2.130420408899373, + "dense_mlp_compute": -0.030752152939768296, + "ep_communication": 6.535508227248577, + "moe_compute": -1.9774463255033616, + "moe_routing": 0.13840596821076812, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 10.902788634037428, + "tp_dp_communication": -6.471757614086094 + } + }, + "proxy_matched": { + "configs": { + "tp4_ep1_mns64": { + "components_ms": { + "attention": 6.303822560333333, + "dense_mlp_compute": 0.17883562666666666, + "ep_communication": 0.0, + "moe_compute": 26.449277218333332, + "moe_routing": 1.2571787503333332, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 46.414892970000004, + "tp_dp_communication": 12.225778813333333 + }, + "coverage": 1.0, + "supported_samples": 45, + "total_samples": 45, + "unsupported": {} + }, + "tp8_ep8_mns64": { + "components_ms": { + "attention": 6.063134093052075, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777717, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.86318467071363, + "tp_dp_communication": 6.824791048020123 + }, + "coverage": 1.0, + "supported_samples": 21, + "total_samples": 21, + "unsupported": {} + } + }, + "method": "exact batch-size lookup; each simulator config is reweighted to its own frozen-real steady Running histogram; no interpolation", + "tp8_minus_tp4_ms": { + "attention": -0.2406884672812577, + "dense_mlp_compute": 0.0017646933333340664, + "ep_communication": 6.824791048020123, + "moe_compute": -16.058086515555615, + "moe_routing": 0.31092423394602053, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -8.551708299286375, + "tp_dp_communication": -5.40098776531321 + } + }, + "real": { + "tp4_ep1_mns64": { + "aggregate": { + "generation_tokens_per_s": { + "cv": 0.01219623132338527, + "distinct_n": 32, + "max": 217.3, + "mean": 204.7488888888889, + "min": 197.1, + "n": 45 + }, + "kv_percent": { + "cv": 0.09908502666308305, + "distinct_n": 2, + "max": 5.6, + "mean": 5.233333333333333, + "min": 4.5, + "n": 45 + }, + "running": { + "cv": 0.10101525445522107, + "distinct_n": 2, + "max": 5.0, + "mean": 4.666666666666667, + "min": 4.0, + "n": 45 + }, + "running_histogram": { + "4": 15, + "5": 30 + }, + "samples": 45, + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 45 + } + }, + "proxy_only": true, + "steady_rule": "prompt throughput >= 95% of 3276.8 tokens/s", + "trials": [ + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.01635665678820842, + "distinct_n": 13, + "max": 217.3, + "mean": 205.58666666666667, + "min": 202.4, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.6, + "mean": 5.6, + "min": 5.6, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.0, + "mean": 5.0, + "min": 5.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial1", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + }, + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.01101205870299425, + "distinct_n": 11, + "max": 206.3, + "mean": 204.18666666666667, + "min": 197.1, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.6, + "mean": 5.6, + "min": 5.6, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.0, + "mean": 5.0, + "min": 5.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial2", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + }, + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.005467774668645034, + "distinct_n": 14, + "max": 206.0, + "mean": 204.47333333333333, + "min": 202.0, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 4.5, + "mean": 4.5, + "min": 4.5, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 4.0, + "mean": 4.0, + "min": 4.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial3", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + } + ] + }, + "tp8_ep8_mns64": { + "aggregate": { + "generation_tokens_per_s": { + "cv": 0.01855015540813005, + "distinct_n": 21, + "max": 415.7, + "mean": 406.5380952380952, + "min": 390.8, + "n": 21 + }, + "kv_percent": { + "cv": 0.03973151445031385, + "distinct_n": 3, + "max": 5.6, + "mean": 5.295238095238095, + "min": 4.8, + "n": 21 + }, + "running": { + "cv": 0.04297805843263526, + "distinct_n": 3, + "max": 13.0, + "mean": 12.238095238095237, + "min": 11.0, + "n": 21 + }, + "running_histogram": { + "11": 1, + "12": 14, + "13": 6 + }, + "samples": 21, + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 21 + } + }, + "proxy_only": true, + "steady_rule": "prompt throughput >= 95% of 6553.6 tokens/s", + "trials": [ + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.018873398222858283, + "distinct_n": 7, + "max": 415.7, + "mean": 406.87142857142857, + "min": 392.3, + "n": 7 + }, + "kv_percent": { + "cv": 0.03400298559320833, + "distinct_n": 2, + "max": 5.6, + "mean": 5.314285714285715, + "min": 5.2, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/logs/server.log", + "running": { + "cv": 0.036770670467074176, + "distinct_n": 2, + "max": 13.0, + "mean": 12.285714285714286, + "min": 12.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial1", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + }, + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.01869328742146201, + "distinct_n": 7, + "max": 412.9, + "mean": 406.2857142857143, + "min": 390.8, + "n": 7 + }, + "kv_percent": { + "cv": 0.04861017342390845, + "distinct_n": 3, + "max": 5.6, + "mean": 5.257142857142857, + "min": 4.8, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/logs/server.log", + "running": { + "cv": 0.052613364176465637, + "distinct_n": 3, + "max": 13.0, + "mean": 12.142857142857142, + "min": 11.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial2", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + }, + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.018043345382112573, + "distinct_n": 7, + "max": 414.9, + "mean": 406.45714285714286, + "min": 393.0, + "n": 7 + }, + "kv_percent": { + "cv": 0.03400298559320833, + "distinct_n": 2, + "max": 5.6, + "mean": 5.314285714285715, + "min": 5.2, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/logs/server.log", + "running": { + "cv": 0.036770670467074176, + "distinct_n": 2, + "max": 13.0, + "mean": 12.285714285714286, + "min": 12.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial3", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + } + ] + } + }, + "real_iteration_state": { + "tp4_ep1_mns64": { + "decode_batch_size": { + "cv": 0.13029119433004072, + "distinct_n": 5, + "max": 5.0, + "mean": 3.8896771904930825, + "min": 1.0, + "n": 8457 + }, + "decode_bearing_rows": 8457, + "decode_token_weighted_iteration_elapsed_ms": 18.47097461620307, + "decode_token_weighted_joint_state_histogram": { + "0:1": 149, + "0:2": 280, + "0:3": 621, + "0:4": 31012, + "0:5": 400, + "4096:1": 1, + "4096:2": 2, + "4096:3": 222, + "4096:4": 208 + }, + "measured_rows": 8460, + "mixed_iteration_elapsed_ms": { + "cv": 0.015940570375581706, + "distinct_n": 114, + "max": 350.16, + "mean": 337.35875, + "min": 331.36, + "n": 128 + }, + "mixed_prefill_decode_rows": 128, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-fixed-pd-state-diagnosis-20260719-r1/iteration-real/tp4_ep1_mns64/logs/server.log", + "pure_decode_batch_histogram": { + "1": 149, + "2": 140, + "3": 207, + "4": 7753, + "5": 80 + }, + "pure_decode_iteration_elapsed_ms": { + "cv": 0.0683725733639485, + "distinct_n": 353, + "max": 63.66, + "mean": 14.141689278424781, + "min": 0.11, + "n": 8329 + }, + "pure_decode_rows": 8329 + }, + "tp8_ep8_mns64": { + "decode_batch_size": { + "cv": 0.24310653962915885, + "distinct_n": 13, + "max": 13.0, + "mean": 10.921314741035857, + "min": 1.0, + "n": 3012 + }, + "decode_bearing_rows": 3012, + "decode_token_weighted_iteration_elapsed_ms": 27.588699194406445, + "decode_token_weighted_joint_state_histogram": { + "0:1": 49, + "0:10": 700, + "0:11": 2431, + "0:12": 24312, + "0:13": 2067, + "0:2": 96, + "0:3": 138, + "0:4": 196, + "0:5": 215, + "0:6": 270, + "0:7": 315, + "0:8": 352, + "0:9": 351, + "4096:1": 1, + "4096:10": 40, + "4096:11": 682, + "4096:12": 636, + "4096:2": 2, + "4096:3": 3, + "4096:4": 4, + "4096:5": 5, + "4096:6": 6, + "4096:7": 7, + "4096:8": 8, + "4096:9": 9 + }, + "measured_rows": 3015, + "mixed_iteration_elapsed_ms": { + "cv": 0.04096511316969508, + "distinct_n": 126, + "max": 345.82, + "mean": 319.430703125, + "min": 294.13, + "n": 128 + }, + "mixed_prefill_decode_rows": 128, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-fixed-pd-state-diagnosis-20260719-r1/iteration-real/tp8_ep8_mns64/logs/server.log", + "pure_decode_batch_histogram": { + "1": 49, + "10": 70, + "11": 221, + "12": 2026, + "13": 159, + "2": 48, + "3": 46, + "4": 49, + "5": 43, + "6": 45, + "7": 45, + "8": 44, + "9": 39 + }, + "pure_decode_iteration_elapsed_ms": { + "cv": 0.08468104731049882, + "distinct_n": 347, + "max": 20.05, + "mean": 14.330596393897366, + "min": 9.46, + "n": 2884 + }, + "pure_decode_rows": 2884 + } + }, + "reference": { + "exact_iteration_tp8_minus_tp4_ms": 9.117724578203376, + "observed_real_tp8_minus_tp4_ms": 6.949999999999999, + "real_tpot_ms": { + "tp4_ep1_mns64": 21.0442, + "tp8_ep8_mns64": 27.9942 + } + }, + "same_batch_contrasts": { + "1": { + "tp4_n": 54, + "tp8_minus_tp4_ms": { + "attention": -0.8935417788173714, + "dense_mlp_compute": 5.551115123125783e-17, + "ep_communication": 7.6409967723488785, + "moe_compute": -1.3674890128222263, + "moe_routing": 0.24700169495130986, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -8.382111054979667, + "tp_dp_communication": -6.785968403651121 + }, + "tp8_n": 34 + }, + "10": { + "tp4_n": 38, + "tp8_minus_tp4_ms": { + "attention": -0.9062451258095203, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.288943843339915, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "11": { + "tp4_n": 42, + "tp8_minus_tp4_ms": { + "attention": -0.90756947260952, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.2707598272793539, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.29026819009679, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "12": { + "tp4_n": 46, + "tp8_minus_tp4_ms": { + "attention": -0.90756947260952, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.780182756222285, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.290268190109693, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 21 + }, + "13": { + "tp4_n": 46, + "tp8_minus_tp4_ms": { + "attention": -0.9271714387939403, + "dense_mlp_compute": 7.771561172376096e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622228, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.309870155730238, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "14": { + "tp4_n": 47, + "tp8_minus_tp4_ms": { + "attention": -0.9273699663939379, + "dense_mlp_compute": 6.938893903907228e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.310068683979324, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 21 + }, + "15": { + "tp4_n": 663, + "tp8_minus_tp4_ms": { + "attention": -0.9280176641939368, + "dense_mlp_compute": 6.938893903907228e-16, + "ep_communication": 6.824791048020124, + "moe_compute": -30.78018275622229, + "moe_routing": 0.2707598272793539, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.310716381302065, + "tp_dp_communication": -4.708641311979875 + }, + "tp8_n": 21 + }, + "16": { + "tp4_n": 1053, + "tp8_minus_tp4_ms": { + "attention": -0.9280176641939377, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.310716381309007, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "2": { + "tp4_n": 52, + "tp8_minus_tp4_ms": { + "attention": -0.893056937985337, + "dense_mlp_compute": 5.551115123125783e-17, + "ep_communication": 7.170214735418557, + "moe_compute": -2.487079396822226, + "moe_routing": 0.23761673495131297, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -7.529038359797266, + "tp_dp_communication": -6.276494110581443 + }, + "tp8_n": 32 + }, + "3": { + "tp4_n": 47, + "tp8_minus_tp4_ms": { + "attention": -0.8994740207853011, + "dense_mlp_compute": 8.326672684688674e-17, + "ep_communication": 6.661276281625032, + "moe_compute": -4.971655204025423, + "moe_routing": 0.23007132819364218, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -9.72273303992905, + "tp_dp_communication": -6.0593006383749675 + }, + "tp8_n": 31 + }, + "4": { + "tp4_n": 48, + "tp8_minus_tp4_ms": { + "attention": -0.9032519198967934, + "dense_mlp_compute": 5.551115123125783e-17, + "ep_communication": 6.661276281625032, + "moe_compute": -4.971655204025423, + "moe_routing": 0.23007132819364196, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -9.726510939439532, + "tp_dp_communication": -6.0593006383749675 + }, + "tp8_n": 30 + }, + "5": { + "tp4_n": 34, + "tp8_minus_tp4_ms": { + "attention": -0.8975988783634543, + "dense_mlp_compute": 7.494005416219807e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.90819860140641, + "moe_routing": 0.2783929890825345, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.211543726727538, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 27 + }, + "6": { + "tp4_n": 33, + "tp8_minus_tp4_ms": { + "attention": -0.8977163413634548, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.90819860140641, + "moe_routing": 0.2783929890825343, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.211661189448645, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 26 + }, + "7": { + "tp4_n": 34, + "tp8_minus_tp4_ms": { + "attention": -0.9209473308539557, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.908198601406404, + "moe_routing": 0.2783929890825345, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.23489217903426, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 26 + }, + "8": { + "tp4_n": 33, + "tp8_minus_tp4_ms": { + "attention": -0.9209473308539557, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.90819860140641, + "moe_routing": 0.27839298908253474, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.23489217902994, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 27 + }, + "9": { + "tp4_n": 32, + "tp8_minus_tp4_ms": { + "attention": -0.9096465718095201, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.292345289360462, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 21 + } + }, + "schema": "qwen235-fixed-pd-state-diagnosis-v1", + "scope": { + "graph_observability": "Frontier state outputs have no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors", + "op_trace_accounting": "For TP8 shared-domain sync, total is ledger stage_end-start (critical path); op categories are serialized work estimates and are not additive because overlap and lane-summed wait diagnostics are represented separately", + "real_proxy_limitation": "vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs", + "workload": "Fixed-PD 4096->256, 0.2 req/s/GPU, MNS64" + }, + "simulator": { + "tp4_ep1_mns64": { + "batch_support": { + "1": { + "components_ms": { + "attention": 5.918822475999999, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 4.277032336, + "moe_routing": 1.234618249, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.035006493, + "tp_dp_communication": 14.426965176 + }, + "n": 54 + }, + "10": { + "components_ms": { + "attention": 6.795322718, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.978072013, + "tp_dp_communication": 11.53343236 + }, + "n": 38 + }, + "11": { + "components_ms": { + "attention": 6.8427973799999995, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.025546675, + "tp_dp_communication": 11.53343236 + }, + "n": 42 + }, + "12": { + "components_ms": { + "attention": 6.84279738, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.02554667500001, + "tp_dp_communication": 11.53343236 + }, + "n": 46 + }, + "13": { + "components_ms": { + "attention": 7.310070996000001, + "dense_mlp_compute": 0.18060031999999998, + "ep_communication": 0.0, + "moe_compute": 41.171373458999994, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.492820290999994, + "tp_dp_communication": 11.53343236 + }, + "n": 46 + }, + "14": { + "components_ms": { + "attention": 7.316185658, + "dense_mlp_compute": 0.18060032000000004, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.49893495300001, + "tp_dp_communication": 11.53343236 + }, + "n": 47 + }, + "15": { + "components_ms": { + "attention": 7.332662654999999, + "dense_mlp_compute": 0.18060032000000004, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.51541195, + "tp_dp_communication": 11.533432359999999 + }, + "n": 663 + }, + "16": { + "components_ms": { + "attention": 7.332662655, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.51541195, + "tp_dp_communication": 11.53343236 + }, + "n": 1053 + }, + "2": { + "components_ms": { + "attention": 6.048001424230768, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 5.39662272, + "moe_routing": 1.237987209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.30688845623077, + "tp_dp_communication": 13.446708846 + }, + "n": 52 + }, + "3": { + "components_ms": { + "attention": 6.09964707180851, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 9.978847425, + "moe_routing": 1.2434858329999998, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 30.22012550680851, + "tp_dp_communication": 12.72057692 + }, + "n": 47 + }, + "4": { + "components_ms": { + "attention": 6.285367077, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 9.978847425, + "moe_routing": 1.243485833, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 30.405845512, + "tp_dp_communication": 12.72057692 + }, + "n": 48 + }, + "5": { + "components_ms": { + "attention": 6.313050302, + "dense_mlp_compute": 0.17946931199999996, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.419416699, + "tp_dp_communication": 11.97837976 + }, + "n": 34 + }, + "6": { + "components_ms": { + "attention": 6.356479957, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.462846354, + "tp_dp_communication": 11.97837976 + }, + "n": 33 + }, + "7": { + "components_ms": { + "attention": 6.734075413, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.68449211499999, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.84044181, + "tp_dp_communication": 11.97837976 + }, + "n": 34 + }, + "8": { + "components_ms": { + "attention": 6.734075413, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.84044181, + "tp_dp_communication": 11.97837976 + }, + "n": 33 + }, + "9": { + "components_ms": { + "attention": 6.782511044, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.965260339, + "tp_dp_communication": 11.53343236 + }, + "n": 32 + } + }, + "component_source": "stage_batch_ledger", + "decode_batch_size": { + "cv": 0.3076467178753811, + "distinct_n": 16, + "max": 16.0, + "mean": 13.537037037037036, + "min": 1.0, + "n": 2430 + }, + "decode_bearing_rows": 2430, + "decode_token_weighted_all_step_components_ms": { + "attention": 13.842547047292292, + "dense_mlp_compute": 0.2927251518073871, + "ep_communication": 0.0, + "moe_compute": 45.47878637854458, + "moe_routing": 1.8023597240381213, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 74.2229273813438, + "tp_dp_communication": 12.806509080469617 + }, + "decode_token_weighted_mixed_step_share": 0.05481076151390789, + "mixed_prefill_decode_rows": 128, + "pure_decode_rows": 2302, + "pure_decode_token_weighted_components_ms": { + "attention": 7.263972797140871, + "dense_mlp_compute": 0.18052079084832112, + "ep_communication": 0.0, + "moe_compute": 40.472043520877655, + "moe_routing": 1.2955265717923583, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.78208599545992, + "tp_dp_communication": 11.57002231571491 + }, + "scorer_equivalence": { + "base_metrics": { + "e2e_mean_ms": 19261.684938205322, + "e2e_p50_ms": 19681.421068277567, + "e2e_p90_ms": 19712.678973231064, + "e2e_p95_ms": 19717.18006692948, + "slos": { + "tpot_100ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_150ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_180ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_50ms": { + "feasible": false, + "pass_rate": 0.0, + "passed": 0 + } + }, + "tpot_mean_ms": 74.22292738142716, + "tpot_p50_ms": 75.8976524911963, + "tpot_p90_ms": 75.8976524912086, + "tpot_p95_ms": 75.89765249120967, + "ttft_mean_ms": 334.8384559413977, + "ttft_p50_ms": 334.75193988430976, + "ttft_p90_ms": 359.7779859328796, + "ttft_p95_ms": 363.2786816744229 + }, + "request_metrics_byte_identical": true, + "request_metrics_sha256": "01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f" + }, + "stage_rows": 2431, + "trace_validation": null + }, + "tp8_ep8_mns64": { + "batch_support": { + "1": { + "components_ms": { + "attention": 5.025280697182628, + "dense_mlp_compute": 0.17756825600000006, + "ep_communication": 7.6409967723488785, + "moe_compute": 2.909543323177774, + "moe_routing": 1.4816199439513098, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 17.652895438020334, + "tp_dp_communication": 7.6409967723488785 + }, + "n": 34 + }, + "10": { + "components_ms": { + "attention": 5.88907759219048, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.68912816966009, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "11": { + "components_ms": { + "attention": 5.9352279073904795, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.568102984279354, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.73527848490321, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "12": { + "components_ms": { + "attention": 5.93522790739048, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777717, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.735278484890316, + "tp_dp_communication": 6.824791048020123 + }, + "n": 21 + }, + "13": { + "components_ms": { + "attention": 6.382899557206061, + "dense_mlp_compute": 0.18060032000000076, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.182950135269756, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "14": { + "components_ms": { + "attention": 6.388815691606062, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.188866269020686, + "tp_dp_communication": 6.824791048020123 + }, + "n": 21 + }, + "15": { + "components_ms": { + "attention": 6.404644990806062, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020124, + "moe_compute": 10.391190702777713, + "moe_routing": 1.568102984279354, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.204695568697936, + "tp_dp_communication": 6.824791048020124 + }, + "n": 21 + }, + "16": { + "components_ms": { + "attention": 6.404644990806062, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.204695568690994, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "17": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.63356500064447, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.2554552675483, + "tp_dp_communication": 6.763788700699805 + }, + "n": 31 + }, + "18": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644474, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.25545526754183, + "tp_dp_communication": 6.763788700699805 + }, + "n": 30 + }, + "19": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644472, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.25545526755436, + "tp_dp_communication": 6.763788700699805 + }, + "n": 32 + }, + "2": { + "components_ms": { + "attention": 5.154944486245431, + "dense_mlp_compute": 0.17756825600000006, + "ep_communication": 7.170214735418557, + "moe_compute": 2.909543323177774, + "moe_routing": 1.475603943951313, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 18.777850096433504, + "tp_dp_communication": 7.170214735418557 + }, + "n": 32 + }, + "20": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699804, + "moe_compute": 14.633565000644472, + "moe_routing": 1.6479551064984042, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.25545526756006, + "tp_dp_communication": 6.763788700699804 + }, + "n": 33 + }, + "21": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.18267583999999895, + "ep_communication": 6.763788700699806, + "moe_compute": 14.633565000644474, + "moe_routing": 1.6479551064984037, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.58688465210873, + "tp_dp_communication": 6.763788700699806 + }, + "n": 37 + }, + "22": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644472, + "moe_routing": 1.6479551064984042, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.586884652057584, + "tp_dp_communication": 6.763788700699805 + }, + "n": 51 + }, + "23": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.182675839999999, + "ep_communication": 6.763788700699805, + "moe_compute": 14.63356500064447, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.58688465207786, + "tp_dp_communication": 6.763788700699805 + }, + "n": 906 + }, + "24": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.18267583999999895, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644472, + "moe_routing": 1.6479551064984037, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.58688465207813, + "tp_dp_communication": 6.763788700699805 + }, + "n": 84 + }, + "3": { + "components_ms": { + "attention": 5.200173051023209, + "dense_mlp_compute": 0.1775682560000001, + "ep_communication": 6.661276281625032, + "moe_compute": 5.007192220974576, + "moe_routing": 1.473557161193642, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 20.49739246687946, + "tp_dp_communication": 6.661276281625032 + }, + "n": 31 + }, + "4": { + "components_ms": { + "attention": 5.382115157103207, + "dense_mlp_compute": 0.17756825600000006, + "ep_communication": 6.661276281625032, + "moe_compute": 5.007192220974576, + "moe_routing": 1.473557161193642, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 20.679334572560467, + "tp_dp_communication": 6.661276281625032 + }, + "n": 30 + }, + "5": { + "components_ms": { + "attention": 5.415451423636545, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825344, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.207872972272465, + "tp_dp_communication": 6.619871115982532 + }, + "n": 27 + }, + "6": { + "components_ms": { + "attention": 5.4587636156365456, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825342, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.251185164551355, + "tp_dp_communication": 6.619871115982532 + }, + "n": 26 + }, + "7": { + "components_ms": { + "attention": 5.8131280821460445, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825344, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.60554963096574, + "tp_dp_communication": 6.619871115982532 + }, + "n": 26 + }, + "8": { + "components_ms": { + "attention": 5.8131280821460445, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825347, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.60554963097006, + "tp_dp_communication": 6.619871115982532 + }, + "n": 27 + }, + "9": { + "components_ms": { + "attention": 5.8728644721904795, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.672915049639535, + "tp_dp_communication": 6.824791048020123 + }, + "n": 21 + } + }, + "component_source": "op_trace_execution_time_override", + "decode_batch_size": { + "cv": 0.3602756967919564, + "distinct_n": 24, + "max": 24.0, + "mean": 18.93782383419689, + "min": 1.0, + "n": 1737 + }, + "decode_bearing_rows": 1737, + "decode_token_weighted_all_step_components_ms": { + "attention": 6.38060442207261, + "dense_mlp_compute": 0.16958377884622666, + "ep_communication": 6.235402017662346, + "moe_compute": 13.024343946852467, + "moe_routing": 1.5161854734738416, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.156672699566656, + "tp_dp_communication": 6.235402017662346 + }, + "decode_token_weighted_mixed_step_share": 0.08077215382276942, + "mixed_prefill_decode_rows": 128, + "pure_decode_rows": 1609, + "pure_decode_token_weighted_components_ms": { + "attention": 6.853695012970485, + "dense_mlp_compute": 0.1824026696707446, + "ep_communication": 6.765971286395407, + "moe_compute": 14.092651702097841, + "moe_routing": 1.638092363428458, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.94606316261678, + "tp_dp_communication": 6.765971286395407 + }, + "scorer_equivalence": { + "base_metrics": { + "e2e_mean_ms": 14056.464119438264, + "e2e_p50_ms": 14650.653983203298, + "e2e_p90_ms": 14674.997135818061, + "e2e_p95_ms": 14677.859760866291, + "slos": { + "tpot_100ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_150ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_180ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_50ms": { + "feasible": false, + "pass_rate": 0.14728682170542637, + "passed": 19 + } + }, + "tpot_mean_ms": 54.15667269956665, + "tpot_p50_ms": 56.52350191837838, + "tpot_p90_ms": 56.52350850045728, + "tpot_p95_ms": 56.523546100039816, + "ttft_mean_ms": 246.51258104876885, + "ttft_p50_ms": 246.7781280760306, + "ttft_p90_ms": 263.0087113349084, + "ttft_p95_ms": 264.77576248352364 + }, + "request_metrics_byte_identical": true, + "request_metrics_sha256": "51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b" + }, + "stage_rows": 1738, + "trace_validation": { + "diagnostic_wait_ms": { + "cv": 0.3169483933455645, + "distinct_n": 26, + "max": 67.2823542576566, + "mean": 38.03960569005978, + "min": 7.116029381293295, + "n": 1609 + }, + "event_names": { + "attn_decode": 1737, + "attn_kv_cache_save": 1738, + "attn_post_proj": 1738, + "attn_pre_proj": 1738, + "attn_prefill": 129, + "attn_rope": 1738, + "attn_tensor_parallel_allreduce": 1738, + "expert_parallel_allreduce": 1738, + "expert_parallel_allreduce_wait": 1609, + "input_layernorm": 1738, + "moe_gating_linear": 1738, + "moe_gating_routing_topk": 1738, + "moe_grouped_gemm": 1738, + "moe_shuffling": 1738, + "post_attention_layernorm": 1738 + }, + "ledger_batches": 1738, + "serialized_component_sum_over_stage_span": { + "cv": 0.30467075878025907, + "distinct_n": 69, + "max": 1.4091742542979306, + "mean": 0.8931198465670052, + "min": 0.010604399884068217, + "n": 1738 + }, + "trace_batches": 1738, + "trace_path": "/home/admin/cpfs/wjh/aituner/qwen235-fixed-pd-state-diagnosis-20260719-r1/frontier-optrace/tp8_ep8_mns64/frontier_metrics/qwen3_235b_a22b/online_serving/qwen235_fixed_pd_state_tp8_ep8_mns64/op_traces.jsonl" + } + } + }, + "simulator_internal_all_step_tp8_minus_tp4_ms": { + "attention": -7.461942625219683, + "dense_mlp_compute": -0.12314137296116046, + "ep_communication": 6.235402017662346, + "moe_compute": -32.45444243169211, + "moe_routing": -0.2861742505642797, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -20.06625468177714, + "tp_dp_communication": -6.571107062807271 + }, + "status": "PASS", + "verdict": "Exact real state composition flips Frontier to the correct TP4 winner; the ranking error is caused by closed-loop state/composition mismatch, not a conditional stage reversal." +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-exact.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-exact.md new file mode 100644 index 0000000..70092bd --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-exact.md @@ -0,0 +1,54 @@ +# Qwen235 Fixed-PD state diagnosis + +**Verdict:** Exact real state composition flips Frontier to the correct TP4 winner; the ranking error is caused by closed-loop state/composition mismatch, not a conditional stage reversal. + +| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage | +|---|---:|---:|---:| +| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% | +| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% | + +| Config | Real exact decode batch mean | Real token-weighted iteration ms | Matched simulator ms | Exact coverage | +|---|---:|---:|---:|---:| +| tp4_ep1_mns64 | 3.890 | 18.4710 | 34.2575 | 100.0% | +| tp8_ep8_mns64 | 10.921 | 27.5887 | 45.1603 | 100.0% | + +## Frontier internal component contrast over its own executed composition + +Decode-token-weighted over both pure-decode and mixed prefill/decode steps. + +| Component | TP8 - TP4 (ms/decoded token step) | +|---|---:| +| moe_compute | -32.4544 | +| total | -20.0663 | +| attention | -7.4619 | +| tp_dp_communication | -6.5711 | +| ep_communication | +6.2354 | +| moe_routing | -0.2862 | +| dense_mlp_compute | -0.1231 | +| pipeline_communication | +0.0000 | +| runtime_overhead | +0.0000 | + +## Frontier component contrast at exact real token composition + +Positive means TP8 slower; negative means Frontier gives TP8 an advantage. + +| Component | TP8 - TP4 (ms/step) | +|---|---:| +| total | +10.9028 | +| ep_communication | +6.5355 | +| tp_dp_communication | -6.4718 | +| attention | -2.1304 | +| moe_compute | -1.9774 | +| moe_routing | +0.1384 | +| dense_mlp_compute | -0.0308 | +| pipeline_communication | +0.0000 | +| runtime_overhead | +0.0000 | + +## Interpretation boundary + +- Real observed TPOT contrast: +6.9500 ms/token. +- Real exact-iteration contrast: +9.1177 ms/decode-token-weighted step. +- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs. +- Frontier state outputs have no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors. +- For TP8 shared-domain sync, total is ledger stage_end-start (critical path); op categories are serialized work estimates and are not additive because overlap and lane-summed wait diagnostics are represented separately. +- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof. diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-optrace.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-optrace.json new file mode 100644 index 0000000..00dc957 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-optrace.json @@ -0,0 +1,1405 @@ +{ + "exact_state_matched": null, + "proxy_matched": { + "configs": { + "tp4_ep1_mns64": { + "components_ms": { + "attention": 6.303822560333333, + "dense_mlp_compute": 0.17883562666666666, + "ep_communication": 0.0, + "moe_compute": 26.449277218333332, + "moe_routing": 1.2571787503333332, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 46.414892970000004, + "tp_dp_communication": 12.225778813333333 + }, + "coverage": 1.0, + "supported_samples": 45, + "total_samples": 45, + "unsupported": {} + }, + "tp8_ep8_mns64": { + "components_ms": { + "attention": 6.063134093052075, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777717, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.86318467071363, + "tp_dp_communication": 6.824791048020123 + }, + "coverage": 1.0, + "supported_samples": 21, + "total_samples": 21, + "unsupported": {} + } + }, + "method": "exact batch-size lookup; each simulator config is reweighted to its own frozen-real steady Running histogram; no interpolation", + "tp8_minus_tp4_ms": { + "attention": -0.2406884672812577, + "dense_mlp_compute": 0.0017646933333340664, + "ep_communication": 6.824791048020123, + "moe_compute": -16.058086515555615, + "moe_routing": 0.31092423394602053, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -8.551708299286375, + "tp_dp_communication": -5.40098776531321 + } + }, + "real": { + "tp4_ep1_mns64": { + "aggregate": { + "generation_tokens_per_s": { + "cv": 0.01219623132338527, + "distinct_n": 32, + "max": 217.3, + "mean": 204.7488888888889, + "min": 197.1, + "n": 45 + }, + "kv_percent": { + "cv": 0.09908502666308305, + "distinct_n": 2, + "max": 5.6, + "mean": 5.233333333333333, + "min": 4.5, + "n": 45 + }, + "running": { + "cv": 0.10101525445522107, + "distinct_n": 2, + "max": 5.0, + "mean": 4.666666666666667, + "min": 4.0, + "n": 45 + }, + "running_histogram": { + "4": 15, + "5": 30 + }, + "samples": 45, + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 45 + } + }, + "proxy_only": true, + "steady_rule": "prompt throughput >= 95% of 3276.8 tokens/s", + "trials": [ + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.01635665678820842, + "distinct_n": 13, + "max": 217.3, + "mean": 205.58666666666667, + "min": 202.4, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.6, + "mean": 5.6, + "min": 5.6, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.0, + "mean": 5.0, + "min": 5.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial1", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + }, + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.01101205870299425, + "distinct_n": 11, + "max": 206.3, + "mean": 204.18666666666667, + "min": 197.1, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.6, + "mean": 5.6, + "min": 5.6, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.0, + "mean": 5.0, + "min": 5.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial2", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + }, + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.005467774668645034, + "distinct_n": 14, + "max": 206.0, + "mean": 204.47333333333333, + "min": 202.0, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 4.5, + "mean": 4.5, + "min": 4.5, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 4.0, + "mean": 4.0, + "min": 4.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial3", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + } + ] + }, + "tp8_ep8_mns64": { + "aggregate": { + "generation_tokens_per_s": { + "cv": 0.01855015540813005, + "distinct_n": 21, + "max": 415.7, + "mean": 406.5380952380952, + "min": 390.8, + "n": 21 + }, + "kv_percent": { + "cv": 0.03973151445031385, + "distinct_n": 3, + "max": 5.6, + "mean": 5.295238095238095, + "min": 4.8, + "n": 21 + }, + "running": { + "cv": 0.04297805843263526, + "distinct_n": 3, + "max": 13.0, + "mean": 12.238095238095237, + "min": 11.0, + "n": 21 + }, + "running_histogram": { + "11": 1, + "12": 14, + "13": 6 + }, + "samples": 21, + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 21 + } + }, + "proxy_only": true, + "steady_rule": "prompt throughput >= 95% of 6553.6 tokens/s", + "trials": [ + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.018873398222858283, + "distinct_n": 7, + "max": 415.7, + "mean": 406.87142857142857, + "min": 392.3, + "n": 7 + }, + "kv_percent": { + "cv": 0.03400298559320833, + "distinct_n": 2, + "max": 5.6, + "mean": 5.314285714285715, + "min": 5.2, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/logs/server.log", + "running": { + "cv": 0.036770670467074176, + "distinct_n": 2, + "max": 13.0, + "mean": 12.285714285714286, + "min": 12.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial1", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + }, + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.01869328742146201, + "distinct_n": 7, + "max": 412.9, + "mean": 406.2857142857143, + "min": 390.8, + "n": 7 + }, + "kv_percent": { + "cv": 0.04861017342390845, + "distinct_n": 3, + "max": 5.6, + "mean": 5.257142857142857, + "min": 4.8, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/logs/server.log", + "running": { + "cv": 0.052613364176465637, + "distinct_n": 3, + "max": 13.0, + "mean": 12.142857142857142, + "min": 11.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial2", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + }, + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.018043345382112573, + "distinct_n": 7, + "max": 414.9, + "mean": 406.45714285714286, + "min": 393.0, + "n": 7 + }, + "kv_percent": { + "cv": 0.03400298559320833, + "distinct_n": 2, + "max": 5.6, + "mean": 5.314285714285715, + "min": 5.2, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/logs/server.log", + "running": { + "cv": 0.036770670467074176, + "distinct_n": 2, + "max": 13.0, + "mean": 12.285714285714286, + "min": 12.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial3", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + } + ] + } + }, + "real_iteration_state": null, + "reference": { + "observed_real_tp8_minus_tp4_ms": 6.949999999999999, + "real_tpot_ms": { + "tp4_ep1_mns64": 21.0442, + "tp8_ep8_mns64": 27.9942 + } + }, + "same_batch_contrasts": { + "1": { + "tp4_n": 54, + "tp8_minus_tp4_ms": { + "attention": -0.8935417788173714, + "dense_mlp_compute": 5.551115123125783e-17, + "ep_communication": 7.6409967723488785, + "moe_compute": -1.3674890128222263, + "moe_routing": 0.24700169495130986, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -8.382111054979667, + "tp_dp_communication": -6.785968403651121 + }, + "tp8_n": 34 + }, + "10": { + "tp4_n": 38, + "tp8_minus_tp4_ms": { + "attention": -0.9062451258095203, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.288943843339915, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "11": { + "tp4_n": 42, + "tp8_minus_tp4_ms": { + "attention": -0.90756947260952, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.2707598272793539, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.29026819009679, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "12": { + "tp4_n": 46, + "tp8_minus_tp4_ms": { + "attention": -0.90756947260952, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.780182756222285, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.290268190109693, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 21 + }, + "13": { + "tp4_n": 46, + "tp8_minus_tp4_ms": { + "attention": -0.9271714387939403, + "dense_mlp_compute": 7.771561172376096e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622228, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.309870155730238, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "14": { + "tp4_n": 47, + "tp8_minus_tp4_ms": { + "attention": -0.9273699663939379, + "dense_mlp_compute": 6.938893903907228e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.310068683979324, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 21 + }, + "15": { + "tp4_n": 663, + "tp8_minus_tp4_ms": { + "attention": -0.9280176641939368, + "dense_mlp_compute": 6.938893903907228e-16, + "ep_communication": 6.824791048020124, + "moe_compute": -30.78018275622229, + "moe_routing": 0.2707598272793539, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.310716381302065, + "tp_dp_communication": -4.708641311979875 + }, + "tp8_n": 21 + }, + "16": { + "tp4_n": 1053, + "tp8_minus_tp4_ms": { + "attention": -0.9280176641939377, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.310716381309007, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 22 + }, + "2": { + "tp4_n": 52, + "tp8_minus_tp4_ms": { + "attention": -0.893056937985337, + "dense_mlp_compute": 5.551115123125783e-17, + "ep_communication": 7.170214735418557, + "moe_compute": -2.487079396822226, + "moe_routing": 0.23761673495131297, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -7.529038359797266, + "tp_dp_communication": -6.276494110581443 + }, + "tp8_n": 32 + }, + "3": { + "tp4_n": 47, + "tp8_minus_tp4_ms": { + "attention": -0.8994740207853011, + "dense_mlp_compute": 8.326672684688674e-17, + "ep_communication": 6.661276281625032, + "moe_compute": -4.971655204025423, + "moe_routing": 0.23007132819364218, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -9.72273303992905, + "tp_dp_communication": -6.0593006383749675 + }, + "tp8_n": 31 + }, + "4": { + "tp4_n": 48, + "tp8_minus_tp4_ms": { + "attention": -0.9032519198967934, + "dense_mlp_compute": 5.551115123125783e-17, + "ep_communication": 6.661276281625032, + "moe_compute": -4.971655204025423, + "moe_routing": 0.23007132819364196, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -9.726510939439532, + "tp_dp_communication": -6.0593006383749675 + }, + "tp8_n": 30 + }, + "5": { + "tp4_n": 34, + "tp8_minus_tp4_ms": { + "attention": -0.8975988783634543, + "dense_mlp_compute": 7.494005416219807e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.90819860140641, + "moe_routing": 0.2783929890825345, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.211543726727538, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 27 + }, + "6": { + "tp4_n": 33, + "tp8_minus_tp4_ms": { + "attention": -0.8977163413634548, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.90819860140641, + "moe_routing": 0.2783929890825343, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.211661189448645, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 26 + }, + "7": { + "tp4_n": 34, + "tp8_minus_tp4_ms": { + "attention": -0.9209473308539557, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.908198601406404, + "moe_routing": 0.2783929890825345, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.23489217903426, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 26 + }, + "8": { + "tp4_n": 33, + "tp8_minus_tp4_ms": { + "attention": -0.9209473308539557, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.619871115982532, + "moe_compute": -25.90819860140641, + "moe_routing": 0.27839298908253474, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -28.23489217902994, + "tp_dp_communication": -5.358508644017467 + }, + "tp8_n": 27 + }, + "9": { + "tp4_n": 32, + "tp8_minus_tp4_ms": { + "attention": -0.9096465718095201, + "dense_mlp_compute": 7.216449660063518e-16, + "ep_communication": 6.824791048020123, + "moe_compute": -30.78018275622229, + "moe_routing": 0.27075982727935366, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -23.292345289360462, + "tp_dp_communication": -4.708641311979878 + }, + "tp8_n": 21 + } + }, + "schema": "qwen235-fixed-pd-state-diagnosis-v1", + "scope": { + "graph_observability": "Frontier state outputs have no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors", + "op_trace_accounting": "For TP8 shared-domain sync, total is ledger stage_end-start (critical path); op categories are serialized work estimates and are not additive because overlap and lane-summed wait diagnostics are represented separately", + "real_proxy_limitation": "vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs", + "workload": "Fixed-PD 4096->256, 0.2 req/s/GPU, MNS64" + }, + "simulator": { + "tp4_ep1_mns64": { + "batch_support": { + "1": { + "components_ms": { + "attention": 5.918822475999999, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 4.277032336, + "moe_routing": 1.234618249, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.035006493, + "tp_dp_communication": 14.426965176 + }, + "n": 54 + }, + "10": { + "components_ms": { + "attention": 6.795322718, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.978072013, + "tp_dp_communication": 11.53343236 + }, + "n": 38 + }, + "11": { + "components_ms": { + "attention": 6.8427973799999995, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.025546675, + "tp_dp_communication": 11.53343236 + }, + "n": 42 + }, + "12": { + "components_ms": { + "attention": 6.84279738, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.02554667500001, + "tp_dp_communication": 11.53343236 + }, + "n": 46 + }, + "13": { + "components_ms": { + "attention": 7.310070996000001, + "dense_mlp_compute": 0.18060031999999998, + "ep_communication": 0.0, + "moe_compute": 41.171373458999994, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.492820290999994, + "tp_dp_communication": 11.53343236 + }, + "n": 46 + }, + "14": { + "components_ms": { + "attention": 7.316185658, + "dense_mlp_compute": 0.18060032000000004, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.49893495300001, + "tp_dp_communication": 11.53343236 + }, + "n": 47 + }, + "15": { + "components_ms": { + "attention": 7.332662654999999, + "dense_mlp_compute": 0.18060032000000004, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.51541195, + "tp_dp_communication": 11.533432359999999 + }, + "n": 663 + }, + "16": { + "components_ms": { + "attention": 7.332662655, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.51541195, + "tp_dp_communication": 11.53343236 + }, + "n": 1053 + }, + "2": { + "components_ms": { + "attention": 6.048001424230768, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 5.39662272, + "moe_routing": 1.237987209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.30688845623077, + "tp_dp_communication": 13.446708846 + }, + "n": 52 + }, + "3": { + "components_ms": { + "attention": 6.09964707180851, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 9.978847425, + "moe_routing": 1.2434858329999998, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 30.22012550680851, + "tp_dp_communication": 12.72057692 + }, + "n": 47 + }, + "4": { + "components_ms": { + "attention": 6.285367077, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 9.978847425, + "moe_routing": 1.243485833, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 30.405845512, + "tp_dp_communication": 12.72057692 + }, + "n": 48 + }, + "5": { + "components_ms": { + "attention": 6.313050302, + "dense_mlp_compute": 0.17946931199999996, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.419416699, + "tp_dp_communication": 11.97837976 + }, + "n": 34 + }, + "6": { + "components_ms": { + "attention": 6.356479957, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.462846354, + "tp_dp_communication": 11.97837976 + }, + "n": 33 + }, + "7": { + "components_ms": { + "attention": 6.734075413, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.68449211499999, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.84044181, + "tp_dp_communication": 11.97837976 + }, + "n": 34 + }, + "8": { + "components_ms": { + "attention": 6.734075413, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.84044181, + "tp_dp_communication": 11.97837976 + }, + "n": 33 + }, + "9": { + "components_ms": { + "attention": 6.782511044, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.965260339, + "tp_dp_communication": 11.53343236 + }, + "n": 32 + } + }, + "component_source": "stage_batch_ledger", + "decode_batch_size": { + "cv": 0.3076467178753811, + "distinct_n": 16, + "max": 16.0, + "mean": 13.537037037037036, + "min": 1.0, + "n": 2430 + }, + "decode_bearing_rows": 2430, + "decode_token_weighted_all_step_components_ms": { + "attention": 13.842547047292292, + "dense_mlp_compute": 0.2927251518073871, + "ep_communication": 0.0, + "moe_compute": 45.47878637854458, + "moe_routing": 1.8023597240381213, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 74.2229273813438, + "tp_dp_communication": 12.806509080469617 + }, + "decode_token_weighted_mixed_step_share": 0.05481076151390789, + "mixed_prefill_decode_rows": 128, + "pure_decode_rows": 2302, + "pure_decode_token_weighted_components_ms": { + "attention": 7.263972797140871, + "dense_mlp_compute": 0.18052079084832112, + "ep_communication": 0.0, + "moe_compute": 40.472043520877655, + "moe_routing": 1.2955265717923583, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.78208599545992, + "tp_dp_communication": 11.57002231571491 + }, + "scorer_equivalence": { + "base_metrics": { + "e2e_mean_ms": 19261.684938205322, + "e2e_p50_ms": 19681.421068277567, + "e2e_p90_ms": 19712.678973231064, + "e2e_p95_ms": 19717.18006692948, + "slos": { + "tpot_100ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_150ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_180ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_50ms": { + "feasible": false, + "pass_rate": 0.0, + "passed": 0 + } + }, + "tpot_mean_ms": 74.22292738142716, + "tpot_p50_ms": 75.8976524911963, + "tpot_p90_ms": 75.8976524912086, + "tpot_p95_ms": 75.89765249120967, + "ttft_mean_ms": 334.8384559413977, + "ttft_p50_ms": 334.75193988430976, + "ttft_p90_ms": 359.7779859328796, + "ttft_p95_ms": 363.2786816744229 + }, + "request_metrics_byte_identical": true, + "request_metrics_sha256": "01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f" + }, + "stage_rows": 2431, + "trace_validation": null + }, + "tp8_ep8_mns64": { + "batch_support": { + "1": { + "components_ms": { + "attention": 5.025280697182628, + "dense_mlp_compute": 0.17756825600000006, + "ep_communication": 7.6409967723488785, + "moe_compute": 2.909543323177774, + "moe_routing": 1.4816199439513098, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 17.652895438020334, + "tp_dp_communication": 7.6409967723488785 + }, + "n": 34 + }, + "10": { + "components_ms": { + "attention": 5.88907759219048, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.68912816966009, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "11": { + "components_ms": { + "attention": 5.9352279073904795, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.568102984279354, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.73527848490321, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "12": { + "components_ms": { + "attention": 5.93522790739048, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777717, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.735278484890316, + "tp_dp_communication": 6.824791048020123 + }, + "n": 21 + }, + "13": { + "components_ms": { + "attention": 6.382899557206061, + "dense_mlp_compute": 0.18060032000000076, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.182950135269756, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "14": { + "components_ms": { + "attention": 6.388815691606062, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.188866269020686, + "tp_dp_communication": 6.824791048020123 + }, + "n": 21 + }, + "15": { + "components_ms": { + "attention": 6.404644990806062, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020124, + "moe_compute": 10.391190702777713, + "moe_routing": 1.568102984279354, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.204695568697936, + "tp_dp_communication": 6.824791048020124 + }, + "n": 21 + }, + "16": { + "components_ms": { + "attention": 6.404644990806062, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.204695568690994, + "tp_dp_communication": 6.824791048020123 + }, + "n": 22 + }, + "17": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.63356500064447, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.2554552675483, + "tp_dp_communication": 6.763788700699805 + }, + "n": 31 + }, + "18": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644474, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.25545526754183, + "tp_dp_communication": 6.763788700699805 + }, + "n": 30 + }, + "19": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644472, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.25545526755436, + "tp_dp_communication": 6.763788700699805 + }, + "n": 32 + }, + "2": { + "components_ms": { + "attention": 5.154944486245431, + "dense_mlp_compute": 0.17756825600000006, + "ep_communication": 7.170214735418557, + "moe_compute": 2.909543323177774, + "moe_routing": 1.475603943951313, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 18.777850096433504, + "tp_dp_communication": 7.170214735418557 + }, + "n": 32 + }, + "20": { + "components_ms": { + "attention": 6.652331067872723, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699804, + "moe_compute": 14.633565000644472, + "moe_routing": 1.6479551064984042, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.25545526756006, + "tp_dp_communication": 6.763788700699804 + }, + "n": 33 + }, + "21": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.18267583999999895, + "ep_communication": 6.763788700699806, + "moe_compute": 14.633565000644474, + "moe_routing": 1.6479551064984037, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.58688465210873, + "tp_dp_communication": 6.763788700699806 + }, + "n": 37 + }, + "22": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.18267583999999898, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644472, + "moe_routing": 1.6479551064984042, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.586884652057584, + "tp_dp_communication": 6.763788700699805 + }, + "n": 51 + }, + "23": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.182675839999999, + "ep_communication": 6.763788700699805, + "moe_compute": 14.63356500064447, + "moe_routing": 1.647955106498404, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.58688465207786, + "tp_dp_communication": 6.763788700699805 + }, + "n": 906 + }, + "24": { + "components_ms": { + "attention": 6.9837604522698395, + "dense_mlp_compute": 0.18267583999999895, + "ep_communication": 6.763788700699805, + "moe_compute": 14.633565000644472, + "moe_routing": 1.6479551064984037, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 39.58688465207813, + "tp_dp_communication": 6.763788700699805 + }, + "n": 84 + }, + "3": { + "components_ms": { + "attention": 5.200173051023209, + "dense_mlp_compute": 0.1775682560000001, + "ep_communication": 6.661276281625032, + "moe_compute": 5.007192220974576, + "moe_routing": 1.473557161193642, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 20.49739246687946, + "tp_dp_communication": 6.661276281625032 + }, + "n": 31 + }, + "4": { + "components_ms": { + "attention": 5.382115157103207, + "dense_mlp_compute": 0.17756825600000006, + "ep_communication": 6.661276281625032, + "moe_compute": 5.007192220974576, + "moe_routing": 1.473557161193642, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 20.679334572560467, + "tp_dp_communication": 6.661276281625032 + }, + "n": 30 + }, + "5": { + "components_ms": { + "attention": 5.415451423636545, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825344, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.207872972272465, + "tp_dp_communication": 6.619871115982532 + }, + "n": 27 + }, + "6": { + "components_ms": { + "attention": 5.4587636156365456, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825342, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.251185164551355, + "tp_dp_communication": 6.619871115982532 + }, + "n": 26 + }, + "7": { + "components_ms": { + "attention": 5.8131280821460445, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825344, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.60554963096574, + "tp_dp_communication": 6.619871115982532 + }, + "n": 26 + }, + "8": { + "components_ms": { + "attention": 5.8131280821460445, + "dense_mlp_compute": 0.1794693120000007, + "ep_communication": 6.619871115982532, + "moe_compute": 8.776293513593586, + "moe_routing": 1.5424181980825347, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.60554963097006, + "tp_dp_communication": 6.619871115982532 + }, + "n": 27 + }, + "9": { + "components_ms": { + "attention": 5.8728644721904795, + "dense_mlp_compute": 0.18060032000000073, + "ep_communication": 6.824791048020123, + "moe_compute": 10.391190702777715, + "moe_routing": 1.5681029842793537, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 37.672915049639535, + "tp_dp_communication": 6.824791048020123 + }, + "n": 21 + } + }, + "component_source": "op_trace_execution_time_override", + "decode_batch_size": { + "cv": 0.3602756967919564, + "distinct_n": 24, + "max": 24.0, + "mean": 18.93782383419689, + "min": 1.0, + "n": 1737 + }, + "decode_bearing_rows": 1737, + "decode_token_weighted_all_step_components_ms": { + "attention": 6.38060442207261, + "dense_mlp_compute": 0.16958377884622666, + "ep_communication": 6.235402017662346, + "moe_compute": 13.024343946852467, + "moe_routing": 1.5161854734738416, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.156672699566656, + "tp_dp_communication": 6.235402017662346 + }, + "decode_token_weighted_mixed_step_share": 0.08077215382276942, + "mixed_prefill_decode_rows": 128, + "pure_decode_rows": 1609, + "pure_decode_token_weighted_components_ms": { + "attention": 6.853695012970485, + "dense_mlp_compute": 0.1824026696707446, + "ep_communication": 6.765971286395407, + "moe_compute": 14.092651702097841, + "moe_routing": 1.638092363428458, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 38.94606316261678, + "tp_dp_communication": 6.765971286395407 + }, + "scorer_equivalence": { + "base_metrics": { + "e2e_mean_ms": 14056.464119438264, + "e2e_p50_ms": 14650.653983203298, + "e2e_p90_ms": 14674.997135818061, + "e2e_p95_ms": 14677.859760866291, + "slos": { + "tpot_100ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_150ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_180ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_50ms": { + "feasible": false, + "pass_rate": 0.14728682170542637, + "passed": 19 + } + }, + "tpot_mean_ms": 54.15667269956665, + "tpot_p50_ms": 56.52350191837838, + "tpot_p90_ms": 56.52350850045728, + "tpot_p95_ms": 56.523546100039816, + "ttft_mean_ms": 246.51258104876885, + "ttft_p50_ms": 246.7781280760306, + "ttft_p90_ms": 263.0087113349084, + "ttft_p95_ms": 264.77576248352364 + }, + "request_metrics_byte_identical": true, + "request_metrics_sha256": "51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b" + }, + "stage_rows": 1738, + "trace_validation": { + "diagnostic_wait_ms": { + "cv": 0.3169483933455645, + "distinct_n": 26, + "max": 67.2823542576566, + "mean": 38.03960569005978, + "min": 7.116029381293295, + "n": 1609 + }, + "event_names": { + "attn_decode": 1737, + "attn_kv_cache_save": 1738, + "attn_post_proj": 1738, + "attn_pre_proj": 1738, + "attn_prefill": 129, + "attn_rope": 1738, + "attn_tensor_parallel_allreduce": 1738, + "expert_parallel_allreduce": 1738, + "expert_parallel_allreduce_wait": 1609, + "input_layernorm": 1738, + "moe_gating_linear": 1738, + "moe_gating_routing_topk": 1738, + "moe_grouped_gemm": 1738, + "moe_shuffling": 1738, + "post_attention_layernorm": 1738 + }, + "ledger_batches": 1738, + "serialized_component_sum_over_stage_span": { + "cv": 0.30467075878025907, + "distinct_n": 69, + "max": 1.4091742542979306, + "mean": 0.8931198465670052, + "min": 0.010604399884068217, + "n": 1738 + }, + "trace_batches": 1738, + "trace_path": "/home/admin/cpfs/wjh/aituner/qwen235-fixed-pd-state-diagnosis-20260719-r1/frontier-optrace/tp8_ep8_mns64/frontier_metrics/qwen3_235b_a22b/online_serving/qwen235_fixed_pd_state_tp8_ep8_mns64/op_traces.jsonl" + } + } + }, + "simulator_internal_all_step_tp8_minus_tp4_ms": { + "attention": -7.461942625219683, + "dense_mlp_compute": -0.12314137296116046, + "ep_communication": 6.235402017662346, + "moe_compute": -32.45444243169211, + "moe_routing": -0.2861742505642797, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -20.06625468177714, + "tp_dp_communication": -6.571107062807271 + }, + "status": "PASS", + "verdict": "State-composition mismatch is insufficient: after reweighting Frontier to measured real decode composition, it still predicts TP8 faster." +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-optrace.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-optrace.md new file mode 100644 index 0000000..a93d3bc --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis-optrace.md @@ -0,0 +1,48 @@ +# Qwen235 Fixed-PD state diagnosis + +**Verdict:** State-composition mismatch is insufficient: after reweighting Frontier to measured real decode composition, it still predicts TP8 faster. + +| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage | +|---|---:|---:|---:| +| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% | +| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% | + +## Frontier internal component contrast over its own executed composition + +Decode-token-weighted over both pure-decode and mixed prefill/decode steps. + +| Component | TP8 - TP4 (ms/decoded token step) | +|---|---:| +| moe_compute | -32.4544 | +| total | -20.0663 | +| attention | -7.4619 | +| tp_dp_communication | -6.5711 | +| ep_communication | +6.2354 | +| moe_routing | -0.2862 | +| dense_mlp_compute | -0.1231 | +| pipeline_communication | +0.0000 | +| runtime_overhead | +0.0000 | + +## Frontier internal component contrast at real Running proxy + +Positive means TP8 slower; negative means Frontier gives TP8 an advantage. + +| Component | TP8 - TP4 (ms/step) | +|---|---:| +| moe_compute | -16.0581 | +| total | -8.5517 | +| ep_communication | +6.8248 | +| tp_dp_communication | -5.4010 | +| moe_routing | +0.3109 | +| attention | -0.2407 | +| dense_mlp_compute | +0.0018 | +| pipeline_communication | +0.0000 | +| runtime_overhead | +0.0000 | + +## Interpretation boundary + +- Real observed TPOT contrast: +6.9500 ms/token. +- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs. +- Frontier state outputs have no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors. +- For TP8 shared-domain sync, total is ledger stage_end-start (critical path); op categories are serialized work estimates and are not additive because overlap and lane-summed wait diagnostics are represented separately. +- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof. diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis.json new file mode 100644 index 0000000..579aa9c --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis.json @@ -0,0 +1,1361 @@ +{ + "proxy_matched": { + "configs": { + "tp4_ep1_mns64": { + "components_ms": { + "attention": 6.303822560333333, + "dense_mlp_compute": 0.17883562666666666, + "ep_communication": 0.0, + "moe_compute": 26.449277218333332, + "moe_routing": 1.2571787503333332, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 46.414892969, + "tp_dp_communication": 12.225778813333333 + }, + "coverage": 1.0, + "supported_samples": 45, + "total_samples": 45, + "unsupported": {} + }, + "tp8_ep8_mns64": { + "components_ms": { + "attention": 0.06450142600000001, + "dense_mlp_compute": 0.00192128, + "ep_communication": 0.07260416, + "moe_compute": 0.110544582, + "moe_routing": 0.016681946999999996, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.338857555, + "tp_dp_communication": 0.07260416 + }, + "coverage": 1.0, + "supported_samples": 21, + "total_samples": 21, + "unsupported": {} + } + }, + "method": "exact batch-size lookup; each simulator config is reweighted to its own frozen-real steady Running histogram; no interpolation", + "tp8_minus_tp4_ms": { + "attention": -6.239321134333333, + "dense_mlp_compute": -0.17691434666666667, + "ep_communication": 0.07260416, + "moe_compute": -26.338732636333333, + "moe_routing": -1.2404968033333332, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -46.076035414, + "tp_dp_communication": -12.153174653333334 + } + }, + "real": { + "tp4_ep1_mns64": { + "aggregate": { + "generation_tokens_per_s": { + "cv": 0.01219623132338527, + "distinct_n": 32, + "max": 217.3, + "mean": 204.7488888888889, + "min": 197.1, + "n": 45 + }, + "kv_percent": { + "cv": 0.09908502666308305, + "distinct_n": 2, + "max": 5.6, + "mean": 5.233333333333333, + "min": 4.5, + "n": 45 + }, + "running": { + "cv": 0.10101525445522107, + "distinct_n": 2, + "max": 5.0, + "mean": 4.666666666666667, + "min": 4.0, + "n": 45 + }, + "running_histogram": { + "4": 15, + "5": 30 + }, + "samples": 45, + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 45 + } + }, + "proxy_only": true, + "steady_rule": "prompt throughput >= 95% of 3276.8 tokens/s", + "trials": [ + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.01635665678820842, + "distinct_n": 13, + "max": 217.3, + "mean": 205.58666666666667, + "min": 202.4, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.6, + "mean": 5.6, + "min": 5.6, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial1/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.0, + "mean": 5.0, + "min": 5.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial1", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + }, + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.01101205870299425, + "distinct_n": 11, + "max": 206.3, + "mean": 204.18666666666667, + "min": 197.1, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.6, + "mean": 5.6, + "min": 5.6, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial2/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 5.0, + "mean": 5.0, + "min": 5.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial2", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + }, + { + "all_samples": 17, + "generation_tokens_per_s": { + "cv": 0.005467774668645034, + "distinct_n": 14, + "max": 206.0, + "mean": 204.47333333333333, + "min": 202.0, + "n": 15 + }, + "kv_percent": { + "cv": 0.0, + "distinct_n": 1, + "max": 4.5, + "mean": 4.5, + "min": 4.5, + "n": 15 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp4_ep1_mns64/trial3/logs/server.log", + "running": { + "cv": 0.0, + "distinct_n": 1, + "max": 4.0, + "mean": 4.0, + "min": 4.0, + "n": 15 + }, + "steady_samples": 15, + "trial": "trial3", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 15 + } + } + ] + }, + "tp8_ep8_mns64": { + "aggregate": { + "generation_tokens_per_s": { + "cv": 0.01855015540813005, + "distinct_n": 21, + "max": 415.7, + "mean": 406.5380952380952, + "min": 390.8, + "n": 21 + }, + "kv_percent": { + "cv": 0.03973151445031385, + "distinct_n": 3, + "max": 5.6, + "mean": 5.295238095238095, + "min": 4.8, + "n": 21 + }, + "running": { + "cv": 0.04297805843263526, + "distinct_n": 3, + "max": 13.0, + "mean": 12.238095238095237, + "min": 11.0, + "n": 21 + }, + "running_histogram": { + "11": 1, + "12": 14, + "13": 6 + }, + "samples": 21, + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 21 + } + }, + "proxy_only": true, + "steady_rule": "prompt throughput >= 95% of 6553.6 tokens/s", + "trials": [ + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.018873398222858283, + "distinct_n": 7, + "max": 415.7, + "mean": 406.87142857142857, + "min": 392.3, + "n": 7 + }, + "kv_percent": { + "cv": 0.03400298559320833, + "distinct_n": 2, + "max": 5.6, + "mean": 5.314285714285715, + "min": 5.2, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial1/logs/server.log", + "running": { + "cv": 0.036770670467074176, + "distinct_n": 2, + "max": 13.0, + "mean": 12.285714285714286, + "min": 12.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial1", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + }, + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.01869328742146201, + "distinct_n": 7, + "max": 412.9, + "mean": 406.2857142857143, + "min": 390.8, + "n": 7 + }, + "kv_percent": { + "cv": 0.04861017342390845, + "distinct_n": 3, + "max": 5.6, + "mean": 5.257142857142857, + "min": 4.8, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial2/logs/server.log", + "running": { + "cv": 0.052613364176465637, + "distinct_n": 3, + "max": 13.0, + "mean": 12.142857142857142, + "min": 11.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial2", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + }, + { + "all_samples": 9, + "generation_tokens_per_s": { + "cv": 0.018043345382112573, + "distinct_n": 7, + "max": 414.9, + "mean": 406.45714285714286, + "min": 393.0, + "n": 7 + }, + "kv_percent": { + "cv": 0.03400298559320833, + "distinct_n": 2, + "max": 5.6, + "mean": 5.314285714285715, + "min": 5.2, + "n": 7 + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd/real/tp8_ep8_mns64/trial3/logs/server.log", + "running": { + "cv": 0.036770670467074176, + "distinct_n": 2, + "max": 13.0, + "mean": 12.285714285714286, + "min": 12.0, + "n": 7 + }, + "steady_samples": 7, + "trial": "trial3", + "waiting": { + "cv": 0.0, + "distinct_n": 1, + "max": 0.0, + "mean": 0.0, + "min": 0.0, + "n": 7 + } + } + ] + } + }, + "reference": { + "observed_real_tp8_minus_tp4_ms": 6.949999999999999, + "real_tpot_ms": { + "tp4_ep1_mns64": 21.0442, + "tp8_ep8_mns64": 27.9942 + } + }, + "same_batch_contrasts": { + "1": { + "tp4_n": 54, + "tp8_minus_tp4_ms": { + "attention": -5.865362041999999, + "dense_mlp_compute": -0.17567923200000002, + "ep_communication": 0.0812872, + "moe_compute": -4.2460797470000005, + "moe_routing": -1.218856335, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -25.770368131999998, + "tp_dp_communication": -14.345677976 + }, + "tp8_n": 34 + }, + "10": { + "tp4_n": 38, + "tp8_minus_tp4_ms": { + "attention": -6.732672957, + "dense_mlp_compute": -0.17867904, + "ep_communication": 0.07260416, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -60.641066124000005, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 22 + }, + "11": { + "tp4_n": 42, + "tp8_minus_tp4_ms": { + "attention": -6.7796566579999995, + "dense_mlp_compute": -0.17867904, + "ep_communication": 0.07260415999999999, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -60.68804982500001, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 22 + }, + "12": { + "tp4_n": 46, + "tp8_minus_tp4_ms": { + "attention": -6.779656658, + "dense_mlp_compute": -0.17867904, + "ep_communication": 0.07260416, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -60.68804982500001, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 21 + }, + "13": { + "tp4_n": 46, + "tp8_minus_tp4_ms": { + "attention": -7.2421678100000015, + "dense_mlp_compute": -0.17867903999999998, + "ep_communication": 0.07260416, + "moe_compute": -41.06082887699999, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -61.150560977, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 22 + }, + "14": { + "tp4_n": 47, + "tp8_minus_tp4_ms": { + "attention": -7.248219534, + "dense_mlp_compute": -0.17867904000000004, + "ep_communication": 0.07260416, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -61.156612701, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 21 + }, + "15": { + "tp4_n": 663, + "tp8_minus_tp4_ms": { + "attention": -7.264528133999999, + "dense_mlp_compute": -0.17867904000000004, + "ep_communication": 0.07260416, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -61.172921300999995, + "tp_dp_communication": -11.4608282 + }, + "tp8_n": 21 + }, + "16": { + "tp4_n": 1053, + "tp8_minus_tp4_ms": { + "attention": -7.264528134, + "dense_mlp_compute": -0.17867904, + "ep_communication": 0.07260416, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -61.172921301, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 22 + }, + "2": { + "tp4_n": 52, + "tp8_minus_tp4_ms": { + "attention": -5.993161589230768, + "dense_mlp_compute": -0.17567923200000002, + "ep_communication": 0.07627888, + "moe_compute": -5.365670131, + "moe_routing": -1.222289295, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -26.05095133323077, + "tp_dp_communication": -13.370429966 + }, + "tp8_n": 32 + }, + "3": { + "tp4_n": 47, + "tp8_minus_tp4_ms": { + "attention": -6.04432608180851, + "dense_mlp_compute": -0.17567923200000002, + "ep_communication": 0.070864641, + "moe_compute": -9.925579423, + "moe_routing": -1.2278096929999998, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -29.95224206780851, + "tp_dp_communication": -12.649712279 + }, + "tp8_n": 31 + }, + "4": { + "tp4_n": 48, + "tp8_minus_tp4_ms": { + "attention": -6.228110533, + "dense_mlp_compute": -0.17567923200000002, + "ep_communication": 0.070864641, + "moe_compute": -9.925579423, + "moe_routing": -1.227809693, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -30.136026518999998, + "tp_dp_communication": -12.649712279 + }, + "tp8_n": 30 + }, + "5": { + "tp4_n": 34, + "tp8_minus_tp4_ms": { + "attention": -6.255439117, + "dense_mlp_compute": -0.17756006399999996, + "ep_communication": 0.070424161, + "moe_compute": -34.591127289999996, + "moe_routing": -1.247616504, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -54.109274413, + "tp_dp_communication": -11.907955599 + }, + "tp8_n": 27 + }, + "6": { + "tp4_n": 33, + "tp8_minus_tp4_ms": { + "attention": -6.298408004000001, + "dense_mlp_compute": -0.177560064, + "ep_communication": 0.07042416100000001, + "moe_compute": -34.591127289999996, + "moe_routing": -1.247616504, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -54.152243299999995, + "tp_dp_communication": -11.907955599 + }, + "tp8_n": 26 + }, + "7": { + "tp4_n": 34, + "tp8_minus_tp4_ms": { + "attention": -6.6722336250000005, + "dense_mlp_compute": -0.177560064, + "ep_communication": 0.070424161, + "moe_compute": -34.59112728999999, + "moe_routing": -1.247616504, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -54.52606892099999, + "tp_dp_communication": -11.907955599 + }, + "tp8_n": 26 + }, + "8": { + "tp4_n": 33, + "tp8_minus_tp4_ms": { + "attention": -6.6722336250000005, + "dense_mlp_compute": -0.177560064, + "ep_communication": 0.070424161, + "moe_compute": -34.591127289999996, + "moe_routing": -1.247616504, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -54.526068921, + "tp_dp_communication": -11.907955599 + }, + "tp8_n": 27 + }, + "9": { + "tp4_n": 32, + "tp8_minus_tp4_ms": { + "attention": -6.720033763, + "dense_mlp_compute": -0.17867904, + "ep_communication": 0.07260416, + "moe_compute": -41.060828877, + "moe_routing": -1.28066121, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -60.62842693, + "tp_dp_communication": -11.460828200000002 + }, + "tp8_n": 21 + } + }, + "schema": "qwen235-fixed-pd-state-diagnosis-v1", + "scope": { + "graph_observability": "Frontier ledger exposes additive execution components but no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors", + "real_proxy_limitation": "vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs", + "workload": "Fixed-PD 4096->256, 0.2 req/s/GPU, MNS64" + }, + "simulator": { + "tp4_ep1_mns64": { + "batch_support": { + "1": { + "components_ms": { + "attention": 5.918822475999999, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 4.277032336, + "moe_routing": 1.234618249, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.035006492999997, + "tp_dp_communication": 14.426965176 + }, + "n": 54 + }, + "10": { + "components_ms": { + "attention": 6.795322718, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.978072014000006, + "tp_dp_communication": 11.53343236 + }, + "n": 38 + }, + "11": { + "components_ms": { + "attention": 6.8427973799999995, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.025546676000005, + "tp_dp_communication": 11.53343236 + }, + "n": 42 + }, + "12": { + "components_ms": { + "attention": 6.84279738, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.025546676000005, + "tp_dp_communication": 11.53343236 + }, + "n": 46 + }, + "13": { + "components_ms": { + "attention": 7.310070996000001, + "dense_mlp_compute": 0.18060031999999998, + "ep_communication": 0.0, + "moe_compute": 41.171373458999994, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.492820292, + "tp_dp_communication": 11.53343236 + }, + "n": 46 + }, + "14": { + "components_ms": { + "attention": 7.316185658, + "dense_mlp_compute": 0.18060032000000004, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.498934954, + "tp_dp_communication": 11.53343236 + }, + "n": 47 + }, + "15": { + "components_ms": { + "attention": 7.332662654999999, + "dense_mlp_compute": 0.18060032000000004, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.515411951, + "tp_dp_communication": 11.533432359999999 + }, + "n": 663 + }, + "16": { + "components_ms": { + "attention": 7.332662655, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 61.515411951000004, + "tp_dp_communication": 11.53343236 + }, + "n": 1053 + }, + "2": { + "components_ms": { + "attention": 6.048001424230768, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 5.39662272, + "moe_routing": 1.237987209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 26.30688845523077, + "tp_dp_communication": 13.446708846 + }, + "n": 52 + }, + "3": { + "components_ms": { + "attention": 6.09964707180851, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 9.978847425, + "moe_routing": 1.2434858329999998, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 30.220125505808507, + "tp_dp_communication": 12.72057692 + }, + "n": 47 + }, + "4": { + "components_ms": { + "attention": 6.285367077, + "dense_mlp_compute": 0.177568256, + "ep_communication": 0.0, + "moe_compute": 9.978847425, + "moe_routing": 1.243485833, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 30.405845511, + "tp_dp_communication": 12.72057692 + }, + "n": 48 + }, + "5": { + "components_ms": { + "attention": 6.313050302, + "dense_mlp_compute": 0.17946931199999996, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.419416698, + "tp_dp_communication": 11.97837976 + }, + "n": 34 + }, + "6": { + "components_ms": { + "attention": 6.356479957, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.462846352999996, + "tp_dp_communication": 11.97837976 + }, + "n": 33 + }, + "7": { + "components_ms": { + "attention": 6.734075413, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.68449211499999, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.84044180899999, + "tp_dp_communication": 11.97837976 + }, + "n": 34 + }, + "8": { + "components_ms": { + "attention": 6.734075413, + "dense_mlp_compute": 0.179469312, + "ep_communication": 0.0, + "moe_compute": 34.684492115, + "moe_routing": 1.264025209, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 54.840441809, + "tp_dp_communication": 11.97837976 + }, + "n": 33 + }, + "9": { + "components_ms": { + "attention": 6.782511044, + "dense_mlp_compute": 0.18060032, + "ep_communication": 0.0, + "moe_compute": 41.171373459, + "moe_routing": 1.297343157, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.96526034, + "tp_dp_communication": 11.53343236 + }, + "n": 32 + } + }, + "decode_batch_size": { + "cv": 0.3076467178753811, + "distinct_n": 16, + "max": 16.0, + "mean": 13.537037037037036, + "min": 1.0, + "n": 2430 + }, + "decode_bearing_rows": 2430, + "decode_token_weighted_all_step_components_ms": { + "attention": 13.842547047292292, + "dense_mlp_compute": 0.2927251518073871, + "ep_communication": 0.0, + "moe_compute": 45.47878637854458, + "moe_routing": 1.8023597240381213, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 74.222927382152, + "tp_dp_communication": 12.806509080469617 + }, + "decode_token_weighted_mixed_step_share": 0.05481076151390789, + "mixed_prefill_decode_rows": 128, + "pure_decode_rows": 2302, + "pure_decode_token_weighted_components_ms": { + "attention": 7.263972797140871, + "dense_mlp_compute": 0.18052079084832112, + "ep_communication": 0.0, + "moe_compute": 40.472043520877655, + "moe_routing": 1.2955265717923583, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 60.782085996374114, + "tp_dp_communication": 11.57002231571491 + }, + "scorer_equivalence": { + "base_metrics": { + "e2e_mean_ms": 19261.684938205322, + "e2e_p50_ms": 19681.421068277567, + "e2e_p90_ms": 19712.678973231064, + "e2e_p95_ms": 19717.18006692948, + "slos": { + "tpot_100ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_150ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_180ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_50ms": { + "feasible": false, + "pass_rate": 0.0, + "passed": 0 + } + }, + "tpot_mean_ms": 74.22292738142716, + "tpot_p50_ms": 75.8976524911963, + "tpot_p90_ms": 75.8976524912086, + "tpot_p95_ms": 75.89765249120967, + "ttft_mean_ms": 334.8384559413977, + "ttft_p50_ms": 334.75193988430976, + "ttft_p90_ms": 359.7779859328796, + "ttft_p95_ms": 363.2786816744229 + }, + "request_metrics_byte_identical": true, + "request_metrics_sha256": "01ed41e9babf2eb71286e93d8f4986dda9820f92212c975ed9411a95a5f8758f" + }, + "stage_rows": 2431 + }, + "tp8_ep8_mns64": { + "batch_support": { + "1": { + "components_ms": { + "attention": 0.053460434, + "dense_mlp_compute": 0.0018890240000000002, + "ep_communication": 0.0812872, + "moe_compute": 0.030952589000000003, + "moe_routing": 0.015761914, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.264638361, + "tp_dp_communication": 0.0812872 + }, + "n": 34 + }, + "10": { + "components_ms": { + "attention": 0.062649761, + "dense_mlp_compute": 0.0019212800000000003, + "ep_communication": 0.07260416, + "moe_compute": 0.11054458199999999, + "moe_routing": 0.016681947, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.33700589, + "tp_dp_communication": 0.07260416 + }, + "n": 22 + }, + "11": { + "components_ms": { + "attention": 0.063140722, + "dense_mlp_compute": 0.00192128, + "ep_communication": 0.07260415999999999, + "moe_compute": 0.11054458200000002, + "moe_routing": 0.016681947, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.337496851, + "tp_dp_communication": 0.07260415999999999 + }, + "n": 22 + }, + "12": { + "components_ms": { + "attention": 0.06314072200000001, + "dense_mlp_compute": 0.00192128, + "ep_communication": 0.07260416, + "moe_compute": 0.110544582, + "moe_routing": 0.016681946999999996, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.337496851, + "tp_dp_communication": 0.07260416 + }, + "n": 21 + }, + "13": { + "components_ms": { + "attention": 0.067903186, + "dense_mlp_compute": 0.00192128, + "ep_communication": 0.07260416, + "moe_compute": 0.110544582, + "moe_routing": 0.016681946999999996, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.342259315, + "tp_dp_communication": 0.07260416 + }, + "n": 22 + }, + "14": { + "components_ms": { + "attention": 0.067966124, + "dense_mlp_compute": 0.0019212799999999998, + "ep_communication": 0.07260416, + "moe_compute": 0.110544582, + "moe_routing": 0.016681947, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.342322253, + "tp_dp_communication": 0.07260416 + }, + "n": 21 + }, + "15": { + "components_ms": { + "attention": 0.068134521, + "dense_mlp_compute": 0.0019212799999999998, + "ep_communication": 0.07260416, + "moe_compute": 0.11054458200000002, + "moe_routing": 0.016681947, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.34249065, + "tp_dp_communication": 0.07260416 + }, + "n": 21 + }, + "16": { + "components_ms": { + "attention": 0.068134521, + "dense_mlp_compute": 0.00192128, + "ep_communication": 0.07260416, + "moe_compute": 0.110544582, + "moe_routing": 0.016681947, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.34249065, + "tp_dp_communication": 0.07260416 + }, + "n": 22 + }, + "17": { + "components_ms": { + "attention": 0.070769479, + "dense_mlp_compute": 0.0019433600000000003, + "ep_communication": 0.071955199, + "moe_compute": 0.15567622299999997, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.38983089699999995, + "tp_dp_communication": 0.071955199 + }, + "n": 31 + }, + "18": { + "components_ms": { + "attention": 0.07076947899999998, + "dense_mlp_compute": 0.00194336, + "ep_communication": 0.071955199, + "moe_compute": 0.155676223, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.38983089699999995, + "tp_dp_communication": 0.071955199 + }, + "n": 30 + }, + "19": { + "components_ms": { + "attention": 0.070769479, + "dense_mlp_compute": 0.0019433599999999999, + "ep_communication": 0.071955199, + "moe_compute": 0.155676223, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.389830897, + "tp_dp_communication": 0.071955199 + }, + "n": 32 + }, + "2": { + "components_ms": { + "attention": 0.054839835, + "dense_mlp_compute": 0.001889024, + "ep_communication": 0.07627888, + "moe_compute": 0.030952589, + "moe_routing": 0.015697914, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.255937122, + "tp_dp_communication": 0.07627888 + }, + "n": 32 + }, + "20": { + "components_ms": { + "attention": 0.070769479, + "dense_mlp_compute": 0.0019433600000000003, + "ep_communication": 0.071955199, + "moe_compute": 0.15567622299999997, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.38983089699999995, + "tp_dp_communication": 0.071955199 + }, + "n": 33 + }, + "21": { + "components_ms": { + "attention": 0.074295324, + "dense_mlp_compute": 0.00194336, + "ep_communication": 0.071955199, + "moe_compute": 0.155676223, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.393356742, + "tp_dp_communication": 0.071955199 + }, + "n": 37 + }, + "22": { + "components_ms": { + "attention": 0.074295324, + "dense_mlp_compute": 0.00194336, + "ep_communication": 0.07195519899999998, + "moe_compute": 0.155676223, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.393356742, + "tp_dp_communication": 0.07195519899999998 + }, + "n": 51 + }, + "23": { + "components_ms": { + "attention": 0.074295324, + "dense_mlp_compute": 0.0019433599999999999, + "ep_communication": 0.071955199, + "moe_compute": 0.155676223, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.393356742, + "tp_dp_communication": 0.071955199 + }, + "n": 906 + }, + "24": { + "components_ms": { + "attention": 0.074295324, + "dense_mlp_compute": 0.0019433600000000003, + "ep_communication": 0.071955199, + "moe_compute": 0.155676223, + "moe_routing": 0.017531437, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.393356742, + "tp_dp_communication": 0.071955199 + }, + "n": 84 + }, + "3": { + "components_ms": { + "attention": 0.05532099, + "dense_mlp_compute": 0.0018890240000000002, + "ep_communication": 0.070864641, + "moe_compute": 0.053268002, + "moe_routing": 0.015676139999999998, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.26788343800000003, + "tp_dp_communication": 0.070864641 + }, + "n": 31 + }, + "4": { + "components_ms": { + "attention": 0.057256544, + "dense_mlp_compute": 0.001889024, + "ep_communication": 0.070864641, + "moe_compute": 0.053268002, + "moe_routing": 0.015676139999999998, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.26981899200000004, + "tp_dp_communication": 0.070864641 + }, + "n": 30 + }, + "5": { + "components_ms": { + "attention": 0.057611185, + "dense_mlp_compute": 0.001909248, + "ep_communication": 0.070424161, + "moe_compute": 0.09336482499999998, + "moe_routing": 0.016408705, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.31014228499999996, + "tp_dp_communication": 0.070424161 + }, + "n": 27 + }, + "6": { + "components_ms": { + "attention": 0.058071952999999996, + "dense_mlp_compute": 0.001909248, + "ep_communication": 0.07042416100000001, + "moe_compute": 0.093364825, + "moe_routing": 0.016408705, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.310603053, + "tp_dp_communication": 0.07042416100000001 + }, + "n": 26 + }, + "7": { + "components_ms": { + "attention": 0.061841787999999995, + "dense_mlp_compute": 0.001909248, + "ep_communication": 0.070424161, + "moe_compute": 0.093364825, + "moe_routing": 0.016408705, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.314372888, + "tp_dp_communication": 0.070424161 + }, + "n": 26 + }, + "8": { + "components_ms": { + "attention": 0.061841787999999995, + "dense_mlp_compute": 0.0019092480000000003, + "ep_communication": 0.070424161, + "moe_compute": 0.093364825, + "moe_routing": 0.016408705, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.314372888, + "tp_dp_communication": 0.070424161 + }, + "n": 27 + }, + "9": { + "components_ms": { + "attention": 0.062477281, + "dense_mlp_compute": 0.0019212800000000003, + "ep_communication": 0.07260416, + "moe_compute": 0.11054458199999999, + "moe_routing": 0.016681947, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.33683341, + "tp_dp_communication": 0.07260416 + }, + "n": 21 + } + }, + "decode_batch_size": { + "cv": 0.3602756967919564, + "distinct_n": 24, + "max": 24.0, + "mean": 18.93782383419689, + "min": 1.0, + "n": 1737 + }, + "decode_bearing_rows": 1737, + "decode_token_weighted_all_step_components_ms": { + "attention": 0.14751953504742363, + "dense_mlp_compute": 0.0036978849298677607, + "ep_communication": 0.082097370873993, + "moe_compute": 0.20779839324350205, + "moe_routing": 0.02642429565140599, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.5496348506201855, + "tp_dp_communication": 0.082097370873993 + }, + "decode_token_weighted_mixed_step_share": 0.08077215382276942, + "mixed_prefill_decode_rows": 128, + "pure_decode_rows": 1609, + "pure_decode_token_weighted_components_ms": { + "attention": 0.07291164903783319, + "dense_mlp_compute": 0.0019404539326675046, + "ep_communication": 0.07197841798710232, + "moe_compute": 0.14992182626155828, + "moe_routing": 0.017426514274852836, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": 0.38615727948111644, + "tp_dp_communication": 0.07197841798710232 + }, + "scorer_equivalence": { + "base_metrics": { + "e2e_mean_ms": 14056.464119438264, + "e2e_p50_ms": 14650.653983203298, + "e2e_p90_ms": 14674.997135818061, + "e2e_p95_ms": 14677.859760866291, + "slos": { + "tpot_100ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_150ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_180ms": { + "feasible": true, + "pass_rate": 1.0, + "passed": 129 + }, + "tpot_50ms": { + "feasible": false, + "pass_rate": 0.14728682170542637, + "passed": 19 + } + }, + "tpot_mean_ms": 54.15667269956665, + "tpot_p50_ms": 56.52350191837838, + "tpot_p90_ms": 56.52350850045728, + "tpot_p95_ms": 56.523546100039816, + "ttft_mean_ms": 246.51258104876885, + "ttft_p50_ms": 246.7781280760306, + "ttft_p90_ms": 263.0087113349084, + "ttft_p95_ms": 264.77576248352364 + }, + "request_metrics_byte_identical": true, + "request_metrics_sha256": "51a848c4c12eacabbc72a0f6a8b5c1863e8b624b316b7ab55341e3fe5a3a753b" + }, + "stage_rows": 1738 + } + }, + "simulator_internal_all_step_tp8_minus_tp4_ms": { + "attention": -13.695027512244868, + "dense_mlp_compute": -0.28902726687751934, + "ep_communication": 0.082097370873993, + "moe_compute": -45.270987985301076, + "moe_routing": -1.7759354283867153, + "pipeline_communication": 0.0, + "runtime_overhead": 0.0, + "total": -73.67329253153181, + "tp_dp_communication": -12.724411709595623 + }, + "status": "PASS", + "verdict": "Active-batch-count mismatch alone is insufficient: after exact reweighting to each config's real Running histogram, Frontier still predicts TP8 faster." +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis.md new file mode 100644 index 0000000..381adb4 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q235-state-diag/state-diagnosis.md @@ -0,0 +1,47 @@ +# Qwen235 Fixed-PD state diagnosis + +**Verdict:** Active-batch-count mismatch alone is insufficient: after exact reweighting to each config's real Running histogram, Frontier still predicts TP8 faster. + +| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage | +|---|---:|---:|---:| +| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% | +| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% | + +## Frontier internal component contrast over its own executed composition + +Decode-token-weighted over both pure-decode and mixed prefill/decode steps. + +| Component | TP8 - TP4 (ms/decoded token step) | +|---|---:| +| total | -73.6733 | +| moe_compute | -45.2710 | +| attention | -13.6950 | +| tp_dp_communication | -12.7244 | +| moe_routing | -1.7759 | +| dense_mlp_compute | -0.2890 | +| ep_communication | +0.0821 | +| pipeline_communication | +0.0000 | +| runtime_overhead | +0.0000 | + +## Frontier internal component contrast at real Running proxy + +Positive means TP8 slower; negative means Frontier gives TP8 an advantage. + +| Component | TP8 - TP4 (ms/step) | +|---|---:| +| total | -46.0760 | +| moe_compute | -26.3387 | +| tp_dp_communication | -12.1532 | +| attention | -6.2393 | +| moe_routing | -1.2405 | +| dense_mlp_compute | -0.1769 | +| ep_communication | +0.0726 | +| pipeline_communication | +0.0000 | +| runtime_overhead | +0.0000 | + +## Interpretation boundary + +- Real observed TPOT contrast: +6.9500 ms/token. +- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs. +- Frontier ledger exposes additive execution components but no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors. +- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof. diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-admission-diag/result.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-admission-diag/result.json new file mode 100644 index 0000000..b81ebf0 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-admission-diag/result.json @@ -0,0 +1,173 @@ +{ + "boundary": "This identifies where the selection error is created, but the frozen real logs do not attribute service-time overprediction to one operator.", + "cells": { + "tp2_mns64": { + "arrival_rate_requests_per_s": 2.25, + "mns": 64, + "real": { + "e2e_ms": 3354.8816545874283, + "periodic_queue": { + "running_max": 10, + "running_mean": 7.257142857142857, + "samples": 35, + "waiting_max": 0, + "waiting_mean": 0.0 + }, + "required_slots_upper_bound": 7.548483722821714, + "slot_margin_lower_bound": 56.451516277178285, + "tpot_ms": 12.393283087182931, + "ttft_ms": 194.56193759030603 + }, + "simulator": { + "first_scheduling_delay_ms": 38.22329108338011, + "queue_free_ttft_ms": 191.02937373774816, + "request_execution_ms": 25761.91467886552, + "required_slots": 57.96430802744742, + "slot_margin": 6.035691972552577, + "tpot_ms": 100.27798158873513, + "ttft_ms": 229.2526648211283, + "ttft_queue_fraction": 0.16672997503956338 + }, + "tp": 2, + "tpot_overprediction_ratio": 8.091316956395687 + }, + "tp4_mns16": { + "arrival_rate_requests_per_s": 4.5, + "mns": 16, + "real": { + "e2e_ms": 3252.7993474094965, + "periodic_queue": { + "running_max": 16, + "running_mean": 15.166666666666666, + "samples": 18, + "waiting_max": 0, + "waiting_mean": 0.0 + }, + "required_slots_upper_bound": 14.637597063342735, + "slot_margin_lower_bound": 1.362402936657265, + "tpot_ms": 12.267737219892378, + "ttft_ms": 124.49453797538061 + }, + "simulator": { + "first_scheduling_delay_ms": 64906.25631809643, + "queue_free_ttft_ms": 134.74066995065758, + "request_execution_ms": 12108.627705569606, + "required_slots": 54.488824675063235, + "slot_margin": -38.488824675063235, + "tpot_ms": 46.95641974752248, + "ttft_ms": 65040.99698804708, + "ttft_queue_fraction": 0.9979283732385681 + }, + "tp": 4, + "tpot_overprediction_ratio": 3.8276349505907024 + }, + "tp4_mns32": { + "arrival_rate_requests_per_s": 4.5, + "mns": 32, + "real": { + "e2e_ms": 3227.015463910283, + "periodic_queue": { + "running_max": 16, + "running_mean": 14.944444444444445, + "samples": 18, + "waiting_max": 0, + "waiting_mean": 0.0 + }, + "required_slots_upper_bound": 14.521569587596273, + "slot_margin_lower_bound": 17.478430412403725, + "tpot_ms": 12.171863019540417, + "ttft_ms": 123.15859724408143 + }, + "simulator": { + "first_scheduling_delay_ms": 27180.676177927482, + "queue_free_ttft_ms": 135.62147333612225, + "request_execution_ms": 14643.080360640608, + "required_slots": 65.89386162288272, + "slot_margin": -33.893861622882724, + "tpot_ms": 56.89199563649036, + "ttft_ms": 27316.2976512636, + "ttft_queue_fraction": 0.9950351444010626 + }, + "tp": 4, + "tpot_overprediction_ratio": 4.6740581573385525 + }, + "tp4_mns64": { + "arrival_rate_requests_per_s": 4.5, + "mns": 64, + "real": { + "e2e_ms": 3258.202659738964, + "periodic_queue": { + "running_max": 17, + "running_mean": 14.833333333333334, + "samples": 18, + "waiting_max": 0, + "waiting_mean": 0.0 + }, + "required_slots_upper_bound": 14.661911968825338, + "slot_margin_lower_bound": 49.338088031174664, + "tpot_ms": 12.29409478300949, + "ttft_ms": 123.17715882596677 + }, + "simulator": { + "first_scheduling_delay_ms": 6367.273779218055, + "queue_free_ttft_ms": 136.7304324624105, + "request_execution_ms": 17736.957363112386, + "required_slots": 79.81630813400574, + "slot_margin": -15.816308134005737, + "tpot_ms": 69.02049776725552, + "ttft_ms": 6504.004211680465, + "ttft_queue_fraction": 0.9789774993969319 + }, + "tp": 4, + "tpot_overprediction_ratio": 5.614117914776633 + } + }, + "contrasts": { + "observed_sim_ttft_tp4_minus_tp2_ms": 27087.044986442474, + "queue_free_sim_ttft_tp4_minus_tp2_ms": -55.40790040162591, + "real_ttft_tp4_minus_tp2_ms": -71.4033403462246 + }, + "schema": "qwen30-fixed-pd-ttft-admission-diagnosis-v1", + "selection": { + "agreement": 0.5909090909090909, + "concordant_pairs": 39, + "discordant_pairs": 27, + "informative_pairs": 66, + "real_ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp4_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.5797674051508896, + "sim_ranking": [ + "tp2_mns64", + "tp1_mns64", + "tp4_mns64", + "tp1_mns32", + "tp2_mns32", + "tp4_mns32", + "tp4_mns16", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp2_mns64", + "tied_pairs": 0, + "winner_match": false + }, + "status": "PASS", + "verdict": "Frontier overpredicts decode service time, so TP4's higher global arrival rate crosses the MNS admission cap only in simulation. The resulting first-scheduling queue reverses the TP topology ranking; without that queue Frontier itself predicts TP4 faster than TP2, matching real hardware.", + "workload": "Qwen3-30B Fixed-PD 4096->256 at 1.125 req/s/GPU" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-admission-diag/result.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-admission-diag/result.md new file mode 100644 index 0000000..dd90030 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-admission-diag/result.md @@ -0,0 +1,18 @@ +# Qwen30 Fixed-PD TTFT admission diagnosis + +**Verdict:** Frontier overpredicts decode service time, so TP4's higher global arrival rate crosses the MNS admission cap only in simulation. The resulting first-scheduling queue reverses the TP topology ranking; without that queue Frontier itself predicts TP4 faster than TP2, matching real hardware. + +| Config | Sim TPOT / real | Sim required slots / MNS | Real slots upper bound / MNS | Sim first-schedule wait | Real waiting max | +|---|---:|---:|---:|---:|---:| +| tp2_mns64 | 8.09x | 58.0/64 | 7.5/64 | 38.2 ms | 0 | +| tp4_mns16 | 3.83x | 54.5/16 | 14.6/16 | 64906.3 ms | 0 | +| tp4_mns32 | 4.67x | 65.9/32 | 14.5/32 | 27180.7 ms | 0 | +| tp4_mns64 | 5.61x | 79.8/64 | 14.7/64 | 6367.3 ms | 0 | + +## TP4/MNS32 minus TP2/MNS64 TTFT + +- Observed Frontier: +27087.0 ms. +- Frontier without first-scheduling queue: -55.4 ms. +- Real hardware: -71.4 ms. + +**Boundary:** This identifies where the selection error is created, but the frozen real logs do not attribute service-time overprediction to one operator. diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/analysis-launch.log b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/analysis-launch.log new file mode 100644 index 0000000..2614c49 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/analysis-launch.log @@ -0,0 +1,2 @@ +Q30_ANALYSIS_ECHO source_real=/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6 source_sim=/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim cases=trace-po,fixed-pd,fixed-po requests=129 configs=12 trials=3 metrics=mean,p90(TTFT,TPOT-if-OSL-gt-1,E2E) gpu_cost=0 output=/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6 +Q30_ANALYSIS_COMPLETE diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.json new file mode 100644 index 0000000..dbad285 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.json @@ -0,0 +1,453 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "tpot_ms", + "e2e_ms" + ], + "cells": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "mean": 2624.2011811364387, + "p90": 2624.2011811379484 + }, + "tpot_ms": { + "mean": 19.350639886513438, + "p90": 19.3506398865253 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns16/tp1/result.json" + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "mean": 2624.2011811364387, + "p90": 2624.2011811379484 + }, + "tpot_ms": { + "mean": 19.350639886513438, + "p90": 19.3506398865253 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns32/tp1/result.json" + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "mean": 2624.2011811364387, + "p90": 2624.2011811379484 + }, + "tpot_ms": { + "mean": 19.350639886513438, + "p90": 19.3506398865253 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns64/tp1/result.json" + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "mean": 2624.2011811364387, + "p90": 2624.2011811379484 + }, + "tpot_ms": { + "mean": 19.350639886513438, + "p90": 19.3506398865253 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp1_mns8/tp1/result.json" + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "mean": 2808.3844446026824, + "p90": 2817.9486849812747 + }, + "tpot_ms": { + "mean": 21.06177881019329, + "p90": 21.09350467849898 + }, + "ttft_ms": { + "mean": 133.53853570813533, + "p90": 142.1523942165379 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns16/tp2/result.json" + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "mean": 2808.3844446026824, + "p90": 2817.9486849812747 + }, + "tpot_ms": { + "mean": 21.06177881019329, + "p90": 21.09350467849898 + }, + "ttft_ms": { + "mean": 133.53853570813533, + "p90": 142.1523942165379 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns32/tp2/result.json" + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "mean": 2808.3844446026824, + "p90": 2817.9486849812747 + }, + "tpot_ms": { + "mean": 21.06177881019329, + "p90": 21.09350467849898 + }, + "ttft_ms": { + "mean": 133.53853570813533, + "p90": 142.1523942165379 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns64/tp2/result.json" + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "mean": 2808.3844446026824, + "p90": 2817.9486849812747 + }, + "tpot_ms": { + "mean": 21.06177881019329, + "p90": 21.09350467849898 + }, + "ttft_ms": { + "mean": 133.53853570813533, + "p90": 142.1523942165379 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp2_mns8/tp2/result.json" + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "mean": 3752.684495467103, + "p90": 3794.6349827560653 + }, + "tpot_ms": { + "mean": 28.70502970984996, + "p90": 28.949734665706984 + }, + "ttft_ms": { + "mean": 107.14572231615826, + "p90": 118.03470639992497 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns16/tp4/result.json" + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "mean": 3752.684495467103, + "p90": 3794.6349827560653 + }, + "tpot_ms": { + "mean": 28.70502970984996, + "p90": 28.949734665706984 + }, + "ttft_ms": { + "mean": 107.14572231615826, + "p90": 118.03470639992497 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns32/tp4/result.json" + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "mean": 3752.684495467103, + "p90": 3794.6349827560653 + }, + "tpot_ms": { + "mean": 28.70502970984996, + "p90": 28.949734665706984 + }, + "ttft_ms": { + "mean": 107.14572231615826, + "p90": 118.03470639992497 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns64/tp4/result.json" + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "mean": 3752.684495467103, + "p90": 3794.6349827560653 + }, + "tpot_ms": { + "mean": 28.70502970984996, + "p90": 28.949734665706984 + }, + "ttft_ms": { + "mean": 107.14572231615826, + "p90": 118.03470639992497 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd/runs/tp4_mns8/tp4/result.json" + } + }, + "prefill_only": false, + "real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/fixed-pd-real-audit.json", + "schema": "qwen30-latency-case-frontier-real-comparison-v1", + "selection": { + "e2e_ms:mean": { + "agreement": 0.0, + "concordant_pairs": 0, + "discordant_pairs": 48, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns16", + "tp2_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.26921939465653383, + "sim_ranking": [ + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8" + ], + "sim_winner": "tp1_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "e2e_ms:p90": { + "agreement": 0.0, + "concordant_pairs": 0, + "discordant_pairs": 48, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp2_mns64", + "tp1_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.25730483004000526, + "sim_ranking": [ + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8" + ], + "sim_winner": "tp1_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "tpot_ms:mean": { + "agreement": 0.0, + "concordant_pairs": 0, + "discordant_pairs": 48, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns16", + "tp2_mns32", + "tp2_mns8", + "tp2_mns64", + "tp1_mns64", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.13962124906187545, + "sim_ranking": [ + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8" + ], + "sim_winner": "tp1_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "tpot_ms:p90": { + "agreement": 0.0, + "concordant_pairs": 0, + "discordant_pairs": 48, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns32", + "tp2_mns16", + "tp2_mns8", + "tp2_mns64", + "tp1_mns16", + "tp1_mns64", + "tp1_mns32", + "tp1_mns8" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.13357103696654363, + "sim_ranking": [ + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8" + ], + "sim_winner": "tp1_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "ttft_ms:mean": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns16", + "tp4_mns64", + "tp4_mns32", + "tp4_mns8", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64" + ], + "real_winner": "tp4_mns16", + "selected_config_real_regret": 0.0, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": true + }, + "ttft_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns16", + "tp4_mns64", + "tp4_mns8", + "tp4_mns32", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64", + "tp1_mns16" + ], + "real_winner": "tp4_mns16", + "selected_config_real_regret": 0.0, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": true + } + }, + "sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-pd" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.md new file mode 100644 index 0000000..94484d6 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.md @@ -0,0 +1,10 @@ +# Qwen30 Frontier vs real latency selection + +| Objective | Frontier | Real | Match | Regret | Pairwise | +|---|---|---|---:|---:|---:| +| ttft_ms:mean | tp4_mns16 | tp4_mns16 | yes | 0.0% | 100.0% | +| ttft_ms:p90 | tp4_mns16 | tp4_mns16 | yes | 0.0% | 100.0% | +| tpot_ms:mean | tp1_mns16 | tp4_mns32 | no | 14.0% | 0.0% | +| tpot_ms:p90 | tp1_mns16 | tp4_mns8 | no | 13.4% | 0.0% | +| e2e_ms:mean | tp1_mns16 | tp4_mns32 | no | 26.9% | 0.0% | +| e2e_ms:p90 | tp1_mns16 | tp4_mns8 | no | 25.7% | 0.0% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.stdout.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.stdout.json new file mode 100644 index 0000000..1aa57e0 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-comparison.stdout.json @@ -0,0 +1 @@ +{"e2e_ms:mean": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns32", "selected_config_real_regret": 0.26921939465653383, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "e2e_ms:p90": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.25730483004000526, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "tpot_ms:mean": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns32", "tp2_mns8", "tp2_mns64", "tp1_mns64", "tp1_mns32", "tp1_mns16", "tp1_mns8"], "real_winner": "tp4_mns32", "selected_config_real_regret": 0.13962124906187545, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "tpot_ms:p90": {"agreement": 0.0, "concordant_pairs": 0, "discordant_pairs": 48, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns32", "tp2_mns16", "tp2_mns8", "tp2_mns64", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.13357103696654363, "sim_ranking": ["tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8"], "sim_winner": "tp1_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:mean": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns32", "tp4_mns8", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns16", "selected_config_real_regret": 0.0, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": true}, "ttft_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns8", "tp4_mns32", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64", "tp1_mns16"], "real_winner": "tp4_mns16", "selected_config_real_regret": 0.0, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": true}} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.json new file mode 100644 index 0000000..3cab437 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.json @@ -0,0 +1,1298 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "tpot_ms", + "e2e_ms" + ], + "case_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd", + "configs": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 767.1236270730016, + "pooled_p90_ms": 766.9662160333246, + "pooled_samples": 387, + "trial_mean_of_means_ms": 767.1236270730016, + "trial_stddev_of_means_ms": 3.640625296359905 + }, + "tpot_ms": { + "pooled_mean_ms": 4.8488702313524765, + "pooled_p90_ms": 4.838337488362463, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.8488702313524765, + "trial_stddev_of_means_ms": 0.02240869837530002 + }, + "ttft_ms": { + "pooled_mean_ms": 151.28173231440826, + "pooled_p90_ms": 152.70179708022624, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.28173231440823, + "trial_stddev_of_means_ms": 0.9864045576075636 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 767.2682947591092, + "p90_ms": 803.2620819285512, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.854438677765834, + "p90_ms": 5.140277842155064, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 150.72045104159403, + "p90_ms": 151.41086804214865, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 763.412824322022, + "p90_ms": 763.9202950522304, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.824202359768954, + "p90_ms": 4.827735795250793, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 150.70405107013187, + "p90_ms": 151.4174829935655, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 770.6897621378736, + "p90_ms": 806.9713360164315, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.867969656522642, + "p90_ms": 5.155875693134025, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.42069483149882, + "p90_ms": 153.3631639322266, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 767.2448034298387, + "pooled_p90_ms": 768.11003300827, + "pooled_samples": 387, + "trial_mean_of_means_ms": 767.2448034298387, + "trial_stddev_of_means_ms": 2.5069380140145388 + }, + "tpot_ms": { + "pooled_mean_ms": 4.848521902720499, + "pooled_p90_ms": 4.846741385779923, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.848521902720499, + "trial_stddev_of_means_ms": 0.016759800320354026 + }, + "ttft_ms": { + "pooled_mean_ms": 151.4453163664687, + "pooled_p90_ms": 152.51897496636957, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.4453163664687, + "trial_stddev_of_means_ms": 0.9359392091301725 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 766.5741731901834, + "p90_ms": 802.9849890153855, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.851220601996876, + "p90_ms": 5.135506000113123, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 150.43522373325544, + "p90_ms": 151.12131391651928, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 765.1413833206274, + "p90_ms": 766.2973250262439, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.830576509172996, + "p90_ms": 4.836877464228786, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.61754613085253, + "p90_ms": 152.396842953749, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 770.0188537787054, + "p90_ms": 805.9141940902919, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.863768596991625, + "p90_ms": 5.147916850575253, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.2831792352981, + "p90_ms": 153.04890798870474, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 766.7927716823189, + "pooled_p90_ms": 766.5371090406552, + "pooled_samples": 387, + "trial_mean_of_means_ms": 766.7927716823189, + "trial_stddev_of_means_ms": 2.196495670572406 + }, + "tpot_ms": { + "pooled_mean_ms": 4.84126870357575, + "pooled_p90_ms": 4.839739204919714, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.84126870357575, + "trial_stddev_of_means_ms": 0.017335823614573528 + }, + "ttft_ms": { + "pooled_mean_ms": 151.91121668752177, + "pooled_p90_ms": 152.6928918901831, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.91121668752177, + "trial_stddev_of_means_ms": 0.09417854078816222 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 767.9369603139312, + "p90_ms": 773.8644760102034, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.851052223623522, + "p90_ms": 4.915137134499731, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.8117881212167, + "p90_ms": 152.64031395781785, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 764.260392307571, + "p90_ms": 765.39497601334, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.8212526970581795, + "p90_ms": 4.825043646017398, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.92278562106424, + "p90_ms": 152.61299489066005, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 768.1809624254545, + "p90_ms": 804.694763966836, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.85150119004555, + "p90_ms": 5.139402172520874, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.9990763202844, + "p90_ms": 152.9452829854563, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 768.6282821211396, + "pooled_p90_ms": 768.3246430242434, + "pooled_samples": 387, + "trial_mean_of_means_ms": 768.6282821211395, + "trial_stddev_of_means_ms": 2.3163990743969975 + }, + "tpot_ms": { + "pooled_mean_ms": 4.856374974269665, + "pooled_p90_ms": 4.857163118220514, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.856374974269665, + "trial_stddev_of_means_ms": 0.01888354805682215 + }, + "ttft_ms": { + "pooled_mean_ms": 151.83002628369175, + "pooled_p90_ms": 152.63689204584807, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.83002628369175, + "trial_stddev_of_means_ms": 0.2293889652860975 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 769.7875488309421, + "p90_ms": 806.7109240218997, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.864016504716275, + "p90_ms": 5.151719298565716, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.02143533036198, + "p90_ms": 152.73106703534722, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 765.9611190793776, + "p90_ms": 765.8235019771382, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.83486823588853, + "p90_ms": 4.831250046570529, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.89288162297836, + "p90_ms": 152.7155089424923, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 770.1361784530991, + "p90_ms": 806.5015620086342, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.8702401822041885, + "p90_ms": 5.154327385853536, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.5757618977349, + "p90_ms": 152.41257997695357, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp1_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 683.3139202742204, + "pooled_p90_ms": 691.7126830667257, + "pooled_samples": 387, + "trial_mean_of_means_ms": 683.3139202742204, + "trial_stddev_of_means_ms": 13.480078748721647 + }, + "tpot_ms": { + "pooled_mean_ms": 4.606023389027604, + "pooled_p90_ms": 4.679884338076777, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.606023389027605, + "trial_stddev_of_means_ms": 0.11860164117227036 + }, + "ttft_ms": { + "pooled_mean_ms": 98.30897632000838, + "pooled_p90_ms": 100.99728393834084, + "pooled_samples": 387, + "trial_mean_of_means_ms": 98.30897632000837, + "trial_stddev_of_means_ms": 1.6033196848780447 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 691.7368602395678, + "p90_ms": 693.4372959658504, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.677960039364579, + "p90_ms": 4.685888802962686, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.59817232175679, + "p90_ms": 98.27463503461331, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 667.7665301777292, + "p90_ms": 672.6519849617034, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.4691333712332115, + "p90_ms": 4.478958197026037, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 100.14481820949433, + "p90_ms": 103.45508099999279, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 690.4383704053643, + "p90_ms": 691.623940016143, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.670976756485024, + "p90_ms": 4.677882693033284, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.18393842877401, + "p90_ms": 98.27898698858917, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 689.548538498116, + "pooled_p90_ms": 691.6915030451491, + "pooled_samples": 387, + "trial_mean_of_means_ms": 689.548538498116, + "trial_stddev_of_means_ms": 0.4691564069544124 + }, + "tpot_ms": { + "pooled_mean_ms": 4.66880722232803, + "pooled_p90_ms": 4.676126267240331, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.66880722232803, + "trial_stddev_of_means_ms": 0.006071937772150551 + }, + "ttft_ms": { + "pooled_mean_ms": 96.57110613066877, + "pooled_p90_ms": 98.41530094854534, + "pooled_samples": 387, + "trial_mean_of_means_ms": 96.57110613066875, + "trial_stddev_of_means_ms": 0.8723383104752306 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 689.6087854717995, + "p90_ms": 691.4246910018846, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.661907737752395, + "p90_ms": 4.664850331417571, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.50512631721939, + "p90_ms": 99.51729699969292, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 689.9846611488538, + "p90_ms": 692.5024109659716, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.673336901073146, + "p90_ms": 4.679785157722517, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 96.43071331534996, + "p90_ms": 98.4487070236355, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 689.0521688736947, + "p90_ms": 691.1603320622817, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.671177028158549, + "p90_ms": 4.681181668589904, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 95.77747875943693, + "p90_ms": 97.0232019899413, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 689.733952092541, + "pooled_p90_ms": 691.8661230010912, + "pooled_samples": 387, + "trial_mean_of_means_ms": 689.733952092541, + "trial_stddev_of_means_ms": 1.30873077294457 + }, + "tpot_ms": { + "pooled_mean_ms": 4.680890721121927, + "pooled_p90_ms": 4.687803739674506, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.680890721121927, + "trial_stddev_of_means_ms": 0.0021247475013520785 + }, + "ttft_ms": { + "pooled_mean_ms": 95.22211026624134, + "pooled_p90_ms": 97.45279501657933, + "pooled_samples": 387, + "trial_mean_of_means_ms": 95.22211026624133, + "trial_stddev_of_means_ms": 1.485311605474211 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 688.8176469527575, + "p90_ms": 690.4401639476418, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.680177864121985, + "p90_ms": 4.685609865434996, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.3946194773641, + "p90_ms": 95.91560496482998, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 691.2328087049717, + "p90_ms": 693.3015899267048, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.679214066220331, + "p90_ms": 4.690911558522719, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 96.93685328216083, + "p90_ms": 98.8241919549182, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 689.1514006198939, + "p90_ms": 690.7796860905364, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.683280233023465, + "p90_ms": 4.687803739674506, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.33485803919909, + "p90_ms": 95.42244195472449, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 688.7246717025648, + "pooled_p90_ms": 691.0650019999593, + "pooled_samples": 387, + "trial_mean_of_means_ms": 688.7246717025649, + "trial_stddev_of_means_ms": 1.440994479221059 + }, + "tpot_ms": { + "pooled_mean_ms": 4.67674757633165, + "pooled_p90_ms": 4.68690808693783, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.67674757633165, + "trial_stddev_of_means_ms": 0.014951986782554206 + }, + "ttft_ms": { + "pooled_mean_ms": 94.74037260235858, + "pooled_p90_ms": 96.15806792862713, + "pooled_samples": 387, + "trial_mean_of_means_ms": 94.74037260235859, + "trial_stddev_of_means_ms": 0.5067099323623165 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 687.0613511852097, + "p90_ms": 689.566144021228, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.659691155888254, + "p90_ms": 4.6653019678668946, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 95.2438001354882, + "p90_ms": 96.90531203523278, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 689.5177478770285, + "p90_ms": 691.6546910069883, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.682958287904102, + "p90_ms": 4.686032913214579, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.74687482042657, + "p90_ms": 96.44627594389021, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 689.5949160454563, + "p90_ms": 691.671111038886, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.687593285202594, + "p90_ms": 4.695545803154196, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.23044285116096, + "p90_ms": 95.47765098977834, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp2_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 607.6659436830384, + "pooled_p90_ms": 614.7840449120849, + "pooled_samples": 387, + "trial_mean_of_means_ms": 607.6659436830384, + "trial_stddev_of_means_ms": 5.83329156754365 + }, + "tpot_ms": { + "pooled_mean_ms": 4.288118025279364, + "pooled_p90_ms": 4.3549379049690335, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.288118025279364, + "trial_stddev_of_means_ms": 0.05739660569371236 + }, + "ttft_ms": { + "pooled_mean_ms": 63.03574686938565, + "pooled_p90_ms": 66.4233659626916, + "pooled_samples": 387, + "trial_mean_of_means_ms": 63.035746869385655, + "trial_stddev_of_means_ms": 1.4920342964335762 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 604.923196775882, + "p90_ms": 609.2176410602406, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.258012776043345, + "p90_ms": 4.267158401088686, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.11492886642615, + "p90_ms": 68.74241400510073, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 614.3651023297751, + "p90_ms": 616.209093015641, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.354304061884912, + "p90_ms": 4.365195747552894, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 61.33310341947647, + "p90_ms": 62.62887897901237, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 603.7095319434582, + "p90_ms": 607.3436400620267, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.252037237909835, + "p90_ms": 4.25958238619079, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 63.65920832225432, + "p90_ms": 66.49800203740597, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 604.4058500071964, + "pooled_p90_ms": 611.0399010358378, + "pooled_samples": 387, + "trial_mean_of_means_ms": 604.4058500071964, + "trial_stddev_of_means_ms": 4.461581159639322 + }, + "tpot_ms": { + "pooled_mean_ms": 4.254808547439789, + "pooled_p90_ms": 4.27782233056473, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.254808547439789, + "trial_stddev_of_means_ms": 0.018762002450543058 + }, + "ttft_ms": { + "pooled_mean_ms": 64.01019376246572, + "pooled_p90_ms": 68.59070702921599, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.0101937624657, + "trial_stddev_of_means_ms": 2.2209645174649575 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 609.0962116628192, + "p90_ms": 614.4514320185408, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.276095369246001, + "p90_ms": 4.289275638421807, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 65.99773169953608, + "p90_ms": 71.24563900288194, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 603.9062279290989, + "p90_ms": 610.2306098910049, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.247653137253014, + "p90_ms": 4.260518551459462, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.41997497093452, + "p90_ms": 68.43586196191609, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 600.215110429671, + "p90_ms": 602.216740953736, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.240677135820352, + "p90_ms": 4.246149299204291, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 61.61287461692655, + "p90_ms": 63.24575201142579, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 605.5669432571543, + "pooled_p90_ms": 610.7392100384459, + "pooled_samples": 387, + "trial_mean_of_means_ms": 605.5669432571543, + "trial_stddev_of_means_ms": 2.5583322144334986 + }, + "tpot_ms": { + "pooled_mean_ms": 4.264903840417586, + "pooled_p90_ms": 4.27982993753935, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.264903840417586, + "trial_stddev_of_means_ms": 0.008241890125268254 + }, + "ttft_ms": { + "pooled_mean_ms": 63.8859268507706, + "pooled_p90_ms": 67.59266997687519, + "pooled_samples": 387, + "trial_mean_of_means_ms": 63.8859268507706, + "trial_stddev_of_means_ms": 1.5158748686612207 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 605.7536339084077, + "p90_ms": 610.4160649701953, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.264924569000695, + "p90_ms": 4.281197173161593, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.06844277909502, + "p90_ms": 67.37245502881706, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 602.9203796349539, + "p90_ms": 606.4624059945345, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.256651605550648, + "p90_ms": 4.271073574860265, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 62.287057342022194, + "p90_ms": 64.74815693218261, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 608.0268162281014, + "p90_ms": 613.2307189982384, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.273135346701415, + "p90_ms": 4.29227040974262, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 65.30228043119459, + "p90_ms": 69.0081981010735, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 605.0888788116282, + "pooled_p90_ms": 610.0081680342555, + "pooled_samples": 387, + "trial_mean_of_means_ms": 605.0888788116282, + "trial_stddev_of_means_ms": 0.8901667424570828 + }, + "tpot_ms": { + "pooled_mean_ms": 4.256483071855015, + "pooled_p90_ms": 4.268226101921182, + "pooled_samples": 387, + "trial_mean_of_means_ms": 4.256483071855015, + "trial_stddev_of_means_ms": 0.0026292150472670742 + }, + "ttft_ms": { + "pooled_mean_ms": 64.47646671392322, + "pooled_p90_ms": 68.37363401427865, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.47646671392322, + "trial_stddev_of_means_ms": 1.0383603866006366 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 606.0357820910452, + "p90_ms": 611.1916180234402, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.2545578456668816, + "p90_ms": 4.264190204544213, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 65.67109757204099, + "p90_ms": 71.20707002468407, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 604.2691217230041, + "p90_ms": 608.334549004212, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.2554127314675245, + "p90_ms": 4.26826358307153, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 63.79063503163354, + "p90_ms": 67.93791998643428, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 604.9617326208354, + "p90_ms": 610.0081680342555, + "samples": 129 + }, + "tpot_ms": { + "mean_ms": 4.259478638430637, + "p90_ms": 4.271154976669904, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 63.96766753809512, + "p90_ms": 67.67752906307578, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-pd/real/tp4_mns8/trial3/results/result.json", + "trial": 3 + } + ] + } + }, + "prefill_only": false, + "schema": "qwen30-latency-case-real-audit-v1", + "trace_manifests": { + "tp1": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 0.215, + "last_arrival_s": 595.3488372093024, + "output_tokens": [ + 128 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd/tp1/private/real_requests.jsonl", + "private_jsonl_sha256": "fa903f38c522a1ab5fa6325697f7aebd8ce17c0809ecee5cba7f28ec7eb6a030", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd/tp1/public/frontier.csv", + "public_csv_sha256": "d33dd9fcd1aea6c6a6fb6730d8bfe30392ad6827f4132682ad151cccabe4bf2d", + "requests": 129, + "row_vector_sha256": "6cb1cdd992eca886c8f1611ee83ec1373ff162bd1b608a8ce23974885b393aea", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 1 + }, + "tp2": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 0.43, + "last_arrival_s": 297.6744186046512, + "output_tokens": [ + 128 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd/tp2/private/real_requests.jsonl", + "private_jsonl_sha256": "51d9e597d5efec87f9a6a7ea425ee1fa95eb4d80db8cadaaef8acbab0f024919", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd/tp2/public/frontier.csv", + "public_csv_sha256": "c3d7ce76385f348e8082d9aae1449384e9710a2d4c9a0b8b5d36bac28a5f2434", + "requests": 129, + "row_vector_sha256": "51947bb92449fc5be3ecaafb5b0ba08acd848ff5c6cff9a04a66bd3fda74d45e", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 2 + }, + "tp4": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 0.86, + "last_arrival_s": 148.8372093023256, + "output_tokens": [ + 128 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd/tp4/private/real_requests.jsonl", + "private_jsonl_sha256": "90450c8ca24055ed18d3edeb43255380decb7de285ff982b48cb4211775f0326", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd/tp4/public/frontier.csv", + "public_csv_sha256": "c78ced228c0df8a20874f14cc620a4e259a828d9fc876a8dc022fb0280af1591", + "requests": 129, + "row_vector_sha256": "ce55943d20be72c04040e4bbeb0621d529590594dd11e968eefcf788895564a1", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 4 + } + }, + "traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-pd", + "winners": { + "e2e_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns16", + "tp2_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 604.4058500071964 + }, + "e2e_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp2_mns64", + "tp1_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8" + ], + "winner": "tp4_mns8", + "winner_value_ms": 610.0081680342555 + }, + "tpot_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns16", + "tp2_mns32", + "tp2_mns8", + "tp2_mns64", + "tp1_mns64", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 4.254808547439789 + }, + "tpot_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns32", + "tp2_mns16", + "tp2_mns8", + "tp2_mns64", + "tp1_mns16", + "tp1_mns64", + "tp1_mns32", + "tp1_mns8" + ], + "winner": "tp4_mns8", + "winner_value_ms": 4.268226101921182 + }, + "ttft_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns16", + "tp4_mns64", + "tp4_mns32", + "tp4_mns8", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64" + ], + "winner": "tp4_mns16", + "winner_value_ms": 63.03574686938565 + }, + "ttft_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns16", + "tp4_mns64", + "tp4_mns8", + "tp4_mns32", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64", + "tp1_mns16" + ], + "winner": "tp4_mns16", + "winner_value_ms": 66.4233659626916 + } + } +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.md new file mode 100644 index 0000000..9edf269 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.md @@ -0,0 +1,12 @@ +# Qwen30 real latency case audit + +Prefill-only: `False`. + +| Objective | Real winner | Value (ms) | +|---|---|---:| +| ttft_ms:pooled_mean_ms | tp4_mns16 | 63.04 | +| ttft_ms:pooled_p90_ms | tp4_mns16 | 66.42 | +| tpot_ms:pooled_mean_ms | tp4_mns32 | 4.25 | +| tpot_ms:pooled_p90_ms | tp4_mns8 | 4.27 | +| e2e_ms:pooled_mean_ms | tp4_mns32 | 604.41 | +| e2e_ms:pooled_p90_ms | tp4_mns8 | 610.01 | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.stdout.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.stdout.json new file mode 100644 index 0000000..01ed58b --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-pd-real-audit.stdout.json @@ -0,0 +1 @@ +{"e2e_ms:pooled_mean_ms": {"ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns32", "winner_value_ms": 604.4058500071964}, "e2e_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp2_mns64", "tp1_mns64", "tp1_mns16", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns8", "winner_value_ms": 610.0081680342555}, "tpot_ms:pooled_mean_ms": {"ranking": ["tp4_mns32", "tp4_mns8", "tp4_mns64", "tp4_mns16", "tp2_mns16", "tp2_mns32", "tp2_mns8", "tp2_mns64", "tp1_mns64", "tp1_mns32", "tp1_mns16", "tp1_mns8"], "winner": "tp4_mns32", "winner_value_ms": 4.254808547439789}, "tpot_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns32", "tp2_mns16", "tp2_mns8", "tp2_mns64", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns8", "winner_value_ms": 4.268226101921182}, "ttft_ms:pooled_mean_ms": {"ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns32", "tp4_mns8", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns16", "winner_value_ms": 63.03574686938565}, "ttft_ms:pooled_p90_ms": {"ranking": ["tp4_mns16", "tp4_mns64", "tp4_mns8", "tp4_mns32", "tp2_mns8", "tp2_mns64", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64", "tp1_mns16"], "winner": "tp4_mns16", "winner_value_ms": 66.4233659626916}} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.json new file mode 100644 index 0000000..40f961a --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.json @@ -0,0 +1,326 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "e2e_ms" + ], + "cells": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns16/tp1/result.json" + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns32/tp1/result.json" + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns64/tp1/result.json" + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + }, + "ttft_ms": { + "mean": 166.66991554923203, + "p90": 166.6699155492779 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp1_mns8/tp1/result.json" + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + }, + "ttft_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns16/tp2/result.json" + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + }, + "ttft_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns32/tp2/result.json" + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + }, + "ttft_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns64/tp2/result.json" + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + }, + "ttft_ms": { + "mean": 118.55592059231998, + "p90": 118.55592059232833 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp2_mns8/tp2/result.json" + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + }, + "ttft_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns16/tp4/result.json" + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + }, + "ttft_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns32/tp4/result.json" + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + }, + "ttft_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns64/tp4/result.json" + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + }, + "ttft_ms": { + "mean": 86.52545838364681, + "p90": 86.52545838364745 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po/runs/tp4_mns8/tp4/result.json" + } + }, + "prefill_only": true, + "real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/fixed-po-real-audit.json", + "schema": "qwen30-latency-case-frontier-real-comparison-v1", + "selection": { + "e2e_ms:mean": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.014895639803871355, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "e2e_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.03590282591969048, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "ttft_ms:mean": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.014940997472395743, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "ttft_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.035908453871728634, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + } + }, + "sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/fixed-po" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.md new file mode 100644 index 0000000..5237196 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.md @@ -0,0 +1,8 @@ +# Qwen30 Frontier vs real latency selection + +| Objective | Frontier | Real | Match | Regret | Pairwise | +|---|---|---|---:|---:|---:| +| ttft_ms:mean | tp4_mns16 | tp4_mns8 | no | 1.5% | 100.0% | +| ttft_ms:p90 | tp4_mns16 | tp4_mns8 | no | 3.6% | 100.0% | +| e2e_ms:mean | tp4_mns16 | tp4_mns8 | no | 1.5% | 100.0% | +| e2e_ms:p90 | tp4_mns16 | tp4_mns8 | no | 3.6% | 100.0% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.stdout.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.stdout.json new file mode 100644 index 0000000..0a76756 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-comparison.stdout.json @@ -0,0 +1 @@ +{"e2e_ms:mean": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.014895639803871355, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "e2e_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.03590282591969048, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:mean": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.014940997472395743, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.035908453871728634, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.json new file mode 100644 index 0000000..a4fcbba --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.json @@ -0,0 +1,997 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "e2e_ms" + ], + "case_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po", + "configs": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 151.54830888467202, + "pooled_p90_ms": 152.33826905023307, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.54830888467202, + "trial_stddev_of_means_ms": 0.33839071804993504 + }, + "ttft_ms": { + "pooled_mean_ms": 151.50490087229585, + "pooled_p90_ms": 152.29528304189444, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.50490087229585, + "trial_stddev_of_means_ms": 0.3392777547312034 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 151.25221894176806, + "p90_ms": 152.03110210131854, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.20715750724278, + "p90_ms": 151.9814640050754, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 151.4755454836702, + "p90_ms": 152.21454401034862, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.4332707151915, + "p90_ms": 152.16313605196774, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 151.9171622285778, + "p90_ms": 152.64088998083025, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.87427439445327, + "p90_ms": 152.5908849434927, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 151.4782552960514, + "pooled_p90_ms": 152.49883604701608, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.4782552960514, + "trial_stddev_of_means_ms": 0.5555788512092701 + }, + "ttft_ms": { + "pooled_mean_ms": 151.43368434935846, + "pooled_p90_ms": 152.45853806845844, + "pooled_samples": 387, + "trial_mean_of_means_ms": 151.43368434935846, + "trial_stddev_of_means_ms": 0.5560791927635007 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 151.00544339222762, + "p90_ms": 151.96295792702585, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 150.96285229624618, + "p90_ms": 151.92402398679405, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 151.33915553880178, + "p90_ms": 152.3007940268144, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.29099727893436, + "p90_ms": 152.25290204398334, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 152.09016695712484, + "p90_ms": 152.93943404685706, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.04720347289486, + "p90_ms": 152.89701998699456, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 152.62579285143815, + "pooled_p90_ms": 153.7184970220551, + "pooled_samples": 387, + "trial_mean_of_means_ms": 152.62579285143815, + "trial_stddev_of_means_ms": 0.5522776932799126 + }, + "ttft_ms": { + "pooled_mean_ms": 152.58183204256076, + "pooled_p90_ms": 153.6727879429236, + "pooled_samples": 387, + "trial_mean_of_means_ms": 152.58183204256076, + "trial_stddev_of_means_ms": 0.552880728503269 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 152.98813100463835, + "p90_ms": 153.7133390083909, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.9461515773189, + "p90_ms": 153.67183403577656, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 152.89909483832415, + "p90_ms": 154.20521004125476, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.85368830727253, + "p90_ms": 154.1550400434062, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 151.99015271135195, + "p90_ms": 152.67854696139693, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 151.94565624309084, + "p90_ms": 152.6353599037975, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 152.22744446955332, + "pooled_p90_ms": 152.97825809102505, + "pooled_samples": 387, + "trial_mean_of_means_ms": 152.22744446955332, + "trial_stddev_of_means_ms": 0.15844676271042335 + }, + "ttft_ms": { + "pooled_mean_ms": 152.18195707226792, + "pooled_p90_ms": 152.93638303410262, + "pooled_samples": 387, + "trial_mean_of_means_ms": 152.18195707226792, + "trial_stddev_of_means_ms": 0.1592296957231754 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 152.08507488055747, + "p90_ms": 152.94339996762574, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.04033442473315, + "p90_ms": 152.89624000433832, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 152.3981458541771, + "p90_ms": 153.33553205709904, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.3543132889597, + "p90_ms": 153.2915000570938, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 152.19911267392538, + "p90_ms": 152.6885419152677, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 152.15122350311094, + "p90_ms": 152.64206391293555, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp1_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 97.51495216234325, + "pooled_p90_ms": 98.90919097233564, + "pooled_samples": 387, + "trial_mean_of_means_ms": 97.51495216234325, + "trial_stddev_of_means_ms": 0.41813696851903276 + }, + "ttft_ms": { + "pooled_mean_ms": 97.46812211917545, + "pooled_p90_ms": 98.85558602400124, + "pooled_samples": 387, + "trial_mean_of_means_ms": 97.46812211917545, + "trial_stddev_of_means_ms": 0.41503784318372017 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 97.09843800785814, + "p90_ms": 98.22496003471315, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.05583205674017, + "p90_ms": 98.16953900735825, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 97.93469326682533, + "p90_ms": 99.07599200960249, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.88585422369255, + "p90_ms": 99.02978304307908, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 97.51172521234628, + "p90_ms": 99.18248606845737, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.46268007709364, + "p90_ms": 99.13825406692922, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 97.09498990484747, + "pooled_p90_ms": 98.50096399895847, + "pooled_samples": 387, + "trial_mean_of_means_ms": 97.09498990484747, + "trial_stddev_of_means_ms": 0.27051136106210416 + }, + "ttft_ms": { + "pooled_mean_ms": 97.04873380816761, + "pooled_p90_ms": 98.45811000559479, + "pooled_samples": 387, + "trial_mean_of_means_ms": 97.0487338081676, + "trial_stddev_of_means_ms": 0.27130252759105145 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 96.8018446240372, + "p90_ms": 97.90838800836354, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 96.75326022814035, + "p90_ms": 97.85841300617903, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 97.33497442735373, + "p90_ms": 98.68263697717339, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.28661816668256, + "p90_ms": 98.63975096959621, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 97.14815066315147, + "p90_ms": 98.70580595452338, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.1063230296799, + "p90_ms": 98.66256301756948, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 94.78371690426897, + "pooled_p90_ms": 95.95681494101882, + "pooled_samples": 387, + "trial_mean_of_means_ms": 94.78371690426899, + "trial_stddev_of_means_ms": 0.09464944657839716 + }, + "ttft_ms": { + "pooled_mean_ms": 94.73932527026128, + "pooled_p90_ms": 95.90691793709993, + "pooled_samples": 387, + "trial_mean_of_means_ms": 94.73932527026129, + "trial_stddev_of_means_ms": 0.09810418936504331 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 94.80111797740899, + "p90_ms": 95.66861705388874, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.75897805848297, + "p90_ms": 95.61934601515532, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 94.68157430154149, + "p90_ms": 95.95681494101882, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.63288232525305, + "p90_ms": 95.90691793709993, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 94.86845843385645, + "p90_ms": 96.28609393257648, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.82611542704782, + "p90_ms": 96.24539199285209, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 95.65880776170006, + "pooled_p90_ms": 97.55863400641829, + "pooled_samples": 387, + "trial_mean_of_means_ms": 95.65880776170006, + "trial_stddev_of_means_ms": 1.2232525642643362 + }, + "ttft_ms": { + "pooled_mean_ms": 95.61450884031319, + "pooled_p90_ms": 97.50729007646441, + "pooled_samples": 387, + "trial_mean_of_means_ms": 95.61450884031319, + "trial_stddev_of_means_ms": 1.2245345925842415 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 97.06745062893357, + "p90_ms": 99.17883900925517, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 97.02484949686846, + "p90_ms": 99.13210896775126, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 94.86425999920209, + "p90_ms": 96.06455289758742, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.82164932920259, + "p90_ms": 96.02120891213417, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 95.0447126569645, + "p90_ms": 97.03064302448183, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 94.99702769486855, + "p90_ms": 96.98293695691973, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp2_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 65.28745120113297, + "pooled_p90_ms": 70.42863499373198, + "pooled_samples": 387, + "trial_mean_of_means_ms": 65.28745120113297, + "trial_stddev_of_means_ms": 3.4421132551731515 + }, + "ttft_ms": { + "pooled_mean_ms": 65.24474184372936, + "pooled_p90_ms": 70.38743107113987, + "pooled_samples": 387, + "trial_mean_of_means_ms": 65.24474184372936, + "trial_stddev_of_means_ms": 3.440733102883723 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 64.19509824892405, + "p90_ms": 67.92093603871763, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.15208354917203, + "p90_ms": 67.87848006933928, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 69.1431922154039, + "p90_ms": 78.7114379927516, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 69.09912444791908, + "p90_ms": 78.66681204177439, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 62.524063139070954, + "p90_ms": 63.537837006151676, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 62.48301753409698, + "p90_ms": 63.49677708931267, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 64.54067908164875, + "pooled_p90_ms": 69.61812602821738, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.54067908164875, + "trial_stddev_of_means_ms": 0.8919869576553885 + }, + "ttft_ms": { + "pooled_mean_ms": 64.4944863910101, + "pooled_p90_ms": 69.50995896477252, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.49448639101008, + "trial_stddev_of_means_ms": 0.8942503909647054 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 65.55543364614046, + "p90_ms": 71.84744393453002, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 65.51214804161744, + "p90_ms": 71.80331903509796, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 63.88050908684846, + "p90_ms": 66.97378400713205, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 63.83413878312811, + "p90_ms": 66.91711395978928, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 64.18609451195731, + "p90_ms": 69.55521402414888, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.13717234828474, + "p90_ms": 69.50995896477252, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 65.03269931623949, + "pooled_p90_ms": 68.43663903418928, + "pooled_samples": 387, + "trial_mean_of_means_ms": 65.03269931623947, + "trial_stddev_of_means_ms": 1.1339679108088652 + }, + "ttft_ms": { + "pooled_mean_ms": 64.98623655348516, + "pooled_p90_ms": 68.38861701544374, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.98623655348516, + "trial_stddev_of_means_ms": 1.1341449568113504 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 66.32654335125991, + "p90_ms": 70.97568700555712, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 66.28015979965643, + "p90_ms": 70.93840499874204, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 64.56001544761104, + "p90_ms": 67.99875595606863, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.51422280213225, + "p90_ms": 67.94618198182434, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 64.21153914984748, + "p90_ms": 67.55471904762089, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 64.16432705866679, + "p90_ms": 67.51206307671964, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 64.32922621852013, + "pooled_p90_ms": 67.98768497537822, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.32922621852013, + "trial_stddev_of_means_ms": 1.8052761808674234 + }, + "ttft_ms": { + "pooled_mean_ms": 64.2842707174255, + "pooled_p90_ms": 67.94753996655345, + "pooled_samples": 387, + "trial_mean_of_means_ms": 64.2842707174255, + "trial_stddev_of_means_ms": 1.80289690665978 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 62.84650166739183, + "p90_ms": 63.89774801209569, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 62.80200301792271, + "p90_ms": 63.856702065095305, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 66.3395124548945, + "p90_ms": 70.76063798740506, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 66.29134260132447, + "p90_ms": 70.7101090811193, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 63.801664533274064, + "p90_ms": 67.48319591861218, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 63.759466533029325, + "p90_ms": 67.4400629941374, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/fixed-po/real/tp4_mns8/trial3/results/result.json", + "trial": 3 + } + ] + } + }, + "prefill_only": true, + "schema": "qwen30-latency-case-real-audit-v1", + "trace_manifests": { + "tp1": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 0.215, + "last_arrival_s": 595.3488372093024, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp1/private/real_requests.jsonl", + "private_jsonl_sha256": "ce7582fc13050078a243d7f6adfc2b2e071dbe87b5f760c7bf111af57718ccfb", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp1/public/frontier.csv", + "public_csv_sha256": "e6f1ca0354e9d93c60ff18261ae6e08e2e54dbc888e30e244776f9cc8695a8cf", + "requests": 129, + "row_vector_sha256": "7c8dc7cccd2bd874768c1b06ca40d2327452390577b9907bacb76270b46998fa", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 1 + }, + "tp2": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 0.43, + "last_arrival_s": 297.6744186046512, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp2/private/real_requests.jsonl", + "private_jsonl_sha256": "3fe3c01ac31cd776b2458cafb818d889b20c5e14948154bbd716cce7200c8d5b", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp2/public/frontier.csv", + "public_csv_sha256": "78b900d8f47fec3a1c83b748f82cb9623ce3f0cf08265aa20c3ef1511a4a97e0", + "requests": 129, + "row_vector_sha256": "c9382d9007ea1dc572898836f2d0efb8e2244c4febc1ed59d5aeacfcc5475c0a", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 2 + }, + "tp4": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 0.86, + "last_arrival_s": 148.8372093023256, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp4/private/real_requests.jsonl", + "private_jsonl_sha256": "af3ba5a7888ea5f283bb360dd36f1f455906f69f5ad82cef84e82018545710db", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po/tp4/public/frontier.csv", + "public_csv_sha256": "8b1100ccab34d9dac62b7ff1508d36bdf75aa3c3135e6f6642a0d3ac77af2505", + "requests": 129, + "row_vector_sha256": "a5e4959f28790eaa99a59a1b86c449a4e484967f1e35e47900d1a27fa061607f", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 4 + } + }, + "traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/fixed-po", + "winners": { + "e2e_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64" + ], + "winner": "tp4_mns8", + "winner_value_ms": 64.32922621852013 + }, + "e2e_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64" + ], + "winner": "tp4_mns8", + "winner_value_ms": 67.98768497537822 + }, + "ttft_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64" + ], + "winner": "tp4_mns8", + "winner_value_ms": 64.2842707174255 + }, + "ttft_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns8", + "tp1_mns64" + ], + "winner": "tp4_mns8", + "winner_value_ms": 67.94753996655345 + } + } +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.md new file mode 100644 index 0000000..2ffe8bf --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.md @@ -0,0 +1,10 @@ +# Qwen30 real latency case audit + +Prefill-only: `True`. + +| Objective | Real winner | Value (ms) | +|---|---|---:| +| ttft_ms:pooled_mean_ms | tp4_mns8 | 64.28 | +| ttft_ms:pooled_p90_ms | tp4_mns8 | 67.95 | +| e2e_ms:pooled_mean_ms | tp4_mns8 | 64.33 | +| e2e_ms:pooled_p90_ms | tp4_mns8 | 67.99 | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.stdout.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.stdout.json new file mode 100644 index 0000000..282995c --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/fixed-po-real-audit.stdout.json @@ -0,0 +1 @@ +{"e2e_ms:pooled_mean_ms": {"ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 64.32922621852013}, "e2e_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 67.98768497537822}, "ttft_ms:pooled_mean_ms": {"ranking": ["tp4_mns8", "tp4_mns32", "tp4_mns64", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns32", "tp1_mns16", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 64.2842707174255}, "ttft_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns32", "tp1_mns8", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 67.94753996655345}} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.json new file mode 100644 index 0000000..2d8bdff --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.json @@ -0,0 +1,326 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "e2e_ms" + ], + "cells": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "mean": 340.75522510609915, + "p90": 832.752247743656 + }, + "ttft_ms": { + "mean": 340.75522510609915, + "p90": 832.752247743656 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns16/tp1/result.json" + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "mean": 340.75598360435777, + "p90": 832.752247743656 + }, + "ttft_ms": { + "mean": 340.75598360435777, + "p90": 832.752247743656 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns32/tp1/result.json" + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "mean": 340.7569941487134, + "p90": 832.752247743656 + }, + "ttft_ms": { + "mean": 340.7569941487134, + "p90": 832.752247743656 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns64/tp1/result.json" + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "mean": 340.75522510609915, + "p90": 832.752247743656 + }, + "ttft_ms": { + "mean": 340.75522510609915, + "p90": 832.752247743656 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp1_mns8/tp1/result.json" + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "mean": 222.73961839402438, + "p90": 513.3421414172687 + }, + "ttft_ms": { + "mean": 222.73961839402438, + "p90": 513.3421414172687 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns16/tp2/result.json" + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "mean": 222.7393083124536, + "p90": 513.3421414172687 + }, + "ttft_ms": { + "mean": 222.7393083124536, + "p90": 513.3421414172687 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns32/tp2/result.json" + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "mean": 222.73932355314494, + "p90": 513.3421414172687 + }, + "ttft_ms": { + "mean": 222.73932355314494, + "p90": 513.3421414172687 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns64/tp2/result.json" + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "mean": 222.73961839402438, + "p90": 513.3421414172687 + }, + "ttft_ms": { + "mean": 222.73961839402438, + "p90": 513.3421414172687 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp2_mns8/tp2/result.json" + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "mean": 164.583607484367, + "p90": 355.93888195481327 + }, + "ttft_ms": { + "mean": 164.583607484367, + "p90": 355.93888195481327 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns16/tp4/result.json" + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "mean": 164.58396574139448, + "p90": 355.93888195481327 + }, + "ttft_ms": { + "mean": 164.58396574139448, + "p90": 355.93888195481327 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns32/tp4/result.json" + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "mean": 164.58471606968033, + "p90": 355.93888195481327 + }, + "ttft_ms": { + "mean": 164.58471606968033, + "p90": 355.93888195481327 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns64/tp4/result.json" + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "mean": 164.583607484367, + "p90": 355.93888195481327 + }, + "ttft_ms": { + "mean": 164.583607484367, + "p90": 355.93888195481327 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po/runs/tp4_mns8/tp4/result.json" + } + }, + "prefill_only": true, + "real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/trace-po-real-audit.json", + "schema": "qwen30-latency-case-frontier-real-comparison-v1", + "selection": { + "e2e_ms:mean": { + "agreement": 0.8412698412698413, + "concordant_pairs": 53, + "discordant_pairs": 10, + "informative_pairs": 63, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns8", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns64", + "tp1_mns32", + "tp1_mns8" + ], + "real_winner": "tp4_mns64", + "selected_config_real_regret": 0.0318784105178655, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns32", + "tp2_mns64", + "tp2_mns16", + "tp2_mns8", + "tp1_mns16", + "tp1_mns8", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 3, + "winner_match": false + }, + "e2e_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.0033158177113489728, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "ttft_ms:mean": { + "agreement": 0.8412698412698413, + "concordant_pairs": 53, + "discordant_pairs": 10, + "informative_pairs": 63, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns8", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns64", + "tp1_mns32", + "tp1_mns8" + ], + "real_winner": "tp4_mns64", + "selected_config_real_regret": 0.03187213306057872, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns32", + "tp2_mns64", + "tp2_mns16", + "tp2_mns8", + "tp1_mns16", + "tp1_mns8", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 3, + "winner_match": false + }, + "ttft_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.0037779133092611917, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + } + }, + "sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/sim/trace-po" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.md new file mode 100644 index 0000000..12ecc66 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.md @@ -0,0 +1,8 @@ +# Qwen30 Frontier vs real latency selection + +| Objective | Frontier | Real | Match | Regret | Pairwise | +|---|---|---|---:|---:|---:| +| ttft_ms:mean | tp4_mns16 | tp4_mns64 | no | 3.2% | 84.1% | +| ttft_ms:p90 | tp4_mns16 | tp4_mns8 | no | 0.4% | 100.0% | +| e2e_ms:mean | tp4_mns16 | tp4_mns64 | no | 3.2% | 84.1% | +| e2e_ms:p90 | tp4_mns16 | tp4_mns8 | no | 0.3% | 100.0% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.stdout.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.stdout.json new file mode 100644 index 0000000..c28456a --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-comparison.stdout.json @@ -0,0 +1 @@ +{"e2e_ms:mean": {"agreement": 0.8412698412698413, "concordant_pairs": 53, "discordant_pairs": 10, "informative_pairs": 63, "real_ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns64", "selected_config_real_regret": 0.0318784105178655, "sim_ranking": ["tp4_mns16", "tp4_mns8", "tp4_mns32", "tp4_mns64", "tp2_mns32", "tp2_mns64", "tp2_mns16", "tp2_mns8", "tp1_mns16", "tp1_mns8", "tp1_mns32", "tp1_mns64"], "sim_winner": "tp4_mns16", "tied_pairs": 3, "winner_match": false}, "e2e_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.0033158177113489728, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}, "ttft_ms:mean": {"agreement": 0.8412698412698413, "concordant_pairs": 53, "discordant_pairs": 10, "informative_pairs": 63, "real_ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "real_winner": "tp4_mns64", "selected_config_real_regret": 0.03187213306057872, "sim_ranking": ["tp4_mns16", "tp4_mns8", "tp4_mns32", "tp4_mns64", "tp2_mns32", "tp2_mns64", "tp2_mns16", "tp2_mns8", "tp1_mns16", "tp1_mns8", "tp1_mns32", "tp1_mns64"], "sim_winner": "tp4_mns16", "tied_pairs": 3, "winner_match": false}, "ttft_ms:p90": {"agreement": 1.0, "concordant_pairs": 48, "discordant_pairs": 0, "informative_pairs": 48, "real_ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "real_winner": "tp4_mns8", "selected_config_real_regret": 0.0037779133092611917, "sim_ranking": ["tp4_mns16", "tp4_mns32", "tp4_mns64", "tp4_mns8", "tp2_mns16", "tp2_mns32", "tp2_mns64", "tp2_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64", "tp1_mns8"], "sim_winner": "tp4_mns16", "tied_pairs": 18, "winner_match": false}} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.json new file mode 100644 index 0000000..9421973 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.json @@ -0,0 +1,997 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "e2e_ms" + ], + "case_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po", + "configs": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 552.8720716136033, + "pooled_p90_ms": 1849.5772210881114, + "pooled_samples": 387, + "trial_mean_of_means_ms": 552.8720716136035, + "trial_stddev_of_means_ms": 2.521785864921604 + }, + "ttft_ms": { + "pooled_mean_ms": 552.8045099093855, + "pooled_p90_ms": 1849.544870085083, + "pooled_samples": 387, + "trial_mean_of_means_ms": 552.8045099093856, + "trial_stddev_of_means_ms": 2.520195001336989 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 551.4591356410912, + "p90_ms": 1849.5772210881114, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 551.3877547012152, + "p90_ms": 1849.544870085083, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 551.3735196675904, + "p90_ms": 1854.1427809977904, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 551.311527398711, + "p90_ms": 1854.09467399586, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 555.7835595321286, + "p90_ms": 1865.3576440410689, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 555.7142476282306, + "p90_ms": 1865.324669983238, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 553.9655365312751, + "pooled_p90_ms": 1858.5613030008972, + "pooled_samples": 387, + "trial_mean_of_means_ms": 553.9655365312751, + "trial_stddev_of_means_ms": 0.562579474030608 + }, + "ttft_ms": { + "pooled_mean_ms": 553.8995644734638, + "pooled_p90_ms": 1858.5308340843767, + "pooled_samples": 387, + "trial_mean_of_means_ms": 553.8995644734638, + "trial_stddev_of_means_ms": 0.5595598051891147 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 554.533127208002, + "p90_ms": 1863.7738740071654, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 554.4650907711263, + "p90_ms": 1863.7414400000125, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 553.4081058838412, + "p90_ms": 1860.4962331010029, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 553.3461683987488, + "p90_ms": 1860.462275100872, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 553.9553765019821, + "p90_ms": 1858.5613030008972, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 553.8874342505163, + "p90_ms": 1858.5308340843767, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 553.5116215725009, + "pooled_p90_ms": 1862.3707490041852, + "pooled_samples": 387, + "trial_mean_of_means_ms": 553.5116215725008, + "trial_stddev_of_means_ms": 2.551946974085908 + }, + "ttft_ms": { + "pooled_mean_ms": 553.4431933351968, + "pooled_p90_ms": 1862.3419440118596, + "pooled_samples": 387, + "trial_mean_of_means_ms": 553.4431933351968, + "trial_stddev_of_means_ms": 2.558476651513527 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 551.9927626163446, + "p90_ms": 1863.004481070675, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 551.9189011646695, + "p90_ms": 1862.9671899834648, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 556.4578829970312, + "p90_ms": 1876.7878300277516, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 556.3969604667392, + "p90_ms": 1876.7514559440315, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 552.0842191041266, + "p90_ms": 1862.3707490041852, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 552.0137183741818, + "p90_ms": 1862.3419440118596, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 554.3623829256859, + "pooled_p90_ms": 1849.5380710810423, + "pooled_samples": 387, + "trial_mean_of_means_ms": 554.3623829256859, + "trial_stddev_of_means_ms": 1.2895027720806873 + }, + "ttft_ms": { + "pooled_mean_ms": 554.2973810096947, + "pooled_p90_ms": 1849.499023053795, + "pooled_samples": 387, + "trial_mean_of_means_ms": 554.2973810096947, + "trial_stddev_of_means_ms": 1.2879745625238963 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 552.8872783905863, + "p90_ms": 1849.5380710810423, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 552.8246594192243, + "p90_ms": 1849.499023053795, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 555.2756262775293, + "p90_ms": 1865.9542039968073, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 555.2131765045572, + "p90_ms": 1865.9242549911141, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 554.9242441089419, + "p90_ms": 1863.5734919225797, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 554.8543071053027, + "p90_ms": 1863.5354900034145, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp1_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 354.36655512622264, + "pooled_p90_ms": 1137.23775499966, + "pooled_samples": 387, + "trial_mean_of_means_ms": 354.36655512622264, + "trial_stddev_of_means_ms": 1.8755237488897505 + }, + "ttft_ms": { + "pooled_mean_ms": 354.2987044818834, + "pooled_p90_ms": 1137.2028539190069, + "pooled_samples": 387, + "trial_mean_of_means_ms": 354.29870448188336, + "trial_stddev_of_means_ms": 1.8738441773930947 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 352.474492817554, + "p90_ms": 1146.7004970181733, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 352.4049650534785, + "p90_ms": 1146.5505469823256, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 356.2250907998532, + "p90_ms": 1142.300286097452, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 356.1519986188198, + "p90_ms": 1142.2789900097996, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 354.40008176126076, + "p90_ms": 1137.23775499966, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 354.3391497733519, + "p90_ms": 1137.2028539190069, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 353.6182792498718, + "pooled_p90_ms": 1133.6436610436067, + "pooled_samples": 387, + "trial_mean_of_means_ms": 353.6182792498718, + "trial_stddev_of_means_ms": 1.6113304304047518 + }, + "ttft_ms": { + "pooled_mean_ms": 353.5506138878872, + "pooled_p90_ms": 1133.6100719636306, + "pooled_samples": 387, + "trial_mean_of_means_ms": 353.5506138878872, + "trial_stddev_of_means_ms": 1.6067873099412449 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 355.363455723769, + "p90_ms": 1137.4261770397425, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 355.2912532689425, + "p90_ms": 1137.4054140178487, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 353.3043994003358, + "p90_ms": 1133.6436610436067, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 353.2365282921684, + "p90_ms": 1133.6100719636306, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 352.1869826255106, + "p90_ms": 1137.214289046824, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 352.12406010255074, + "p90_ms": 1137.1923619881272, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 344.2724434967521, + "pooled_p90_ms": 1112.3684010235593, + "pooled_samples": 387, + "trial_mean_of_means_ms": 344.27244349675215, + "trial_stddev_of_means_ms": 1.2293359433030469 + }, + "ttft_ms": { + "pooled_mean_ms": 344.2076703236325, + "pooled_p90_ms": 1112.340173101984, + "pooled_samples": 387, + "trial_mean_of_means_ms": 344.20767032363256, + "trial_stddev_of_means_ms": 1.2250537007477627 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 342.85445375793955, + "p90_ms": 1112.3684010235593, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 342.79442101431823, + "p90_ms": 1112.340173101984, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 345.03840929702244, + "p90_ms": 1119.2332479404286, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 344.96722287271, + "p90_ms": 1119.2107619717717, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 344.92446743529433, + "p90_ms": 1121.5684060007334, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 344.86136708386937, + "p90_ms": 1121.5376510517672, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 347.7476487716525, + "pooled_p90_ms": 1112.5693990616128, + "pooled_samples": 387, + "trial_mean_of_means_ms": 347.74764877165256, + "trial_stddev_of_means_ms": 0.717530137997859 + }, + "ttft_ms": { + "pooled_mean_ms": 347.67954972228495, + "pooled_p90_ms": 1112.5206090509892, + "pooled_samples": 387, + "trial_mean_of_means_ms": 347.67954972228495, + "trial_stddev_of_means_ms": 0.7137879732940912 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 347.8964949396706, + "p90_ms": 1126.5470329672098, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 347.825654325718, + "p90_ms": 1126.4982499415055, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 348.37908198508296, + "p90_ms": 1108.5135659668595, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 348.3089811246221, + "p90_ms": 1108.4930589422584, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 346.967369390204, + "p90_ms": 1112.5693990616128, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 346.90401371651467, + "p90_ms": 1112.5206090509892, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp2_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 230.30740981957652, + "pooled_p90_ms": 669.3113599903882, + "pooled_samples": 387, + "trial_mean_of_means_ms": 230.30740981957652, + "trial_stddev_of_means_ms": 2.164263687280578 + }, + "ttft_ms": { + "pooled_mean_ms": 230.23591205587212, + "pooled_p90_ms": 669.2857249872759, + "pooled_samples": 387, + "trial_mean_of_means_ms": 230.2359120558721, + "trial_stddev_of_means_ms": 2.1675472527430966 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 230.70611610156737, + "p90_ms": 682.0242760004476, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 230.64041384333356, + "p90_ms": 681.8402550416067, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 232.2445988110014, + "p90_ms": 669.8487129760906, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 232.17271348053603, + "p90_ms": 669.6453359909356, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 227.9715145461608, + "p90_ms": 668.2263769907877, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 227.89460884374674, + "p90_ms": 667.8900560364127, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 227.01489424738475, + "pooled_p90_ms": 668.1078639812768, + "pooled_samples": 387, + "trial_mean_of_means_ms": 227.01489424738475, + "trial_stddev_of_means_ms": 1.0302265785061928 + }, + "ttft_ms": { + "pooled_mean_ms": 226.9508410934046, + "pooled_p90_ms": 668.0838000029325, + "pooled_samples": 387, + "trial_mean_of_means_ms": 226.9508410934046, + "trial_stddev_of_means_ms": 1.0309199236641802 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 227.36105044253256, + "p90_ms": 668.9074879977852, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 227.29304120623212, + "p90_ms": 668.8842560397461, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 227.82746249586054, + "p90_ms": 668.1078639812768, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 227.7671468305192, + "p90_ms": 668.0838000029325, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 225.8561698037611, + "p90_ms": 668.5177159961313, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 225.79233524346247, + "p90_ms": 668.1753649609163, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 223.1923911500318, + "pooled_p90_ms": 668.0814319988713, + "pooled_samples": 387, + "trial_mean_of_means_ms": 223.1923911500318, + "trial_stddev_of_means_ms": 1.0834054130133073 + }, + "ttft_ms": { + "pooled_mean_ms": 223.12445959072676, + "pooled_p90_ms": 668.0591680342332, + "pooled_samples": 387, + "trial_mean_of_means_ms": 223.1244595907268, + "trial_stddev_of_means_ms": 1.078830920161661 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 224.0322316110553, + "p90_ms": 668.6453759903088, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 223.96008559961103, + "p90_ms": 668.1041569681838, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 223.57544556978453, + "p90_ms": 697.0440980512649, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 223.50675472538455, + "p90_ms": 697.0191639848053, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 221.96949626925553, + "p90_ms": 668.0814319988713, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 221.9065384471847, + "p90_ms": 668.0591680342332, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 229.07142115104702, + "pooled_p90_ms": 667.0993800507858, + "pooled_samples": 387, + "trial_mean_of_means_ms": 229.07142115104702, + "trial_stddev_of_means_ms": 1.6982436653662605 + }, + "ttft_ms": { + "pooled_mean_ms": 229.00724306236953, + "pooled_p90_ms": 666.7667380534112, + "pooled_samples": 387, + "trial_mean_of_means_ms": 229.00724306236953, + "trial_stddev_of_means_ms": 1.6999412520227009 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 227.73505919955872, + "p90_ms": 667.3817320261151, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 227.67112628275225, + "p90_ms": 667.0240580569953, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 230.98243522013775, + "p90_ms": 667.0993800507858, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 230.92065048909407, + "p90_ms": 666.7667380534112, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 228.4967690334446, + "p90_ms": 667.9027479840443, + "samples": 129 + }, + "ttft_ms": { + "mean_ms": 228.42995241526225, + "p90_ms": 667.7060039946809, + "samples": 129 + } + }, + "requests": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/real-r6/trace-po/real/tp4_mns8/trial3/results/result.json", + "trial": 3 + } + ] + } + }, + "prefill_only": true, + "schema": "qwen30-latency-case-real-audit-v1", + "trace_manifests": { + "tp1": { + "description": "trace-derived: exact input/arrival/session/prefix; output override only", + "first_arrival_s": 6.698, + "global_offered_request_rate": 0.215, + "last_arrival_s": 597.037, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": true, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp1/private/real_requests.jsonl", + "private_jsonl_sha256": "525bf779bbd111bc3c42ac65af5cfed36ce59806acc521ee16d1c5ce7b3a6152", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp1/public/frontier.csv", + "public_csv_sha256": "eccab2e2f50f566eb4ec78291d126fce5fe9cb426eb89993c1b63b18f83dfc29", + "requests": 129, + "row_vector_sha256": "3df3a57e6a796e76d42f846fbfef59049f2bd804f4d3eda8ecbc9d96b43dc6ce", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 1 + }, + "tp2": { + "description": "trace-derived: exact input/arrival/session/prefix; output override only", + "first_arrival_s": 3.349, + "global_offered_request_rate": 0.43, + "last_arrival_s": 298.5185, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": true, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp2/private/real_requests.jsonl", + "private_jsonl_sha256": "61bdb8a7245426a7c283d0f2944657d3eda46b1c04998c88b025a2411fe15340", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp2/public/frontier.csv", + "public_csv_sha256": "26701486cfb655a301b129acd14ab8d78b94ec843b4ceca8e222cee466e40535", + "requests": 129, + "row_vector_sha256": "f20dd473240fd243d38cb727f7896457e86da9e965b4f23f565aef5f3fc70351", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 2 + }, + "tp4": { + "description": "trace-derived: exact input/arrival/session/prefix; output override only", + "first_arrival_s": 1.6745, + "global_offered_request_rate": 0.86, + "last_arrival_s": 149.25925, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 0.215, + "prefix_caching": true, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp4/private/real_requests.jsonl", + "private_jsonl_sha256": "c38514fa524fc215cd9097d990a2985532f83ac6314bc46ad89ebfc1bd89b4f8", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po/tp4/public/frontier.csv", + "public_csv_sha256": "f4112a2b0366b559bfced6a0986a01e2b7b39899a650d030c3e680b3dc63c92e", + "requests": 129, + "row_vector_sha256": "851e796f6d3e87444b58da87e0ef4b7482a13bb6502cb6873fa80523877ab74e", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 4 + } + }, + "traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/traces/trace-po", + "winners": { + "e2e_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns8", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns64", + "tp1_mns32", + "tp1_mns8" + ], + "winner": "tp4_mns64", + "winner_value_ms": 223.1923911500318 + }, + "e2e_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "winner": "tp4_mns8", + "winner_value_ms": 667.0993800507858 + }, + "ttft_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns8", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns16", + "tp1_mns64", + "tp1_mns32", + "tp1_mns8" + ], + "winner": "tp4_mns64", + "winner_value_ms": 223.12445959072676 + }, + "ttft_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "winner": "tp4_mns8", + "winner_value_ms": 666.7667380534112 + } + } +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.md new file mode 100644 index 0000000..fee21bb --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.md @@ -0,0 +1,10 @@ +# Qwen30 real latency case audit + +Prefill-only: `True`. + +| Objective | Real winner | Value (ms) | +|---|---|---:| +| ttft_ms:pooled_mean_ms | tp4_mns64 | 223.12 | +| ttft_ms:pooled_p90_ms | tp4_mns8 | 666.77 | +| e2e_ms:pooled_mean_ms | tp4_mns64 | 223.19 | +| e2e_ms:pooled_p90_ms | tp4_mns8 | 667.10 | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.stdout.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.stdout.json new file mode 100644 index 0000000..a5cdce0 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-expansion-lo/trace-po-real-audit.stdout.json @@ -0,0 +1 @@ +{"e2e_ms:pooled_mean_ms": {"ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns64", "winner_value_ms": 223.1923911500318}, "e2e_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 667.0993800507858}, "ttft_ms:pooled_mean_ms": {"ranking": ["tp4_mns64", "tp4_mns32", "tp4_mns8", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns16", "tp1_mns64", "tp1_mns32", "tp1_mns8"], "winner": "tp4_mns64", "winner_value_ms": 223.12445959072676}, "ttft_ms:pooled_p90_ms": {"ranking": ["tp4_mns8", "tp4_mns64", "tp4_mns32", "tp4_mns16", "tp2_mns64", "tp2_mns8", "tp2_mns32", "tp2_mns16", "tp1_mns8", "tp1_mns16", "tp1_mns32", "tp1_mns64"], "winner": "tp4_mns8", "winner_value_ms": 666.7667380534112}} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/comparison.json new file mode 100644 index 0000000..c370177 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/comparison.json @@ -0,0 +1,453 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "tpot_ms", + "e2e_ms" + ], + "cells": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "mean": 128201.44370110576, + "p90": 216221.8716312423 + }, + "tpot_ms": { + "mean": 106.74127671151493, + "p90": 107.53204073371177 + }, + "ttft_ms": { + "mean": 100982.41813966946, + "p90": 188801.20124415262 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns16/tp1/result.json" + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "mean": 54134.6392206195, + "p90": 70757.16788718625 + }, + "tpot_ms": { + "mean": 131.73148052557949, + "p90": 134.09269340154515 + }, + "ttft_ms": { + "mean": 20543.111686596734, + "p90": 40978.20939546926 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns32/tp1/result.json" + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "mean": 36992.839178769755, + "p90": 38253.17393690583 + }, + "tpot_ms": { + "mean": 143.73592141724885, + "p90": 148.50084455017992 + }, + "ttft_ms": { + "mean": 340.1792173712992, + "p90": 390.1667904730175 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns64/tp1/result.json" + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "mean": 231355.5665072295, + "p90": 401099.3133242681 + }, + "tpot_ms": { + "mean": 79.72075104221494, + "p90": 80.05841803805441 + }, + "ttft_ms": { + "mean": 211026.77499146466, + "p90": 380684.4167245642 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp1_mns8/tp1/result.json" + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "mean": 99415.72204671855, + "p90": 170023.18522015438 + }, + "tpot_ms": { + "mean": 69.3294093088488, + "p90": 69.78173796567563 + }, + "ttft_ms": { + "mean": 81736.72267296212, + "p90": 152228.84203890432 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns16/tp2/result.json" + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "mean": 47656.99443299207, + "p90": 71358.22494490076 + }, + "tpot_ms": { + "mean": 83.38213205445625, + "p90": 84.68169232450846 + }, + "ttft_ms": { + "mean": 26394.55075910572, + "p90": 52520.58767404907 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns32/tp2/result.json" + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "mean": 25800.137969948584, + "p90": 26994.699921135867 + }, + "tpot_ms": { + "mean": 100.27798158873513, + "p90": 104.85288258852235 + }, + "ttft_ms": { + "mean": 229.2526648211283, + "p90": 261.37367146904467 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns64/tp2/result.json" + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "mean": 173225.23923299377, + "p90": 301133.20586389 + }, + "tpot_ms": { + "mean": 53.275061700270285, + "p90": 53.466591976089035 + }, + "ttft_ms": { + "mean": 159640.09849942484, + "p90": 287499.22490998777 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp2_mns8/tp2/result.json" + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "mean": 77014.8840236653, + "p90": 132893.6304418779 + }, + "tpot_ms": { + "mean": 46.95641974752248, + "p90": 47.23194326182317 + }, + "ttft_ms": { + "mean": 65040.99698804708, + "p90": 120849.48491011641 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns16/tp4/result.json" + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "mean": 41823.75653856865, + "p90": 66900.19803040712 + }, + "tpot_ms": { + "mean": 56.89199563649036, + "p90": 57.75799749592927 + }, + "ttft_ms": { + "mean": 27316.2976512636, + "p90": 54230.649753904916 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns32/tp4/result.json" + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "mean": 24104.231142330624, + "p90": 29174.179941975835 + }, + "tpot_ms": { + "mean": 69.02049776725552, + "p90": 72.06184134879742 + }, + "ttft_ms": { + "mean": 6504.004211680465, + "p90": 13004.945097644395 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns64/tp4/result.json" + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "mean": 133968.39678804282, + "p90": 233355.7233519645 + }, + "tpot_ms": { + "mean": 37.645089303294014, + "p90": 37.75522144488961 + }, + "ttft_ms": { + "mean": 124368.89901570287, + "p90": 223728.14188352478 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd/runs/tp4_mns8/tp4/result.json" + } + }, + "prefill_only": false, + "real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/analysis/fixed-pd/real-audit.json", + "schema": "qwen30-latency-case-frontier-real-comparison-v1", + "selection": { + "e2e_ms:mean": { + "agreement": 0.5151515151515151, + "concordant_pairs": 34, + "discordant_pairs": 32, + "informative_pairs": 66, + "real_ranking": [ + "tp1_mns32", + "tp4_mns32", + "tp1_mns16", + "tp1_mns8", + "tp4_mns16", + "tp4_mns64", + "tp1_mns64", + "tp2_mns8", + "tp2_mns64", + "tp2_mns16", + "tp2_mns32", + "tp4_mns8" + ], + "real_winner": "tp1_mns32", + "selected_config_real_regret": 0.017005214686575575, + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns32", + "tp4_mns16", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": false + }, + "e2e_ms:p90": { + "agreement": 0.5151515151515151, + "concordant_pairs": 34, + "discordant_pairs": 32, + "informative_pairs": 66, + "real_ranking": [ + "tp1_mns32", + "tp1_mns8", + "tp1_mns16", + "tp1_mns64", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp4_mns8" + ], + "real_winner": "tp1_mns32", + "selected_config_real_regret": 0.054933980018932745, + "sim_ranking": [ + "tp2_mns64", + "tp4_mns64", + "tp1_mns64", + "tp4_mns32", + "tp1_mns32", + "tp2_mns32", + "tp4_mns16", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp2_mns64", + "tied_pairs": 0, + "winner_match": false + }, + "tpot_ms:mean": { + "agreement": 0.48484848484848486, + "concordant_pairs": 32, + "discordant_pairs": 34, + "informative_pairs": 66, + "real_ranking": [ + "tp4_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64", + "tp4_mns32", + "tp2_mns8", + "tp4_mns16", + "tp4_mns64", + "tp2_mns64", + "tp2_mns16", + "tp2_mns32" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.0, + "sim_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns16", + "tp1_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns8", + "tied_pairs": 0, + "winner_match": true + }, + "tpot_ms:p90": { + "agreement": 0.42424242424242425, + "concordant_pairs": 28, + "discordant_pairs": 38, + "informative_pairs": 66, + "real_ranking": [ + "tp4_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64", + "tp2_mns8", + "tp4_mns32", + "tp2_mns64", + "tp4_mns64", + "tp4_mns16", + "tp2_mns32", + "tp2_mns16" + ], + "real_winner": "tp4_mns8", + "selected_config_real_regret": 0.0, + "sim_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp4_mns32", + "tp2_mns16", + "tp4_mns64", + "tp1_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns8", + "tied_pairs": 0, + "winner_match": true + }, + "ttft_ms:mean": { + "agreement": 0.5909090909090909, + "concordant_pairs": 39, + "discordant_pairs": 27, + "informative_pairs": 66, + "real_ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp4_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.5797674051508896, + "sim_ranking": [ + "tp2_mns64", + "tp1_mns64", + "tp4_mns64", + "tp1_mns32", + "tp2_mns32", + "tp4_mns32", + "tp4_mns16", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp2_mns64", + "tied_pairs": 0, + "winner_match": false + }, + "ttft_ms:p90": { + "agreement": 0.5909090909090909, + "concordant_pairs": 39, + "discordant_pairs": 27, + "informative_pairs": 66, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns16", + "tp2_mns32", + "tp1_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "selected_config_real_regret": 0.5620663582204116, + "sim_ranking": [ + "tp2_mns64", + "tp1_mns64", + "tp4_mns64", + "tp1_mns32", + "tp2_mns32", + "tp4_mns32", + "tp4_mns16", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp2_mns64", + "tied_pairs": 0, + "winner_match": false + } + }, + "sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-pd" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/comparison.md new file mode 100644 index 0000000..a53acbe --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/comparison.md @@ -0,0 +1,10 @@ +# Qwen30 Frontier vs real latency selection + +| Objective | Frontier | Real | Match | Regret | Pairwise | +|---|---|---|---:|---:|---:| +| ttft_ms:mean | tp2_mns64 | tp4_mns32 | no | 58.0% | 59.1% | +| ttft_ms:p90 | tp2_mns64 | tp4_mns64 | no | 56.2% | 59.1% | +| tpot_ms:mean | tp4_mns8 | tp4_mns8 | yes | 0.0% | 48.5% | +| tpot_ms:p90 | tp4_mns8 | tp4_mns8 | yes | 0.0% | 42.4% | +| e2e_ms:mean | tp4_mns64 | tp1_mns32 | no | 1.7% | 51.5% | +| e2e_ms:p90 | tp2_mns64 | tp1_mns32 | no | 5.5% | 51.5% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/real-audit.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/real-audit.json new file mode 100644 index 0000000..035bf0b --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/real-audit.json @@ -0,0 +1,1298 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "tpot_ms", + "e2e_ms" + ], + "case_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd", + "configs": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3244.8699812067493, + "pooled_p90_ms": 3425.266431993805, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3244.8699812067493, + "trial_stddev_of_means_ms": 54.58553425230737 + }, + "tpot_ms": { + "pooled_mean_ms": 11.469757659740084, + "pooled_p90_ms": 12.171326144872342, + "pooled_samples": 771, + "trial_mean_of_means_ms": 11.469757659740083, + "trial_stddev_of_means_ms": 0.21211086007491214 + }, + "ttft_ms": { + "pooled_mean_ms": 320.0479193761807, + "pooled_p90_ms": 323.83561704773456, + "pooled_samples": 771, + "trial_mean_of_means_ms": 320.0479193761807, + "trial_stddev_of_means_ms": 1.3881860518161795 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3193.4511822196273, + "p90_ms": 3384.1337399790064, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.273388112562651, + "p90_ms": 12.023557262842123, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 318.70415179214734, + "p90_ms": 322.2650409443304, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3239.0094443264234, + "p90_ms": 3415.5735310632735, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.441173873868195, + "p90_ms": 12.136339439152211, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 321.4766235065425, + "p90_ms": 325.45425510033965, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3302.1493170741974, + "p90_ms": 3488.414889900014, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.694710992789403, + "p90_ms": 12.419273937577564, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 319.96298282985225, + "p90_ms": 323.75812309328467, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3203.7226679738205, + "pooled_p90_ms": 3389.1853449167684, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3203.722667973821, + "trial_stddev_of_means_ms": 37.62095572231703 + }, + "tpot_ms": { + "pooled_mean_ms": 11.31386031557959, + "pooled_p90_ms": 12.033569831035885, + "pooled_samples": 771, + "trial_mean_of_means_ms": 11.31386031557959, + "trial_stddev_of_means_ms": 0.13821363848478146 + }, + "ttft_ms": { + "pooled_mean_ms": 318.65480573091116, + "pooled_p90_ms": 322.9008379857987, + "pooled_samples": 771, + "trial_mean_of_means_ms": 318.65480573091116, + "trial_stddev_of_means_ms": 2.378102510617389 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3163.616114210406, + "p90_ms": 3361.8536689318717, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.166290408409983, + "p90_ms": 11.937463062577972, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 316.1799317087203, + "p90_ms": 320.07085403893143, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3238.2305139731907, + "p90_ms": 3415.6174729578197, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.440279400074546, + "p90_ms": 12.137762799986875, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 320.92259243105315, + "p90_ms": 324.3406399851665, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3209.3213757378653, + "p90_ms": 3402.1775929722935, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.33501113825424, + "p90_ms": 12.083764478345127, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 318.86189305296, + "p90_ms": 322.42463901638985, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3260.1686794512834, + "pooled_p90_ms": 3455.085408058949, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3260.168679451284, + "trial_stddev_of_means_ms": 40.661689073362574 + }, + "tpot_ms": { + "pooled_mean_ms": 11.527938706280123, + "pooled_p90_ms": 12.289238744871874, + "pooled_samples": 771, + "trial_mean_of_means_ms": 11.52793870628012, + "trial_stddev_of_means_ms": 0.15742476320695403 + }, + "ttft_ms": { + "pooled_mean_ms": 320.50949874761045, + "pooled_p90_ms": 324.3287180084735, + "pooled_samples": 771, + "trial_mean_of_means_ms": 320.50949874761045, + "trial_stddev_of_means_ms": 1.1136627349407815 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3238.780882805658, + "p90_ms": 3410.9200169332325, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.449106841169336, + "p90_ms": 12.123953999823653, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 319.22388932548085, + "p90_ms": 322.88106298074126, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3307.060569075864, + "p90_ms": 3503.4443179611117, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.709205648183815, + "p90_ms": 12.485718306190535, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 321.1778181112201, + "p90_ms": 324.64756595436484, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3234.6645864723287, + "p90_ms": 3445.0729619711637, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.425503629487213, + "p90_ms": 12.253773529721679, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 321.1267888061304, + "p90_ms": 324.92771092802286, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3245.776986038003, + "pooled_p90_ms": 3423.4000670257956, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3245.776986038003, + "trial_stddev_of_means_ms": 56.42282161792048 + }, + "tpot_ms": { + "pooled_mean_ms": 11.482428719576262, + "pooled_p90_ms": 12.175601455109085, + "pooled_samples": 771, + "trial_mean_of_means_ms": 11.482428719576262, + "trial_stddev_of_means_ms": 0.2240148980484202 + }, + "ttft_ms": { + "pooled_mean_ms": 317.7225427747669, + "pooled_p90_ms": 322.28669500909746, + "pooled_samples": 771, + "trial_mean_of_means_ms": 317.7225427747669, + "trial_stddev_of_means_ms": 2.390798892755192 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3182.410548419822, + "p90_ms": 3377.4496220285073, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.228627621663684, + "p90_ms": 11.996991521514515, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 319.07594205896777, + "p90_ms": 322.90874095633626, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3290.577206677297, + "p90_ms": 3467.5696010235697, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.652589256644216, + "p90_ms": 12.3542864939344, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 319.1296262616755, + "p90_ms": 322.9702590033412, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3264.34320301689, + "p90_ms": 3448.5452079679817, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 11.566069280420887, + "p90_ms": 12.271564404078413, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 314.96206000365737, + "p90_ms": 318.7974439933896, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp1_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3894.695937868006, + "pooled_p90_ms": 4727.010502945632, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3894.6959378680062, + "trial_stddev_of_means_ms": 321.92084658162594 + }, + "tpot_ms": { + "pooled_mean_ms": 14.481473999748076, + "pooled_p90_ms": 17.730250070784606, + "pooled_samples": 771, + "trial_mean_of_means_ms": 14.481473999748076, + "trial_stddev_of_means_ms": 1.2582346470555563 + }, + "ttft_ms": { + "pooled_mean_ms": 201.88473135410493, + "pooled_p90_ms": 207.47957099229097, + "pooled_samples": 771, + "trial_mean_of_means_ms": 201.88473135410493, + "trial_stddev_of_means_ms": 1.1331300161517917 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3589.8835412693184, + "p90_ms": 4331.327080959454, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 13.28914437973162, + "p90_ms": 16.179336294276165, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 201.11680352527662, + "p90_ms": 206.07955392915756, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 4231.357620817479, + "p90_ms": 4734.955659951083, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 15.796608200603078, + "p90_ms": 17.764389682469854, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 203.18613669487144, + "p90_ms": 208.71902198996395, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3862.846651517221, + "p90_ms": 4734.006996965036, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 14.35866941890953, + "p90_ms": 17.76524113317696, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 201.35125384216673, + "p90_ms": 206.80058794096112, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3990.0007042352863, + "pooled_p90_ms": 4719.314776011743, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3990.0007042352863, + "trial_stddev_of_means_ms": 337.87670595445104 + }, + "tpot_ms": { + "pooled_mean_ms": 14.855925269877513, + "pooled_p90_ms": 17.70052198393672, + "pooled_samples": 771, + "trial_mean_of_means_ms": 14.855925269877515, + "trial_stddev_of_means_ms": 1.318249751801272 + }, + "ttft_ms": { + "pooled_mean_ms": 201.7057607532932, + "pooled_p90_ms": 207.7565270010382, + "pooled_samples": 771, + "trial_mean_of_means_ms": 201.70576075329322, + "trial_stddev_of_means_ms": 1.7268857493870262 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 4231.524751395266, + "p90_ms": 4735.202983021736, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 15.798723248864277, + "p90_ms": 17.75633337651836, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 202.81657537468982, + "p90_ms": 208.523430977948, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3603.8893459863953, + "p90_ms": 4285.463593085296, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 13.349571505411829, + "p90_ms": 16.020474662346874, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 199.7162326146642, + "p90_ms": 204.20879207085818, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 4134.588015324197, + "p90_ms": 4732.604659046046, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 15.419481055356435, + "p90_ms": 17.74456359208653, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 202.58447427052556, + "p90_ms": 208.65151204634458, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3354.8816545874283, + "pooled_p90_ms": 3575.366784934886, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3354.8816545874283, + "trial_stddev_of_means_ms": 72.1597865775507 + }, + "tpot_ms": { + "pooled_mean_ms": 12.393283087182931, + "pooled_p90_ms": 13.247703140893696, + "pooled_samples": 771, + "trial_mean_of_means_ms": 12.393283087182931, + "trial_stddev_of_means_ms": 0.2814906609729632 + }, + "ttft_ms": { + "pooled_mean_ms": 194.56193759030603, + "pooled_p90_ms": 198.1358069460839, + "pooled_samples": 771, + "trial_mean_of_means_ms": 194.56193759030603, + "trial_stddev_of_means_ms": 0.3780334869245422 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3309.949615624179, + "p90_ms": 3545.9875999949872, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.218028092730933, + "p90_ms": 13.146208078308286, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 194.32007442482163, + "p90_ms": 197.8240980533883, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3316.578695923362, + "p90_ms": 3547.178103006445, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.243842540580335, + "p90_ms": 13.162248380317846, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 194.3681693571558, + "p90_ms": 197.80025607906282, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3438.1166522147437, + "p90_ms": 3960.4067360050976, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.71797862823753, + "p90_ms": 14.748573207807745, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 194.99756898894066, + "p90_ms": 198.64740199409425, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3300.440718642715, + "pooled_p90_ms": 3546.2497010594234, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3300.4407186427147, + "trial_stddev_of_means_ms": 5.520242664690669 + }, + "tpot_ms": { + "pooled_mean_ms": 12.177555298535195, + "pooled_p90_ms": 13.137051212035265, + "pooled_samples": 771, + "trial_mean_of_means_ms": 12.177555298535195, + "trial_stddev_of_means_ms": 0.02014276811960413 + }, + "ttft_ms": { + "pooled_mean_ms": 195.13009636540295, + "pooled_p90_ms": 198.6226630397141, + "pooled_samples": 771, + "trial_mean_of_means_ms": 195.13009636540292, + "trial_stddev_of_means_ms": 0.6728905571074729 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3302.7315898458673, + "p90_ms": 3547.509171999991, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.188114766245736, + "p90_ms": 13.144916544814466, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 194.72804888747706, + "p90_ms": 198.29899398609996, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3304.4466916761626, + "p90_ms": 3553.687784005888, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.190222830175609, + "p90_ms": 13.150935062645551, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 195.90692395201236, + "p90_ms": 199.7501190053299, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3294.143874406115, + "p90_ms": 3543.0204999865964, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.154328299184236, + "p90_ms": 13.12024300397538, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 194.75531625671942, + "p90_ms": 198.02888797130436, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp2_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3252.7993474094965, + "pooled_p90_ms": 3511.0875230748206, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3252.7993474094965, + "trial_stddev_of_means_ms": 13.124080948502684 + }, + "tpot_ms": { + "pooled_mean_ms": 12.267737219892378, + "pooled_p90_ms": 13.2822172821262, + "pooled_samples": 771, + "trial_mean_of_means_ms": 12.26773721989238, + "trial_stddev_of_means_ms": 0.04262875248570821 + }, + "ttft_ms": { + "pooled_mean_ms": 124.49453797538061, + "pooled_p90_ms": 127.14553298428655, + "pooled_samples": 771, + "trial_mean_of_means_ms": 124.4945379753806, + "trial_stddev_of_means_ms": 2.4015198039164893 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3266.5248520723053, + "p90_ms": 3540.4481209116057, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.310757102663121, + "p90_ms": 13.360472113423635, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 127.24864069084322, + "p90_ms": 131.06893200892955, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3240.373377299121, + "p90_ms": 3492.592430091463, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.225510671562784, + "p90_ms": 13.21439097279354, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 122.83733301136483, + "p90_ms": 126.26808194909245, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3251.4998128570633, + "p90_ms": 3508.000510977581, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.266943885451232, + "p90_ms": 13.268142372515856, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 123.39764022393376, + "p90_ms": 126.87448901124299, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3227.015463910283, + "pooled_p90_ms": 3487.1332910843194, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3227.0154639102825, + "trial_stddev_of_means_ms": 9.757849039220336 + }, + "tpot_ms": { + "pooled_mean_ms": 12.171863019540417, + "pooled_p90_ms": 13.192119505986863, + "pooled_samples": 771, + "trial_mean_of_means_ms": 12.171863019540416, + "trial_stddev_of_means_ms": 0.03973904147877756 + }, + "ttft_ms": { + "pooled_mean_ms": 123.15859724408143, + "pooled_p90_ms": 126.9288290059194, + "pooled_samples": 771, + "trial_mean_of_means_ms": 123.15859724408143, + "trial_stddev_of_means_ms": 0.49114501616099215 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3219.0692696932356, + "p90_ms": 3485.3774840012193, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.140566183956079, + "p90_ms": 13.175986364812536, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 123.19124268154081, + "p90_ms": 126.52519403491169, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3237.906589348536, + "p90_ms": 3513.072752044536, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.216572966532839, + "p90_ms": 13.287458070279921, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 122.65194388847537, + "p90_ms": 126.55710394028574, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3224.0705326890775, + "p90_ms": 3485.7966180425137, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.158449908132331, + "p90_ms": 13.179041823262677, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 123.63260516222813, + "p90_ms": 127.68025300465524, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 3258.202659738964, + "pooled_p90_ms": 3510.0540370913222, + "pooled_samples": 771, + "trial_mean_of_means_ms": 3258.2026597389645, + "trial_stddev_of_means_ms": 26.16015239977281 + }, + "tpot_ms": { + "pooled_mean_ms": 12.29409478300949, + "pooled_p90_ms": 13.276502498261193, + "pooled_samples": 771, + "trial_mean_of_means_ms": 12.29409478300949, + "trial_stddev_of_means_ms": 0.10176478535551588 + }, + "ttft_ms": { + "pooled_mean_ms": 123.17715882596677, + "pooled_p90_ms": 126.84211903251708, + "pooled_samples": 771, + "trial_mean_of_means_ms": 123.17715882596677, + "trial_stddev_of_means_ms": 0.21946599752752585 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 3248.8852222309656, + "p90_ms": 3485.874078935012, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.257590223944844, + "p90_ms": 13.18505671395756, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 123.16796821379012, + "p90_ms": 127.00659106485546, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3287.745963928384, + "p90_ms": 3542.219298076816, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.409076760226009, + "p90_ms": 13.403489419222609, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 123.40107575332536, + "p90_ms": 126.91527209244668, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 3237.9767930575426, + "p90_ms": 3510.0540370913222, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 12.215617364857616, + "p90_ms": 13.270442501879206, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 122.96243251078484, + "p90_ms": 126.39114900957793, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 9930.415717170099, + "pooled_p90_ms": 16230.610172031447, + "pooled_samples": 771, + "trial_mean_of_means_ms": 9930.415717170099, + "trial_stddev_of_means_ms": 265.43228327053174 + }, + "tpot_ms": { + "pooled_mean_ms": 8.417185619746073, + "pooled_p90_ms": 8.771115207734207, + "pooled_samples": 771, + "trial_mean_of_means_ms": 8.417185619746073, + "trial_stddev_of_means_ms": 0.07114582388337083 + }, + "ttft_ms": { + "pooled_mean_ms": 7783.999912441683, + "pooled_p90_ms": 14005.500467028469, + "pooled_samples": 771, + "trial_mean_of_means_ms": 7783.999912441683, + "trial_stddev_of_means_ms": 247.57894746806267 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 9796.72206207704, + "p90_ms": 15780.092388973571, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 8.369231803923293, + "p90_ms": 8.733943933347131, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 7662.533099055091, + "p90_ms": 13565.21666701883, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 10236.111454999053, + "p90_ms": 16542.091568931937, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 8.498929749479789, + "p90_ms": 8.837188466671197, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 8068.853142362809, + "p90_ms": 14296.956199919805, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 9758.413634434204, + "p90_ms": 15677.250495995395, + "samples": 257 + }, + "tpot_ms": { + "mean_ms": 8.383395305835137, + "p90_ms": 8.721436341019238, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 7620.613495907149, + "p90_ms": 13464.818625012413, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-pd/real/tp4_mns8/trial3/results/result.json", + "trial": 3 + } + ] + } + }, + "prefill_only": false, + "schema": "qwen30-latency-case-real-audit-v1", + "trace_manifests": { + "tp1": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 1.125, + "last_arrival_s": 227.55555555555554, + "output_tokens": [ + 256 + ], + "per_gpu_offered_request_rate": 1.125, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd/tp1/private/real_requests.jsonl", + "private_jsonl_sha256": "86cf52a5016ff06c3d5728605a63fc5b5f1712a9e8d17a0b2935a21f991ca4c3", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd/tp1/public/frontier.csv", + "public_csv_sha256": "8257aefe7b57cd24056cde49e07a1be8a5a05c3279742bc71439671626a06051", + "requests": 257, + "row_vector_sha256": "a92a626fc5d89c625b2f63dbb353165a544e07c21523027103ffe1bc9a4f64ae", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 1 + }, + "tp2": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 2.25, + "last_arrival_s": 113.77777777777777, + "output_tokens": [ + 256 + ], + "per_gpu_offered_request_rate": 1.125, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd/tp2/private/real_requests.jsonl", + "private_jsonl_sha256": "b60dd3dd128b18905497a50223dfd7a88cad6e262f650794ac2a71cb4db0608f", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd/tp2/public/frontier.csv", + "public_csv_sha256": "3a2250170a362bd640416309fc04a4cb503454f1e01554b2af7d1577d21be554", + "requests": 257, + "row_vector_sha256": "1c92d235e3f7006367e2e0ad1a70ef4a8cf472b1fecc8aebae7f95218c20205d", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 2 + }, + "tp4": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 4.5, + "last_arrival_s": 56.888888888888886, + "output_tokens": [ + 256 + ], + "per_gpu_offered_request_rate": 1.125, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd/tp4/private/real_requests.jsonl", + "private_jsonl_sha256": "aa08175b73187df17f69ef13946e947929ddf40cc0b5caf53e8209b9c342e174", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd/tp4/public/frontier.csv", + "public_csv_sha256": "5fbb14f2d04287096d9e5902db94041167cf6926d8c155dfc7d4b831fa94048c", + "requests": 257, + "row_vector_sha256": "2b6ecbd4c729db310300082f8362826c0ec407ae2451751c340dcf81c7a11859", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 4 + } + }, + "traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-pd", + "winners": { + "e2e_ms:pooled_mean_ms": { + "ranking": [ + "tp1_mns32", + "tp4_mns32", + "tp1_mns16", + "tp1_mns8", + "tp4_mns16", + "tp4_mns64", + "tp1_mns64", + "tp2_mns8", + "tp2_mns64", + "tp2_mns16", + "tp2_mns32", + "tp4_mns8" + ], + "winner": "tp1_mns32", + "winner_value_ms": 3203.7226679738205 + }, + "e2e_ms:pooled_p90_ms": { + "ranking": [ + "tp1_mns32", + "tp1_mns8", + "tp1_mns16", + "tp1_mns64", + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp4_mns8" + ], + "winner": "tp1_mns32", + "winner_value_ms": 3389.1853449167684 + }, + "tpot_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64", + "tp4_mns32", + "tp2_mns8", + "tp4_mns16", + "tp4_mns64", + "tp2_mns64", + "tp2_mns16", + "tp2_mns32" + ], + "winner": "tp4_mns8", + "winner_value_ms": 8.417185619746073 + }, + "tpot_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns8", + "tp1_mns64", + "tp2_mns8", + "tp4_mns32", + "tp2_mns64", + "tp4_mns64", + "tp4_mns16", + "tp2_mns32", + "tp2_mns16" + ], + "winner": "tp4_mns8", + "winner_value_ms": 8.771115207734207 + }, + "ttft_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns32", + "tp2_mns16", + "tp1_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp4_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 123.15859724408143 + }, + "ttft_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns64", + "tp2_mns8", + "tp2_mns16", + "tp2_mns32", + "tp1_mns8", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp4_mns8" + ], + "winner": "tp4_mns64", + "winner_value_ms": 126.84211903251708 + } + } +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/real-audit.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/real-audit.md new file mode 100644 index 0000000..cd51259 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-pd/real-audit.md @@ -0,0 +1,12 @@ +# Qwen30 real latency case audit + +Prefill-only: `False`. + +| Objective | Real winner | Value (ms) | +|---|---|---:| +| ttft_ms:pooled_mean_ms | tp4_mns32 | 123.16 | +| ttft_ms:pooled_p90_ms | tp4_mns64 | 126.84 | +| tpot_ms:pooled_mean_ms | tp4_mns8 | 8.42 | +| tpot_ms:pooled_p90_ms | tp4_mns8 | 8.77 | +| e2e_ms:pooled_mean_ms | tp1_mns32 | 3203.72 | +| e2e_ms:pooled_p90_ms | tp1_mns32 | 3389.19 | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/comparison.json new file mode 100644 index 0000000..9074b19 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/comparison.json @@ -0,0 +1,326 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "e2e_ms" + ], + "cells": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + }, + "ttft_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns16/tp1/result.json" + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + }, + "ttft_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns32/tp1/result.json" + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + }, + "ttft_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns64/tp1/result.json" + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + }, + "ttft_ms": { + "mean": 267.46878507222704, + "p90": 267.46878507223215 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp1_mns8/tp1/result.json" + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + }, + "ttft_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns16/tp2/result.json" + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + }, + "ttft_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns32/tp2/result.json" + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + }, + "ttft_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns64/tp2/result.json" + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + }, + "ttft_ms": { + "mean": 175.26431810890125, + "p90": 175.2643181089013 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp2_mns8/tp2/result.json" + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + }, + "ttft_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns16/tp4/result.json" + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + }, + "ttft_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns32/tp4/result.json" + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + }, + "ttft_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns64/tp4/result.json" + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + }, + "ttft_ms": { + "mean": 127.7919646347385, + "p90": 127.79196463473852 + } + }, + "path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po/runs/tp4_mns8/tp4/result.json" + } + }, + "prefill_only": true, + "real_audit": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/analysis/fixed-po/real-audit.json", + "schema": "qwen30-latency-case-frontier-real-comparison-v1", + "selection": { + "e2e_ms:mean": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.0034531419781375313, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "e2e_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.005277055274057908, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "ttft_ms:mean": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.003463041945107654, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + }, + "ttft_ms:p90": { + "agreement": 1.0, + "concordant_pairs": 48, + "discordant_pairs": 0, + "informative_pairs": 48, + "real_ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "real_winner": "tp4_mns32", + "selected_config_real_regret": 0.005266030780421245, + "sim_ranking": [ + "tp4_mns16", + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp1_mns8" + ], + "sim_winner": "tp4_mns16", + "tied_pairs": 18, + "winner_match": false + } + }, + "sim_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/sim/fixed-po" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/comparison.md new file mode 100644 index 0000000..d38638c --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/comparison.md @@ -0,0 +1,8 @@ +# Qwen30 Frontier vs real latency selection + +| Objective | Frontier | Real | Match | Regret | Pairwise | +|---|---|---|---:|---:|---:| +| ttft_ms:mean | tp4_mns16 | tp4_mns32 | no | 0.3% | 100.0% | +| ttft_ms:p90 | tp4_mns16 | tp4_mns32 | no | 0.5% | 100.0% | +| e2e_ms:mean | tp4_mns16 | tp4_mns32 | no | 0.3% | 100.0% | +| e2e_ms:p90 | tp4_mns16 | tp4_mns32 | no | 0.5% | 100.0% | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/real-audit.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/real-audit.json new file mode 100644 index 0000000..74c5c48 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/real-audit.json @@ -0,0 +1,997 @@ +{ + "applicable_metrics": [ + "ttft_ms", + "e2e_ms" + ], + "case_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po", + "configs": { + "tp1_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 311.5405924270961, + "pooled_p90_ms": 313.7628498952836, + "pooled_samples": 771, + "trial_mean_of_means_ms": 311.5405924270961, + "trial_stddev_of_means_ms": 1.3039412575877052 + }, + "ttft_ms": { + "pooled_mean_ms": 311.49589566449873, + "pooled_p90_ms": 313.7203580699861, + "pooled_samples": 771, + "trial_mean_of_means_ms": 311.49589566449873, + "trial_stddev_of_means_ms": 1.3039649961427384 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 310.6955070939125, + "p90_ms": 311.31368898786604, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 310.65253322356983, + "p90_ms": 311.2668279791251, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 310.8839521212648, + "p90_ms": 311.4629740593955, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 310.837354891071, + "p90_ms": 311.4192739594728, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 313.04231806611097, + "p90_ms": 314.1881680348888, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 312.99779887885535, + "p90_ms": 314.14560007397085, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 311.5105919916504, + "pooled_p90_ms": 312.67331005074084, + "pooled_samples": 771, + "trial_mean_of_means_ms": 311.5105919916504, + "trial_stddev_of_means_ms": 0.8370728024636384 + }, + "ttft_ms": { + "pooled_mean_ms": 311.4647780706316, + "pooled_p90_ms": 312.62694601900876, + "pooled_samples": 771, + "trial_mean_of_means_ms": 311.4647780706316, + "trial_stddev_of_means_ms": 0.8370818779367518 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 311.46654921424386, + "p90_ms": 311.97385501582175, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 311.4168197754589, + "p90_ms": 311.92471098620445, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 310.69641002473793, + "p90_ms": 311.3162890076637, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 310.6527063394951, + "p90_ms": 311.2770210718736, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 312.36881673596946, + "p90_ms": 313.099671038799, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 312.3248080969408, + "p90_ms": 313.04874701891094, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 312.1343500969265, + "pooled_p90_ms": 313.89150698669255, + "pooled_samples": 771, + "trial_mean_of_means_ms": 312.1343500969265, + "trial_stddev_of_means_ms": 0.46642112108308753 + }, + "ttft_ms": { + "pooled_mean_ms": 312.0841491107084, + "pooled_p90_ms": 313.84513492230326, + "pooled_samples": 771, + "trial_mean_of_means_ms": 312.0841491107083, + "trial_stddev_of_means_ms": 0.46656234571063265 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 311.59666514998946, + "p90_ms": 313.3960220729932, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 311.546319629997, + "p90_ms": 313.33198107313365, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 312.4300223219857, + "p90_ms": 314.37386700417846, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 312.3801749132588, + "p90_ms": 314.32278105057776, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 312.37636281880424, + "p90_ms": 313.45780298579484, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 312.32595278886924, + "p90_ms": 313.3977819234133, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp1_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 312.4610948317168, + "pooled_p90_ms": 314.16700792033225, + "pooled_samples": 771, + "trial_mean_of_means_ms": 312.4610948317168, + "trial_stddev_of_means_ms": 0.7149524029740935 + }, + "ttft_ms": { + "pooled_mean_ms": 312.4132300384316, + "pooled_p90_ms": 314.1121099470183, + "pooled_samples": 771, + "trial_mean_of_means_ms": 312.4132300384316, + "trial_stddev_of_means_ms": 0.7170007678929775 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 313.2865619732444, + "p90_ms": 314.3677569460124, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 313.24100631591773, + "p90_ms": 314.32040000800043, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 312.0378761661895, + "p90_ms": 313.87826695572585, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 311.9859282132999, + "p90_ms": 313.8229319592938, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 312.05884635571664, + "p90_ms": 314.0185490483418, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 312.01275558607716, + "p90_ms": 313.97431797813624, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp1_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 190.31096146856925, + "pooled_p90_ms": 191.60501298028976, + "pooled_samples": 771, + "trial_mean_of_means_ms": 190.31096146856922, + "trial_stddev_of_means_ms": 0.5354377240256494 + }, + "ttft_ms": { + "pooled_mean_ms": 190.26420441538883, + "pooled_p90_ms": 191.55984604731202, + "pooled_samples": 771, + "trial_mean_of_means_ms": 190.26420441538883, + "trial_stddev_of_means_ms": 0.5389874750392776 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 190.07371114938363, + "p90_ms": 191.55416695866734, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 190.0275621526052, + "p90_ms": 191.50883494876325, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 190.92403356246794, + "p90_ms": 191.83196197263896, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 190.88103114138084, + "p90_ms": 191.79061893373728, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 189.93513969385612, + "p90_ms": 191.3305459311232, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 189.88401995218044, + "p90_ms": 191.23233994469047, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 190.1976155063099, + "pooled_p90_ms": 191.3924120599404, + "pooled_samples": 771, + "trial_mean_of_means_ms": 190.19761550630986, + "trial_stddev_of_means_ms": 0.4305067654025827 + }, + "ttft_ms": { + "pooled_mean_ms": 190.15199505472157, + "pooled_p90_ms": 191.34706107433885, + "pooled_samples": 771, + "trial_mean_of_means_ms": 190.1519950547216, + "trial_stddev_of_means_ms": 0.4287092219589841 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 190.51239040683853, + "p90_ms": 191.31595105864108, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 190.46796200088505, + "p90_ms": 191.2707129959017, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 189.70702594537505, + "p90_ms": 191.25314499251544, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 189.66398845564444, + "p90_ms": 191.21390499640256, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 190.37343016671605, + "p90_ms": 191.65610091295093, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 190.32403470763524, + "p90_ms": 191.5953279240057, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 188.49150093478283, + "pooled_p90_ms": 190.28858200181276, + "pooled_samples": 771, + "trial_mean_of_means_ms": 188.49150093478283, + "trial_stddev_of_means_ms": 1.6533226552993832 + }, + "ttft_ms": { + "pooled_mean_ms": 188.44684911725727, + "pooled_p90_ms": 190.2463739970699, + "pooled_samples": 771, + "trial_mean_of_means_ms": 188.44684911725724, + "trial_stddev_of_means_ms": 1.650697495689325 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 189.15035290614583, + "p90_ms": 190.26379100978374, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 189.106691004038, + "p90_ms": 190.2244269149378, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 189.7138195680382, + "p90_ms": 190.8034710213542, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 189.66555941971345, + "p90_ms": 190.7602889696136, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 186.61033033016446, + "p90_ms": 187.4971800716594, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 186.56829692802035, + "p90_ms": 187.45643505826592, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp2_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 185.58736601101546, + "pooled_p90_ms": 186.94222904741764, + "pooled_samples": 771, + "trial_mean_of_means_ms": 185.58736601101546, + "trial_stddev_of_means_ms": 0.17900602117493175 + }, + "ttft_ms": { + "pooled_mean_ms": 185.54133709128124, + "pooled_p90_ms": 186.88921502325684, + "pooled_samples": 771, + "trial_mean_of_means_ms": 185.54133709128124, + "trial_stddev_of_means_ms": 0.17412945433635296 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 185.46277399905068, + "p90_ms": 186.92227499559522, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 185.41920935401714, + "p90_ms": 186.8766910629347, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 185.79249331095266, + "p90_ms": 187.07626103423536, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 185.74072911579827, + "p90_ms": 187.01596499886364, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 185.50683072304304, + "p90_ms": 186.73917709384114, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 185.46407280402832, + "p90_ms": 186.69728806708008, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp2_mns8/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns16": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 113.83082875217904, + "pooled_p90_ms": 115.34429900348186, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.83082875217904, + "trial_stddev_of_means_ms": 0.9461418127163538 + }, + "ttft_ms": { + "pooled_mean_ms": 113.78785713766635, + "pooled_p90_ms": 115.29833707027137, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.78785713766636, + "trial_stddev_of_means_ms": 0.9438950942019103 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 114.32171626612616, + "p90_ms": 115.71828299202025, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 114.27366546969427, + "p90_ms": 115.66144204698503, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns16/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 112.74012975786098, + "p90_ms": 113.76826802734286, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 112.70000880352197, + "p90_ms": 113.72840602416545, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns16/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 114.43064023254998, + "p90_ms": 115.64736906439066, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 114.38989713978283, + "p90_ms": 115.60670402832329, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns16/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns32": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 113.43910740842456, + "pooled_p90_ms": 114.73881592974067, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.43910740842455, + "trial_stddev_of_means_ms": 0.15342403653521097 + }, + "ttft_ms": { + "pooled_mean_ms": 113.39516492515813, + "pooled_p90_ms": 114.69435307662934, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.39516492515811, + "trial_stddev_of_means_ms": 0.15129782609682924 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.3981759986996, + "p90_ms": 114.83101104386151, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.35384492909152, + "p90_ms": 114.78637403342873, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns32/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.60884601692398, + "p90_ms": 115.0080730440095, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.56283011230194, + "p90_ms": 114.94788806885481, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns32/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.3103002096501, + "p90_ms": 114.50691497884691, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.2688197340809, + "p90_ms": 114.4664689199999, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns32/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns64": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 113.57268320527, + "pooled_p90_ms": 115.08622206747532, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.57268320527, + "trial_stddev_of_means_ms": 0.1834744740593822 + }, + "ttft_ms": { + "pooled_mean_ms": 113.52880459490815, + "pooled_p90_ms": 115.04431103821844, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.52880459490815, + "trial_stddev_of_means_ms": 0.18387162741378107 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.70978420210734, + "p90_ms": 115.44408102054149, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.66415427361727, + "p90_ms": 115.39141100365669, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns64/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.64400897406998, + "p90_ms": 114.92579500190914, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.60279529025698, + "p90_ms": 114.8861589608714, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns64/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.36425643963268, + "p90_ms": 114.78174000512809, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.31946422085021, + "p90_ms": 114.745571045205, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns64/trial3/results/result.json", + "trial": 3 + } + ] + }, + "tp4_mns8": { + "metrics": { + "e2e_ms": { + "pooled_mean_ms": 113.80060324454426, + "pooled_p90_ms": 114.74568990524858, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.80060324454426, + "trial_stddev_of_means_ms": 0.1793280070989901 + }, + "ttft_ms": { + "pooled_mean_ms": 113.75673118469592, + "pooled_p90_ms": 114.70199096947908, + "pooled_samples": 771, + "trial_mean_of_means_ms": 113.75673118469592, + "trial_stddev_of_means_ms": 0.17717556416711291 + } + }, + "trials": [ + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.60963180313314, + "p90_ms": 114.72724401392043, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.56837538357259, + "p90_ms": 114.6864800248295, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns8/trial1/results/result.json", + "trial": 1 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.82676389363706, + "p90_ms": 115.33373803831637, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.78175165850293, + "p90_ms": 115.29060895554721, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns8/trial2/results/result.json", + "trial": 2 + }, + { + "metrics": { + "e2e_ms": { + "mean_ms": 113.96541403686258, + "p90_ms": 114.28226600401103, + "samples": 257 + }, + "ttft_ms": { + "mean_ms": 113.92006651201224, + "p90_ms": 114.24182495102286, + "samples": 257 + } + }, + "requests": 257, + "result_path": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/real/fixed-po/real/tp4_mns8/trial3/results/result.json", + "trial": 3 + } + ] + } + }, + "prefill_only": true, + "schema": "qwen30-latency-case-real-audit-v1", + "trace_manifests": { + "tp1": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 1.125, + "last_arrival_s": 227.55555555555554, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 1.125, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp1/private/real_requests.jsonl", + "private_jsonl_sha256": "eb048c65428f2530bdea135e6607405fd7ebaefb103267daaec29839462aa3ab", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp1/public/frontier.csv", + "public_csv_sha256": "82c186222779c4a176f18e425d371966012ca7c1c6560f13d2c7036ac4f38d86", + "requests": 257, + "row_vector_sha256": "9829ec5834913fa49f900f4e07c8e83206726fc5a70d88de1498426ec91c51f3", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 1 + }, + "tp2": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 2.25, + "last_arrival_s": 113.77777777777777, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 1.125, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp2/private/real_requests.jsonl", + "private_jsonl_sha256": "aa61aed87875ed792c37efdd62bfeee4753bd3de110a1de92fa32273be13ca12", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp2/public/frontier.csv", + "public_csv_sha256": "b32676281c8f5651f47ee3d88a88d433a01a0863d8794751d2d638b988e1b8b2", + "requests": 257, + "row_vector_sha256": "1c032691cc4624b125e5589ee8466e46a67ddad7fe9e7380d5857bb81633cc3c", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 2 + }, + "tp4": { + "description": "fixed-shape: deterministic token IDs, uniform TP-normalized QPS, no prefix reuse", + "first_arrival_s": 0.0, + "global_offered_request_rate": 4.5, + "last_arrival_s": 56.888888888888886, + "output_tokens": [ + 1 + ], + "per_gpu_offered_request_rate": 1.125, + "prefix_caching": false, + "private_jsonl": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp4/private/real_requests.jsonl", + "private_jsonl_sha256": "b82a17cd707c3c54069298b764031a1c18775571aa6e54c55be0746dd1fe94ca", + "public_csv": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po/tp4/public/frontier.csv", + "public_csv_sha256": "67429bf430436a23979a0007707a03f4c67af962b9466cd230294a3c62696931", + "requests": 257, + "row_vector_sha256": "ab5af2eddc7d07420c6fcdf28b3a014d7241b2b35471ee581c6d7a8e3afb93e6", + "schema": "qwen30-latency-case-v1", + "tensor_parallel_size": 4 + } + }, + "traces_root": "/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/traces/fixed-po", + "winners": { + "e2e_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 113.43910740842456 + }, + "e2e_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 114.73881592974067 + }, + "ttft_ms:pooled_mean_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 113.39516492515813 + }, + "ttft_ms:pooled_p90_ms": { + "ranking": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64", + "tp4_mns16", + "tp2_mns8", + "tp2_mns64", + "tp2_mns32", + "tp2_mns16", + "tp1_mns32", + "tp1_mns16", + "tp1_mns64", + "tp1_mns8" + ], + "winner": "tp4_mns32", + "winner_value_ms": 114.69435307662934 + } + } +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/real-audit.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/real-audit.md new file mode 100644 index 0000000..44d6ecb --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-fixed-hi/fixed-po/real-audit.md @@ -0,0 +1,10 @@ +# Qwen30 real latency case audit + +Prefill-only: `True`. + +| Objective | Real winner | Value (ms) | +|---|---|---:| +| ttft_ms:pooled_mean_ms | tp4_mns32 | 113.40 | +| ttft_ms:pooled_p90_ms | tp4_mns32 | 114.69 | +| e2e_ms:pooled_mean_ms | tp4_mns32 | 113.44 | +| e2e_ms:pooled_p90_ms | tp4_mns32 | 114.74 | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/comparison.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/comparison.json new file mode 100644 index 0000000..f07a37a --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/comparison.json @@ -0,0 +1,726 @@ +{ + "cells": { + "tp1_mns16": { + "config": "tp1_mns16", + "elapsed_seconds": 351.88990092277527, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 13.842098628638448, + "p90_ratio_sim_over_real": 13.072241203568879, + "real_mean_ms": 87355.16047904849, + "real_p90_ms": 156177.17167094816, + "sim_mean_ms": 1209178.7470715286, + "sim_p90_ms": 2041585.6585738186 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.816528397308329, + "p90_ratio_sim_over_real": 4.062740013130375, + "real_mean_ms": 20.58492915882586, + "real_p90_ms": 25.316827059621797, + "sim_mean_ms": 99.14789585006503, + "sim_p90_ms": 102.85568630062728 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 50.0951473194418, + "p90_ratio_sim_over_real": 40.15634837465801, + "real_mean_ms": 17378.17525871318, + "real_p90_ms": 42739.809082006104, + "sim_mean_ms": 870562.2497283154, + "sim_p90_ms": 1716274.6629634094 + } + }, + "mns": 16, + "offered_request_rate": 0.215, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns16/tp1/result.json", + "result_sha256": "d892cbc79c7ac3a2158310d346aea87fdd4a4515ada6465d02087066e412fe89", + "tp": 1, + "trace_label": "tp1", + "trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a" + }, + "tp1_mns32": { + "config": "tp1_mns32", + "elapsed_seconds": 342.20933651924133, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 8.911637753525204, + "p90_ratio_sim_over_real": 7.8084062855710945, + "real_mean_ms": 81051.19884917597, + "real_p90_ms": 162908.61000702716, + "sim_mean_ms": 722298.9236327951, + "sim_p90_ms": 1272056.614352521 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 5.03371341949048, + "p90_ratio_sim_over_real": 4.17277168119042, + "real_mean_ms": 23.75712112165005, + "real_p90_ms": 29.99736783063116, + "sim_mean_ms": 119.5865393985106, + "sim_p90_ms": 125.1721669939102 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 510.8831583325707, + "p90_ratio_sim_over_real": 384.77102901825737, + "real_mean_ms": 620.4562074374239, + "real_p90_ms": 1931.474626995623, + "sim_mean_ms": 316980.6268626798, + "sim_p90_ms": 743175.4797517607 + } + }, + "mns": 32, + "offered_request_rate": 0.215, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns32/tp1/result.json", + "result_sha256": "babdafcbea1bd890e9d2af8043ef104fac262ef2b2ae6f9eca9ed6e3787824b8", + "tp": 1, + "trace_label": "tp1", + "trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a" + }, + "tp1_mns64": { + "config": "tp1_mns64", + "elapsed_seconds": 347.8728907108307, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 6.680377846313092, + "p90_ratio_sim_over_real": 5.889817911531751, + "real_mean_ms": 82253.7312629709, + "real_p90_ms": 165355.2623600699, + "sim_mean_ms": 549486.0041057414, + "sim_p90_ms": 973912.3860143718 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 5.715675260466855, + "p90_ratio_sim_over_real": 4.93808404053013, + "real_mean_ms": 24.114592194742286, + "real_p90_ms": 30.355145227612027, + "sim_mean_ms": 137.83117802373562, + "sim_p90_ms": 149.8962581964453 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 169.41590110493394, + "p90_ratio_sim_over_real": 153.2295314740789, + "real_mean_ms": 619.7960944975623, + "real_p90_ms": 1956.7296199966222, + "sim_mean_ms": 105003.31385062331, + "sim_p90_ms": 299828.76289353485 + } + }, + "mns": 64, + "offered_request_rate": 0.215, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns64/tp1/result.json", + "result_sha256": "a870a95f9f088e0c098025d3d7d4001409b28ab26f8633ff27ea878d2bbd50d2", + "tp": 1, + "trace_label": "tp1", + "trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a" + }, + "tp1_mns8": { + "config": "tp1_mns8", + "elapsed_seconds": 369.526850938797, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 13.991474955968794, + "p90_ratio_sim_over_real": 15.108002258550664, + "real_mean_ms": 137996.3827215202, + "real_p90_ms": 222136.04307791684, + "sim_mean_ms": 1930772.9328624348, + "sim_p90_ms": 3356031.8405266753 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 5.3194009645510905, + "p90_ratio_sim_over_real": 4.790770710747499, + "real_mean_ms": 14.282724020345517, + "real_p90_ms": 16.331995667962982, + "sim_mean_ms": 75.97553593024297, + "sim_p90_ms": 78.2428464941321 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 18.86307572133342, + "p90_ratio_sim_over_real": 18.60057703879594, + "real_mean_ms": 88479.94003801016, + "real_p90_ms": 168185.5451229494, + "sim_mean_ms": 1669003.8087560262, + "sim_p90_ms": 3128348.188871311 + } + }, + "mns": 8, + "offered_request_rate": 0.215, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp1_mns8/tp1/result.json", + "result_sha256": "0db2d1f388f20252bea08c982cd53c24c3d33c2d3b36b70bda82c4e501b694e6", + "tp": 1, + "trace_label": "tp1", + "trace_sha256": "0014176c946a4c838f2ba127915fda8f2728e8cbfed070a001800b2f02f02d2a" + }, + "tp2_mns16": { + "config": "tp2_mns16", + "elapsed_seconds": 282.5357069969177, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 9.98451139764714, + "p90_ratio_sim_over_real": 10.19790282414335, + "real_mean_ms": 82936.70731938194, + "real_p90_ms": 137726.37015290093, + "sim_mean_ms": 828082.499513694, + "sim_p90_ms": 1404520.1391412807 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.569770850102063, + "p90_ratio_sim_over_real": 4.120000340068115, + "real_mean_ms": 14.09701977962943, + "real_p90_ms": 16.149621762221777, + "sim_mean_ms": 64.42015006226278, + "sim_p90_ms": 66.53644715232515 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 17.620001287194583, + "p90_ratio_sim_over_real": 15.324208422508217, + "real_mean_ms": 34442.970551674625, + "real_p90_ms": 77922.3588249879, + "sim_mean_ms": 606885.185455312, + "sim_p90_ms": 1194098.467407587 + } + }, + "mns": 16, + "offered_request_rate": 0.43, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns16/tp2/result.json", + "result_sha256": "20b811f38251d62930977529a55c434d62aede91b22ccaeec812a493999edbba", + "tp": 2, + "trace_label": "tp2", + "trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef" + }, + "tp2_mns32": { + "config": "tp2_mns32", + "elapsed_seconds": 341.36838245391846, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 7.698077489599209, + "p90_ratio_sim_over_real": 7.1828008112452935, + "real_mean_ms": 62267.50376182025, + "real_p90_ms": 117239.33770309668, + "sim_mean_ms": 479340.0690424025, + "sim_p90_ms": 842106.8099636638 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.155703711688009, + "p90_ratio_sim_over_real": 3.6490372704567418, + "real_mean_ms": 17.99748422275332, + "real_p90_ms": 21.280696114935413, + "sim_mean_ms": 74.79221198554237, + "sim_p90_ms": 77.6540532646633 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 213.67619737350176, + "p90_ratio_sim_over_real": 204.45454462882793, + "real_mean_ms": 1050.6337350342683, + "real_p90_ms": 2521.7670870479196, + "sim_mean_ms": 224495.42133444166, + "sim_p90_ms": 515586.7414423483 + } + }, + "mns": 32, + "offered_request_rate": 0.43, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns32/tp2/result.json", + "result_sha256": "38b6ff12a60024e98bbb27ddea682439059b5b8481b661bd7b1ac720b3351b7d", + "tp": 2, + "trace_label": "tp2", + "trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef" + }, + "tp2_mns64": { + "config": "tp2_mns64", + "elapsed_seconds": 342.2184634208679, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 4.990247689497755, + "p90_ratio_sim_over_real": 4.806085707327923, + "real_mean_ms": 62353.63015175185, + "real_p90_ms": 118392.33276096638, + "sim_mean_ms": 311160.0587965772, + "sim_p90_ms": 569003.698339692 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.529200811393724, + "p90_ratio_sim_over_real": 4.064210807294688, + "real_mean_ms": 18.239052597893746, + "real_p90_ms": 21.560091553416488, + "sim_mean_ms": 82.60833182543317, + "sim_p90_ms": 87.6247570976582 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 81.23526532158742, + "p90_ratio_sim_over_real": 98.0031513549519, + "real_mean_ms": 375.75464864569483, + "real_p90_ms": 1148.593939957209, + "sim_mean_ms": 30524.52857855288, + "sim_p90_ms": 112565.82574300688 + } + }, + "mns": 64, + "offered_request_rate": 0.43, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns64/tp2/result.json", + "result_sha256": "2293c4ab15356f6f0753a0cec6e4113b4c74f89f7564032a0c1d58ebd97383ee", + "tp": 2, + "trace_label": "tp2", + "trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef" + }, + "tp2_mns8": { + "config": "tp2_mns8", + "elapsed_seconds": 369.0675718784332, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 10.511662746977503, + "p90_ratio_sim_over_real": 10.881956427090019, + "real_mean_ms": 128296.94038681783, + "real_p90_ms": 215848.26136403717, + "sim_mean_ms": 1348614.1688153066, + "sim_p90_ms": 2348851.3750265907 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 5.523719280021353, + "p90_ratio_sim_over_real": 5.149568696131382, + "real_mean_ms": 9.244370614307101, + "real_p90_ms": 10.190386958999568, + "sim_mean_ms": 51.06330819391097, + "sim_p90_ms": 52.476097685529645 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 12.214904033795035, + "p90_ratio_sim_over_real": 11.92062856279977, + "real_mean_ms": 95920.08463817694, + "real_p90_ms": 183878.83744493593, + "sim_mean_ms": 1171654.6287688287, + "sim_p90_ms": 2191951.3217405193 + } + }, + "mns": 8, + "offered_request_rate": 0.43, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp2_mns8/tp2/result.json", + "result_sha256": "0d75820000a471d607f5c0eac3571355d6666235e3bb1ae69080fa20fbf36160", + "tp": 2, + "trace_label": "tp2", + "trace_sha256": "8bab0da359e55feda0f228157a88a5ac2d0537881e8efbd55325f4dd343bf0ef" + }, + "tp4_mns16": { + "config": "tp4_mns16", + "elapsed_seconds": 353.6330111026764, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 8.790176207252758, + "p90_ratio_sim_over_real": 8.919807800227431, + "real_mean_ms": 65779.08377038215, + "real_p90_ms": 110374.3328540586, + "sim_mean_ms": 578209.7370932992, + "sim_p90_ms": 984517.8351365307 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.905169954916736, + "p90_ratio_sim_over_real": 4.616446999000005, + "real_mean_ms": 8.806983272811275, + "real_p90_ms": 9.626243685920313, + "sim_mean_ms": 43.19974974324813, + "sim_p90_ms": 44.439043775509575 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 12.245054441740509, + "p90_ratio_sim_over_real": 11.334316081110904, + "real_mean_ms": 35042.61012925094, + "real_p90_ms": 74281.28085809294, + "sim_mean_ms": 429098.66881336516, + "sim_p90_ms": 841927.5161553983 + } + }, + "mns": 16, + "offered_request_rate": 0.86, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns16/tp4/result.json", + "result_sha256": "c174b3fec57e9bebcd3e41a407c4ea17b6b44929f440336489e529c2ec4e1ffc", + "tp": 4, + "trace_label": "tp4", + "trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69" + }, + "tp4_mns32": { + "config": "tp4_mns32", + "elapsed_seconds": 342.3954610824585, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 7.0145630517428215, + "p90_ratio_sim_over_real": 6.923394844590732, + "real_mean_ms": 47944.52887711636, + "real_p90_ms": 83825.7212981116, + "sim_mean_ms": 336309.9207946372, + "sim_p90_ms": 580358.5666794453 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.058731155235696, + "p90_ratio_sim_over_real": 3.7164401630936927, + "real_mean_ms": 12.181116414313188, + "real_p90_ms": 13.75045588274547, + "sim_mean_ms": 49.43987669632586, + "sim_p90_ms": 51.1027465034832 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 26.481641572689746, + "p90_ratio_sim_over_real": 19.75959391638082, + "real_mean_ms": 6296.228585964807, + "real_p90_ms": 19159.870272967964, + "sim_mean_ms": 166734.46867324322, + "sim_p90_ms": 378591.2560843835 + } + }, + "mns": 32, + "offered_request_rate": 0.86, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns32/tp4/result.json", + "result_sha256": "46efd5bd348ce93a9dc075441ebfd1f2df5ff1646c53610e8f6bdd72105071e4", + "tp": 4, + "trace_label": "tp4", + "trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69" + }, + "tp4_mns64": { + "config": "tp4_mns64", + "elapsed_seconds": 340.36316752433777, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 4.663503091378936, + "p90_ratio_sim_over_real": 4.414200438612224, + "real_mean_ms": 44985.13828221082, + "real_p90_ms": 83763.61756504048, + "sim_mean_ms": 209788.3314451991, + "sim_p90_ms": 369749.3973953483 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 4.078641583467541, + "p90_ratio_sim_over_real": 3.6320345146531077, + "real_mean_ms": 13.178025610291787, + "real_p90_ms": 15.389331985205848, + "sim_mean_ms": 53.748443242136304, + "sim_p90_ms": 55.89458492772267 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 108.51050461378274, + "p90_ratio_sim_over_real": 152.34030824222364, + "real_mean_ms": 245.9527667526406, + "real_p90_ms": 685.5076659703627, + "sim_mean_ms": 26688.458831485037, + "sim_p90_ms": 104430.44913633233 + } + }, + "mns": 64, + "offered_request_rate": 0.86, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns64/tp4/result.json", + "result_sha256": "c6c34b800a646938f507dde0a5e2530f3e94358b9580050845cfd255176dd391", + "tp": 4, + "trace_label": "tp4", + "trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69" + }, + "tp4_mns8": { + "config": "tp4_mns8", + "elapsed_seconds": 379.0038528442383, + "metrics": { + "e2e_ms": { + "mean_ratio_sim_over_real": 7.5775112842809955, + "p90_ratio_sim_over_real": 7.997579501403945, + "real_mean_ms": 126405.41922729714, + "real_p90_ms": 209092.4072249327, + "sim_mean_ms": 957838.490589114, + "sim_p90_ms": 1672233.1499213278 + }, + "tpot_ms": { + "mean_ratio_sim_over_real": 5.242973269896895, + "p90_ratio_sim_over_real": 4.960553326136223, + "real_mean_ms": 6.757398869596114, + "real_p90_ms": 7.242299378971089, + "sim_mean_ms": 35.42886164732392, + "sim_p90_ms": 35.92581227322934 + }, + "ttft_ms": { + "mean_ratio_sim_over_real": 8.133065025278983, + "p90_ratio_sim_over_real": 8.084332290199812, + "real_mean_ms": 102605.87212059843, + "real_p90_ms": 193359.7398570273, + "sim_mean_ms": 834500.2299322869, + "sim_p90_ms": 1563184.3885508017 + } + }, + "mns": 8, + "offered_request_rate": 0.86, + "offered_request_rate_per_gpu": 0.215, + "request_count": 129, + "result_path": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/runs/tp4_mns8/tp4/result.json", + "result_sha256": "9361fe342d3b0394d6a9b130965304923a6edf424d0b964368079b0baacf9e75", + "tp": 4, + "trace_label": "tp4", + "trace_sha256": "cc2caa329990414496d3cbf6fb54af73e6e73027766deaf5885d56f4e362ab69" + } + }, + "contract": { + "configs": [ + "tp1_mns8", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp2_mns8", + "tp2_mns16", + "tp2_mns32", + "tp2_mns64", + "tp4_mns8", + "tp4_mns16", + "tp4_mns32", + "tp4_mns64" + ], + "graph_semantics": "Frontier piecewise; CUDA_EVENT for prefill/mixed and KERNEL_ONLY for captured decode", + "real_trials_per_cell": 3, + "request_count_per_cell": 129, + "trace_policy": "TP-normalized arrivals with full 16-token prefix blocks only" + }, + "real_audit": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/ground-truth/qwen30-tracepd-u0p01-real-audit.json", + "real_audit_sha256": "6e2c17bce5b0edb5f6f92aae4cc1b00f9390f692d31d059efe647158b386c8e9", + "schema": "frontier-qwen30-piecewise-graph-comparison-v1", + "selection": { + "e2e_ms:mean": { + "concordant_pairs": 61, + "discordant_pairs": 5, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.9242424242424242, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp2_mns32", + "tp2_mns64", + "tp4_mns16", + "tp1_mns32", + "tp1_mns64", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns64", + "tp4_mns16", + "tp1_mns32", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + }, + "e2e_ms:p90": { + "concordant_pairs": 55, + "discordant_pairs": 11, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.8333333333333334, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns64", + "tp4_mns16", + "tp1_mns32", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + }, + "tpot_ms:mean": { + "concordant_pairs": 64, + "discordant_pairs": 2, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.9696969696969697, + "real_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns16", + "tp1_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "sim_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp4_mns32", + "tp2_mns8", + "tp4_mns64", + "tp2_mns16", + "tp2_mns32", + "tp1_mns8", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns8", + "tied_pairs": 0, + "winner_match": true + }, + "tpot_ms:p90": { + "concordant_pairs": 64, + "discordant_pairs": 2, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.9696969696969697, + "real_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns16", + "tp1_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "sim_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp4_mns32", + "tp2_mns8", + "tp4_mns64", + "tp2_mns16", + "tp2_mns32", + "tp1_mns8", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns8", + "tied_pairs": 0, + "winner_match": true + }, + "ttft_ms:mean": { + "concordant_pairs": 56, + "discordant_pairs": 10, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.8484848484848485, + "real_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp1_mns32", + "tp2_mns32", + "tp4_mns32", + "tp1_mns16", + "tp2_mns16", + "tp4_mns16", + "tp1_mns8", + "tp2_mns8", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns32", + "tp4_mns16", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + }, + "ttft_ms:p90": { + "concordant_pairs": 56, + "discordant_pairs": 10, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.8484848484848485, + "real_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns32", + "tp1_mns64", + "tp2_mns32", + "tp4_mns32", + "tp1_mns16", + "tp4_mns16", + "tp2_mns16", + "tp1_mns8", + "tp2_mns8", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns32", + "tp4_mns16", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + } + }, + "sim_root": "/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2" +} diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/comparison.md b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/comparison.md new file mode 100644 index 0000000..83f66a7 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/comparison.md @@ -0,0 +1,29 @@ +# Frontier piecewise graph-profile vs. real vLLM + +Each cell uses its TP-normalized, 129-request trace. Frontier values are one deterministic simulation; real values pool three fresh-server trials (387 requests/cell). + +| Config | TTFT sim / real mean (ms) | TPOT sim / real mean (ms) | E2E sim / real mean (ms) | +|---|---:|---:|---:| +| tp1_mns8 | 1669003.8 / 88479.9 | 76.0 / 14.3 | 1930772.9 / 137996.4 | +| tp1_mns16 | 870562.2 / 17378.2 | 99.1 / 20.6 | 1209178.7 / 87355.2 | +| tp1_mns32 | 316980.6 / 620.5 | 119.6 / 23.8 | 722298.9 / 81051.2 | +| tp1_mns64 | 105003.3 / 619.8 | 137.8 / 24.1 | 549486.0 / 82253.7 | +| tp2_mns8 | 1171654.6 / 95920.1 | 51.1 / 9.2 | 1348614.2 / 128296.9 | +| tp2_mns16 | 606885.2 / 34443.0 | 64.4 / 14.1 | 828082.5 / 82936.7 | +| tp2_mns32 | 224495.4 / 1050.6 | 74.8 / 18.0 | 479340.1 / 62267.5 | +| tp2_mns64 | 30524.5 / 375.8 | 82.6 / 18.2 | 311160.1 / 62353.6 | +| tp4_mns8 | 834500.2 / 102605.9 | 35.4 / 6.8 | 957838.5 / 126405.4 | +| tp4_mns16 | 429098.7 / 35042.6 | 43.2 / 8.8 | 578209.7 / 65779.1 | +| tp4_mns32 | 166734.5 / 6296.2 | 49.4 / 12.2 | 336309.9 / 47944.5 | +| tp4_mns64 | 26688.5 / 246.0 | 53.7 / 13.2 | 209788.3 / 44985.1 | + +## Selection agreement + +| Target | Frontier winner | Real winner | Match | Pairwise agreement | +|---|---|---|---:|---:| +| ttft_ms:mean | tp4_mns64 | tp4_mns64 | yes | 84.8% (56/66) | +| ttft_ms:p90 | tp4_mns64 | tp4_mns64 | yes | 84.8% (56/66) | +| tpot_ms:mean | tp4_mns8 | tp4_mns8 | yes | 97.0% (64/66) | +| tpot_ms:p90 | tp4_mns8 | tp4_mns8 | yes | 97.0% (64/66) | +| e2e_ms:mean | tp4_mns64 | tp4_mns64 | yes | 92.4% (61/66) | +| e2e_ms:p90 | tp4_mns64 | tp4_mns64 | yes | 83.3% (55/66) | diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/launch.log b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/launch.log new file mode 100644 index 0000000..4b641d8 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/launch.log @@ -0,0 +1,2 @@ +ANALYSIS_LAUNCH_ECHO sim_root=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2 real_audit=/home/admin/cpfs/wjh/aituner/simulator-tuning-latency-q30-tp-normalized-u0p01-20260717/qwen30-tracepd-u0p01-real-audit.json outputs=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis expected=12 completed cells, rank+latency comparison +ANALYSIS_RETRY_ECHO sim_root=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2 real_audit_sha256=6e2c17bce5b0edb5f6f92aae4cc1b00f9390f692d31d059efe647158b386c8e9 output=/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis diff --git a/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/summary.json b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/summary.json new file mode 100644 index 0000000..7ffaba6 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/frozen-inputs/q30-trace-pd/summary.json @@ -0,0 +1,230 @@ +{ + "e2e_ms:mean": { + "concordant_pairs": 61, + "discordant_pairs": 5, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.9242424242424242, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp2_mns32", + "tp2_mns64", + "tp4_mns16", + "tp1_mns32", + "tp1_mns64", + "tp2_mns16", + "tp1_mns16", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns64", + "tp4_mns16", + "tp1_mns32", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + }, + "e2e_ms:p90": { + "concordant_pairs": 55, + "discordant_pairs": 11, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.8333333333333334, + "real_ranking": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns16", + "tp2_mns32", + "tp2_mns64", + "tp2_mns16", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64", + "tp4_mns8", + "tp2_mns8", + "tp1_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns64", + "tp4_mns16", + "tp1_mns32", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + }, + "tpot_ms:mean": { + "concordant_pairs": 64, + "discordant_pairs": 2, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.9696969696969697, + "real_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns16", + "tp1_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "sim_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp4_mns32", + "tp2_mns8", + "tp4_mns64", + "tp2_mns16", + "tp2_mns32", + "tp1_mns8", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns8", + "tied_pairs": 0, + "winner_match": true + }, + "tpot_ms:p90": { + "concordant_pairs": 64, + "discordant_pairs": 2, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.9696969696969697, + "real_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8", + "tp4_mns32", + "tp4_mns64", + "tp2_mns16", + "tp1_mns8", + "tp2_mns32", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "real_winner": "tp4_mns8", + "sim_ranking": [ + "tp4_mns8", + "tp4_mns16", + "tp4_mns32", + "tp2_mns8", + "tp4_mns64", + "tp2_mns16", + "tp2_mns32", + "tp1_mns8", + "tp2_mns64", + "tp1_mns16", + "tp1_mns32", + "tp1_mns64" + ], + "sim_winner": "tp4_mns8", + "tied_pairs": 0, + "winner_match": true + }, + "ttft_ms:mean": { + "concordant_pairs": 56, + "discordant_pairs": 10, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.8484848484848485, + "real_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp1_mns32", + "tp2_mns32", + "tp4_mns32", + "tp1_mns16", + "tp2_mns16", + "tp4_mns16", + "tp1_mns8", + "tp2_mns8", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns32", + "tp4_mns16", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + }, + "ttft_ms:p90": { + "concordant_pairs": 56, + "discordant_pairs": 10, + "informative_pairs": 66, + "pairwise_agreement_fraction": 0.8484848484848485, + "real_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns32", + "tp1_mns64", + "tp2_mns32", + "tp4_mns32", + "tp1_mns16", + "tp4_mns16", + "tp2_mns16", + "tp1_mns8", + "tp2_mns8", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "sim_ranking": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64", + "tp4_mns32", + "tp2_mns32", + "tp1_mns32", + "tp4_mns16", + "tp2_mns16", + "tp4_mns8", + "tp1_mns16", + "tp2_mns8", + "tp1_mns8" + ], + "sim_winner": "tp4_mns64", + "tied_pairs": 0, + "winner_match": true + } +} diff --git a/runs/frontier-split-rootcause-v0/results/decision-pair-axis.png b/runs/frontier-split-rootcause-v0/results/decision-pair-axis.png new file mode 100644 index 0000000..6d0bc0b Binary files /dev/null and b/runs/frontier-split-rootcause-v0/results/decision-pair-axis.png differ diff --git a/runs/frontier-split-rootcause-v0/results/decomposition.json b/runs/frontier-split-rootcause-v0/results/decomposition.json new file mode 100644 index 0000000..d96b226 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/results/decomposition.json @@ -0,0 +1,7410 @@ +{ + "anomalies": [], + "cross_checks": [ + { + "expected_percent": 58.0, + "name": "q30 high-pressure fixed-pd TTFT mean regret", + "observed_percent": 57.97674051508896, + "passed": true + }, + { + "expected_percent": 33.0, + "name": "q235 A0 fixed-pd TPOT mean regret", + "observed_percent": 33.025510615656906, + "passed": true + }, + { + "expected_percent": 33.0, + "name": "q235 A1 fixed-pd TPOT mean regret", + "observed_percent": 33.025510615656906, + "passed": true + }, + { + "expected_percent": [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0 + ], + "name": "q30 trace-pd all-objective regret", + "observed_percent": [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0 + ], + "passed": true + }, + { + "expected_percent": [ + 21.2, + 0.3 + ], + "name": "q235 Trace-PO TTFT p90 A0 to A1 regret", + "observed_percent": [ + 21.17895828808376, + 0.2607797240047338 + ], + "passed": true + }, + { + "expected_percent": [ + 33.0, + 37.2, + 30.7, + 34.6, + 33.0, + 37.2, + 30.7, + 34.6 + ], + "name": "q235 Fixed-PD TPOT/E2E regrets unchanged from A0 to A1", + "observed_percent": [ + 33.025510615656906, + 37.228890038064755, + 30.67524317507717, + 34.56375801249614, + 33.025510615656906, + 37.228890038064755, + 30.67524317507717, + 34.56375801249614 + ], + "passed": true + } + ], + "data_gaps": [], + "definitions": { + "config_uniform_scale": "geomean_c(sim_c / real_c) over paired configs", + "failure": "regret > 0.05", + "fragile_success": "winner match and real best-vs-second relative margin < 0.01", + "h_scale_verdict": "HOLDS iff failure equals (top-3 log-ratio spread > log1p(real relative margin))", + "log_ratio_spread_all": "max_c(log(sim_c / real_c)) - min_c(log(sim_c / real_c))", + "log_ratio_spread_real_top3": "log-ratio spread over the three configs with lowest real objective values", + "objective_direction": "all objectives are minimized", + "pair_inversion": "real_c > real_w and sim_c < sim_w, equivalently delta_c < -m_c", + "real_pair_margin_percent": "100 * log(real_c / real_w)", + "real_relative_margin_best_vs_second": "real_second / real_best - 1", + "regret": "real_sim_winner / real_best - 1, recomputed from per-config values", + "signed_pairwise_delta_percent": "100 * (log(sim_c / sim_w) - log(real_c / real_w)); negative favors c relative to real", + "winner_deciding_axis": "axis difference between real winner w and sim winner s; none for a winner match", + "winner_pair_tie": "sim_s == sim_w with real_s > real_w; a decision-boundary winner-label mismatch, not a strict inversion" + }, + "directional_summary": { + "axis_inversions_by_case": [ + { + "case": "q30 trace-pd", + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q30 fixed-pd high", + "n_inv_mixed": 6, + "n_inv_mns": 3, + "n_inv_tp": 7, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q30 fixed-po high", + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 4, + "n_tie_tp": 0 + }, + { + "case": "q30 fixed-pd low", + "n_inv_mixed": 24, + "n_inv_mns": 0, + "n_inv_tp": 8, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q30 fixed-po low", + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 4, + "n_tie_tp": 0 + }, + { + "case": "q30 trace-po low", + "n_inv_mixed": 0, + "n_inv_mns": 6, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 2, + "n_tie_tp": 0 + }, + { + "case": "q235 A0 fixed-pd", + "n_inv_mixed": 5, + "n_inv_mns": 0, + "n_inv_tp": 5, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0 + }, + { + "case": "q235 A0 fixed-po", + "n_inv_mixed": 4, + "n_inv_mns": 0, + "n_inv_tp": 4, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q235 A0 trace-pd", + "n_inv_mixed": 0, + "n_inv_mns": 2, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q235 A0 trace-po", + "n_inv_mixed": 4, + "n_inv_mns": 2, + "n_inv_tp": 4, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q235 A1 fixed-pd", + "n_inv_mixed": 4, + "n_inv_mns": 0, + "n_inv_tp": 4, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0 + }, + { + "case": "q235 A1 fixed-po", + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q235 A1 trace-pd", + "n_inv_mixed": 0, + "n_inv_mns": 2, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0 + }, + { + "case": "q235 A1 trace-po", + "n_inv_mixed": 0, + "n_inv_mns": 2, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 2, + "n_tie_tp": 0 + } + ], + "axis_inversions_total": { + "mixed": 47, + "mns-axis": 17, + "tp-axis": 32 + }, + "axis_ties_total": { + "mixed": 0, + "mns-axis": 14, + "tp-axis": 0 + }, + "boundary_winner_ties": [ + { + "case": "q30 fixed-po high", + "objective": "ttft_mean_ms", + "real_winner": "tp4_mns32", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po high", + "objective": "ttft_p90_ms", + "real_winner": "tp4_mns32", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po high", + "objective": "e2e_mean_ms", + "real_winner": "tp4_mns32", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po high", + "objective": "e2e_p90_ms", + "real_winner": "tp4_mns32", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po low", + "objective": "ttft_mean_ms", + "real_winner": "tp4_mns8", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po low", + "objective": "ttft_p90_ms", + "real_winner": "tp4_mns8", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po low", + "objective": "e2e_mean_ms", + "real_winner": "tp4_mns8", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 fixed-po low", + "objective": "e2e_p90_ms", + "real_winner": "tp4_mns8", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 trace-po low", + "objective": "ttft_p90_ms", + "real_winner": "tp4_mns8", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q30 trace-po low", + "objective": "e2e_p90_ms", + "real_winner": "tp4_mns8", + "sim_winner": "tp4_mns16", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q235 A0 fixed-pd", + "objective": "ttft_p90_ms", + "real_winner": "tp8_ep8_mns64", + "sim_winner": "tp8_ep8_mns128", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q235 A1 fixed-pd", + "objective": "ttft_p90_ms", + "real_winner": "tp8_ep8_mns64", + "sim_winner": "tp8_ep8_mns128", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q235 A1 trace-po", + "objective": "ttft_p90_ms", + "real_winner": "tp8_ep8_mns64", + "sim_winner": "tp8_ep8_mns128", + "winner_deciding_axis": "mns-axis" + }, + { + "case": "q235 A1 trace-po", + "objective": "e2e_p90_ms", + "real_winner": "tp8_ep8_mns64", + "sim_winner": "tp8_ep8_mns128", + "winner_deciding_axis": "mns-axis" + } + ], + "fragile_success_count": 6, + "fragile_successes": [ + { + "case": "q30 trace-pd", + "objective": "e2e_p90_ms", + "real_relative_margin_percent": 0.07414165586019372 + }, + { + "case": "q235 A1 fixed-pd", + "objective": "ttft_mean_ms", + "real_relative_margin_percent": 0.37425080108803854 + }, + { + "case": "q235 A1 fixed-po", + "objective": "ttft_mean_ms", + "real_relative_margin_percent": 0.20104594953702648 + }, + { + "case": "q235 A1 fixed-po", + "objective": "ttft_p90_ms", + "real_relative_margin_percent": 0.26166420965441084 + }, + { + "case": "q235 A1 fixed-po", + "objective": "e2e_mean_ms", + "real_relative_margin_percent": 0.20087777023818365 + }, + { + "case": "q235 A1 fixed-po", + "objective": "e2e_p90_ms", + "real_relative_margin_percent": 0.26228330848965253 + } + ], + "margin_robust_success_count": 17, + "material_failures": [ + { + "case": "q30 fixed-pd high", + "objective": "ttft_mean_ms", + "regret": 0.5797674051508896, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -523.7691675731886, + "winner_pair_m_s_percent": 45.7277624272319 + }, + { + "case": "q30 fixed-pd high", + "objective": "ttft_p90_ms", + "regret": 0.5620663582204115, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -435.31434147282096, + "winner_pair_m_s_percent": 44.600953337070045 + }, + { + "case": "q30 fixed-pd high", + "objective": "e2e_p90_ms", + "regret": 0.05493398001893279, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -101.70914835555358, + "winner_pair_m_s_percent": 5.3478186788827395 + }, + { + "case": "q30 fixed-pd low", + "objective": "tpot_mean_ms", + "regret": 0.13962124906187534, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -52.50428401638987, + "winner_pair_m_s_percent": 13.069596953822204 + }, + { + "case": "q30 fixed-pd low", + "objective": "tpot_p90_ms", + "regret": 0.1335710369665437, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -52.8208410178207, + "winner_pair_m_s_percent": 12.537285946185337 + }, + { + "case": "q30 fixed-pd low", + "objective": "e2e_mean_ms", + "regret": 0.2692193946565338, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -59.60969731665489, + "winner_pair_m_s_percent": 23.840206161907986 + }, + { + "case": "q30 fixed-pd low", + "objective": "e2e_p90_ms", + "regret": 0.2573048300400054, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -59.778209091748444, + "winner_pair_m_s_percent": 22.897040620669692 + }, + { + "case": "q235 A0 fixed-pd", + "objective": "tpot_mean_ms", + "regret": 0.33025510615656906, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -63.96663007689711, + "winner_pair_m_s_percent": 28.537073298091865 + }, + { + "case": "q235 A0 fixed-pd", + "objective": "tpot_p90_ms", + "regret": 0.37228890038064755, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -62.84476021464781, + "winner_pair_m_s_percent": 31.648007592775677 + }, + { + "case": "q235 A0 fixed-pd", + "objective": "e2e_mean_ms", + "regret": 0.3067524317507717, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -61.50853261890787, + "winner_pair_m_s_percent": 26.754499952423572 + }, + { + "case": "q235 A0 fixed-pd", + "objective": "e2e_p90_ms", + "regret": 0.3456375801249614, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -60.36185233190644, + "winner_pair_m_s_percent": 29.686793800126072 + }, + { + "case": "q235 A0 fixed-po", + "objective": "ttft_mean_ms", + "regret": 0.058746722254088946, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -12.020280158083734, + "winner_pair_m_s_percent": 5.708587111542836 + }, + { + "case": "q235 A0 fixed-po", + "objective": "e2e_mean_ms", + "regret": 0.05873813644442971, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -12.019469213946493, + "winner_pair_m_s_percent": 5.707776167405595 + }, + { + "case": "q235 A0 trace-pd", + "objective": "e2e_p90_ms", + "regret": 0.062164651568153984, + "winner_deciding_axis": "mns-axis", + "winner_pair_delta_s_percent": -16.715978521917112, + "winner_pair_m_s_percent": 6.030894994380943 + }, + { + "case": "q235 A0 trace-po", + "objective": "ttft_mean_ms", + "regret": 0.07039370625209918, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -36.29522004071061, + "winner_pair_m_s_percent": 6.802653056521642 + }, + { + "case": "q235 A0 trace-po", + "objective": "ttft_p90_ms", + "regret": 0.2117895828808376, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -31.606834004650835, + "winner_pair_m_s_percent": 19.209826109044702 + }, + { + "case": "q235 A0 trace-po", + "objective": "e2e_mean_ms", + "regret": 0.07037121566091509, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -36.29311886734452, + "winner_pair_m_s_percent": 6.800551883155552 + }, + { + "case": "q235 A0 trace-po", + "objective": "e2e_p90_ms", + "regret": 0.21167227304868486, + "winner_deciding_axis": "mixed", + "winner_pair_delta_s_percent": -31.59715282630199, + "winner_pair_m_s_percent": 19.20014493069586 + }, + { + "case": "q235 A1 fixed-pd", + "objective": "tpot_mean_ms", + "regret": 0.33025510615656906, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -60.056263771346096, + "winner_pair_m_s_percent": 28.537073298091865 + }, + { + "case": "q235 A1 fixed-pd", + "objective": "tpot_p90_ms", + "regret": 0.37228890038064755, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -61.12091996120764, + "winner_pair_m_s_percent": 31.648007592775677 + }, + { + "case": "q235 A1 fixed-pd", + "objective": "e2e_mean_ms", + "regret": 0.3067524317507717, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -58.25805162281401, + "winner_pair_m_s_percent": 26.754499952423572 + }, + { + "case": "q235 A1 fixed-pd", + "objective": "e2e_p90_ms", + "regret": 0.3456375801249614, + "winner_deciding_axis": "tp-axis", + "winner_pair_delta_s_percent": -59.1984798883988, + "winner_pair_m_s_percent": 29.686793800126072 + }, + { + "case": "q235 A1 trace-pd", + "objective": "e2e_p90_ms", + "regret": 0.062164651568153984, + "winner_deciding_axis": "mns-axis", + "winner_pair_delta_s_percent": -17.372678635438426, + "winner_pair_m_s_percent": 6.030894994380943 + } + ], + "sim_tie_winner_mismatch_count": 14, + "strict_winner_reversal_count": 33 + }, + "known_attribution": { + "q235_state": { + "components": [ + { + "component": "total", + "exact_state_tp8_minus_tp4_ms": 10.902788634037428, + "own_composition_tp8_minus_tp4_ms": -20.06625468177714 + }, + { + "component": "moe_compute", + "exact_state_tp8_minus_tp4_ms": -1.9774463255033616, + "own_composition_tp8_minus_tp4_ms": -32.45444243169211 + }, + { + "component": "attention", + "exact_state_tp8_minus_tp4_ms": -2.130420408899373, + "own_composition_tp8_minus_tp4_ms": -7.461942625219683 + }, + { + "component": "tp_dp_communication", + "exact_state_tp8_minus_tp4_ms": -6.471757614086094, + "own_composition_tp8_minus_tp4_ms": -6.571107062807271 + }, + { + "component": "ep_communication", + "exact_state_tp8_minus_tp4_ms": 6.535508227248577, + "own_composition_tp8_minus_tp4_ms": 6.235402017662346 + }, + { + "component": "moe_routing", + "exact_state_tp8_minus_tp4_ms": 0.13840596821076812, + "own_composition_tp8_minus_tp4_ms": -0.2861742505642797 + }, + { + "component": "dense_mlp_compute", + "exact_state_tp8_minus_tp4_ms": -0.030752152939768296, + "own_composition_tp8_minus_tp4_ms": -0.12314137296116046 + }, + { + "component": "pipeline_communication", + "exact_state_tp8_minus_tp4_ms": 0.0, + "own_composition_tp8_minus_tp4_ms": 0.0 + }, + { + "component": "runtime_overhead", + "exact_state_tp8_minus_tp4_ms": 0.0, + "own_composition_tp8_minus_tp4_ms": 0.0 + } + ], + "configs": [ + { + "config": "tp4_ep1_mns64", + "exact_state_coverage": 1.0, + "exact_state_matched_sim_ms": 34.25746674314369, + "real_exact_decode_batch_mean": 3.8896771904930825, + "real_token_weighted_iteration_ms": 18.47097461620307, + "sim_own_decode_batch_mean": 13.537037037037036 + }, + { + "config": "tp8_ep8_mns64", + "exact_state_coverage": 1.0, + "exact_state_matched_sim_ms": 45.160255377181116, + "real_exact_decode_batch_mean": 10.921314741035857, + "real_token_weighted_iteration_ms": 27.588699194406445, + "sim_own_decode_batch_mean": 18.93782383419689 + } + ], + "contrast_decomposition": { + "exact_state_matched_tp8_minus_tp4_ms": 10.902788634037428, + "residual_vs_exact_iteration_ms": 1.7850640558340523, + "residual_vs_frozen_real_tpot_ms": 3.9527886340374287, + "simulator_internal_tp8_minus_tp4_ms": -20.06625468177714, + "state_composition_shift_ms": 30.969043315814567 + }, + "reference": { + "exact_iteration_tp8_minus_tp4_ms": 9.117724578203376, + "observed_real_tp8_minus_tp4_ms": 6.949999999999999, + "real_tpot_ms": { + "tp4_ep1_mns64": 21.0442, + "tp8_ep8_mns64": 27.9942 + } + }, + "source": "q235-state-diag/state-diagnosis-exact.json", + "verdict": "Exact real state composition flips Frontier to the correct TP4 winner; the ranking error is caused by closed-loop state/composition mismatch, not a conditional stage reversal." + }, + "q30_admission": { + "cells": [ + { + "config": "tp2_mns64", + "mns": 64, + "real_required_slots_upper_bound": 7.548483722821714, + "real_ttft_ms": 194.56193759030603, + "real_waiting_max": 0, + "sim_first_scheduling_wait_ms": 38.22329108338011, + "sim_queue_free_ttft_ms": 191.02937373774816, + "sim_required_slots": 57.96430802744742, + "sim_tpot_over_real": 8.091316956395687, + "sim_ttft_ms": 229.2526648211283 + }, + { + "config": "tp4_mns16", + "mns": 16, + "real_required_slots_upper_bound": 14.637597063342735, + "real_ttft_ms": 124.49453797538061, + "real_waiting_max": 0, + "sim_first_scheduling_wait_ms": 64906.25631809643, + "sim_queue_free_ttft_ms": 134.74066995065758, + "sim_required_slots": 54.488824675063235, + "sim_tpot_over_real": 3.8276349505907024, + "sim_ttft_ms": 65040.99698804708 + }, + { + "config": "tp4_mns32", + "mns": 32, + "real_required_slots_upper_bound": 14.521569587596273, + "real_ttft_ms": 123.15859724408143, + "real_waiting_max": 0, + "sim_first_scheduling_wait_ms": 27180.676177927482, + "sim_queue_free_ttft_ms": 135.62147333612225, + "sim_required_slots": 65.89386162288272, + "sim_tpot_over_real": 4.6740581573385525, + "sim_ttft_ms": 27316.2976512636 + }, + { + "config": "tp4_mns64", + "mns": 64, + "real_required_slots_upper_bound": 14.661911968825338, + "real_ttft_ms": 123.17715882596677, + "real_waiting_max": 0, + "sim_first_scheduling_wait_ms": 6367.273779218055, + "sim_queue_free_ttft_ms": 136.7304324624105, + "sim_required_slots": 79.81630813400574, + "sim_tpot_over_real": 5.614117914776633, + "sim_ttft_ms": 6504.004211680465 + } + ], + "contrasts": { + "observed_sim_ttft_tp4_minus_tp2_ms": 27087.044986442474, + "queue_free_sim_ttft_tp4_minus_tp2_ms": -55.40790040162591, + "real_ttft_tp4_minus_tp2_ms": -71.4033403462246 + }, + "source": "q30-admission-diag/result.json", + "verdict": "Frontier overpredicts decode service time, so TP4's higher global arrival rate crosses the MNS admission cap only in simulation. The resulting first-scheduling queue reverses the TP topology ranking; without that queue Frontier itself predicts TP4 faster than TP2, matching real hardware." + } + }, + "rows": [ + { + "case": "q30 trace-pd", + "config_uniform_scale": 45.899648513700804, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 4.140203056697399, + "log_ratio_spread_real_top3": 0.7350071901442927, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 425.7830890012937, + "signed_pairwise_delta_percent": -77.29228415027549 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 92.53155134784632, + "signed_pairwise_delta_percent": 154.92939260781299 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 92.42510316771971, + "signed_pairwise_delta_percent": 44.55096600091358 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 588.5391626639711, + "signed_pairwise_delta_percent": -174.96406393770422 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 494.19206953478573, + "signed_pairwise_delta_percent": -181.7812291449898 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 145.20093058332228, + "signed_pairwise_delta_percent": 67.76147877390028 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 42.379688777532195, + "signed_pairwise_delta_percent": -28.94975301351572 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 596.613115965754, + "signed_pairwise_delta_percent": -218.41901367549661 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 495.91805198396617, + "signed_pairwise_delta_percent": -218.17248485782983 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 324.25665835499575, + "signed_pairwise_delta_percent": -141.03952629372947 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 603.3510930798824, + "signed_pairwise_delta_percent": -259.0909130619269 + } + ], + "real_relative_margin_best_vs_second": 0.5277512573119312, + "real_top3_configs": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns64" + ], + "real_winner": "tp4_mns64", + "regret": 0.0, + "sim_winner": "tp4_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-trace-pd/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-pd", + "config_uniform_scale": 43.38413908996929, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 3.8627205241706646, + "log_ratio_spread_real_top3": 1.3676487927064054, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 413.27263799256224, + "signed_pairwise_delta_percent": -133.33363440024485 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 103.58793636206667, + "signed_pairwise_delta_percent": 92.65315392171725 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 104.88701154249979, + "signed_pairwise_delta_percent": 0.5820114595004977 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 550.2663402226464, + "signed_pairwise_delta_percent": -210.29242844968587 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 473.3308528278175, + "signed_pairwise_delta_percent": -229.66830605288865 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 130.25554778064452, + "signed_pairwise_delta_percent": 29.42287867963369 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 51.61341304162133, + "signed_pairwise_delta_percent": -44.11172534892322 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 559.1872645568208, + "signed_pairwise_delta_percent": -254.7846496300535 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 468.54545769902376, + "signed_pairwise_delta_percent": -259.82819430980805 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 333.04135988053577, + "signed_pairwise_delta_percent": -204.24777470078647 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 564.214798802028, + "signed_pairwise_delta_percent": -293.6188984953492 + } + ], + "real_relative_margin_best_vs_second": 0.6755377028954586, + "real_top3_configs": [ + "tp4_mns64", + "tp2_mns64", + "tp1_mns32" + ], + "real_winner": "tp4_mns64", + "regret": 0.0, + "sim_winner": "tp4_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-trace-pd/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-pd", + "config_uniform_scale": 4.798699439392796, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.34234204465206575, + "log_ratio_spread_real_top3": 0.11876167427448592, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 111.39211804152, + "signed_pairwise_delta_percent": -8.483533629441165 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 125.72442888367388, + "signed_pairwise_delta_percent": -4.073078922372719 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 127.21791091204592, + "signed_pairwise_delta_percent": 8.632369074762082 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 74.84126630438477, + "signed_pairwise_delta_percent": 1.447194103380467 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 73.532537849401, + "signed_pairwise_delta_percent": -13.742569389219161 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 97.95939493225517, + "signed_pairwise_delta_percent": -23.240697633755538 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 99.29270089893359, + "signed_pairwise_delta_percent": -14.63432527265468 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 31.337675062299986, + "signed_pairwise_delta_percent": 5.216266068044573 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 26.49069271125039, + "signed_pairwise_delta_percent": -6.6599013594039915 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 58.92488843099623, + "signed_pairwise_delta_percent": -25.60183539044448 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 66.79126824994552, + "signed_pairwise_delta_percent": -25.112476758256985 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.3033096673391522, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8" + ], + "real_winner": "tp4_mns8", + "regret": 0.0, + "sim_winner": "tp4_mns8", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-trace-pd/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-pd", + "config_uniform_scale": 4.291364819104713, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.349119999217536, + "log_ratio_spread_real_top3": 0.10928760145539185, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 125.15305260666022, + "signed_pairwise_delta_percent": -19.96596661376333 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 142.11708890779718, + "signed_pairwise_delta_percent": -17.29368054409164 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 143.30272831211315, + "signed_pairwise_delta_percent": -0.4539882281926477 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 81.3187358452952, + "signed_pairwise_delta_percent": -3.482599377639306 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 80.19578793426079, + "signed_pairwise_delta_percent": -18.566404630125845 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 107.78616263504968, + "signed_pairwise_delta_percent": -30.705392091898254 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 109.09052427237363, + "signed_pairwise_delta_percent": -19.929771139892004 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 34.15060711443027, + "signed_pairwise_delta_percent": 3.7395670641060086 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 28.455433619208037, + "signed_pairwise_delta_percent": -7.189193081433193 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 64.11332289157903, + "signed_pairwise_delta_percent": -28.87510274879303 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 75.37357913831967, + "signed_pairwise_delta_percent": -31.172432857647596 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.32916953334893906, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns16", + "tp2_mns8" + ], + "real_winner": "tp4_mns8", + "regret": 0.0, + "sim_winner": "tp4_mns8", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-trace-pd/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-pd", + "config_uniform_scale": 8.262858317506733, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 1.0986813103518063, + "log_ratio_spread_real_top3": 0.5012037189370788, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 66.36499381882803, + "signed_pairwise_delta_percent": 108.79476716386034 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 58.8748864721832, + "signed_pairwise_delta_percent": 64.75911332399981 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 60.347657868883765, + "signed_pairwise_delta_percent": 35.940764769548515 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 112.0895297823733, + "signed_pairwise_delta_percent": 109.86813103518062 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 61.17455796538679, + "signed_pairwise_delta_percent": 76.12681299620277 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 32.51075059215534, + "signed_pairwise_delta_percent": 50.12037189370784 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 32.64897177653879, + "signed_pairwise_delta_percent": 6.771864363272045 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 104.80152491744161, + "signed_pairwise_delta_percent": 81.27184764853052 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 37.99697371409521, + "signed_pairwise_delta_percent": 63.386785575481056 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 6.3712519410935915, + "signed_pairwise_delta_percent": 40.82215218358013 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 103.31621796003591, + "signed_pairwise_delta_percent": 48.54179170585697 + } + ], + "real_relative_margin_best_vs_second": 0.06578596194014197, + "real_top3_configs": [ + "tp4_mns64", + "tp4_mns32", + "tp2_mns32" + ], + "real_winner": "tp4_mns64", + "regret": 0.0, + "sim_winner": "tp4_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-trace-pd/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-pd", + "config_uniform_scale": 8.066172032770895, + "expected_config_count": 12, + "failure": false, + "fragile_success": true, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 1.2303978377958356, + "log_ratio_spread_real_top3": 0.7034476827520184, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 62.299232337248434, + "signed_pairwise_delta_percent": 108.56642733247624 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 66.51906134872392, + "signed_pairwise_delta_percent": 57.03741657339753 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 68.00975092247843, + "signed_pairwise_delta_percent": 28.839836578727095 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 97.52912455611215, + "signed_pairwise_delta_percent": 123.03978377958349 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 49.72701364992361, + "signed_pairwise_delta_percent": 83.73553771263259 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 33.62187114781614, + "signed_pairwise_delta_percent": 48.68626755803108 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 34.60052079875263, + "signed_pairwise_delta_percent": 8.505625400877559 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 94.65769118557269, + "signed_pairwise_delta_percent": 90.22793273806666 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 27.58788592694011, + "signed_pairwise_delta_percent": 70.3447682752018 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.07411418451216154, + "signed_pairwise_delta_percent": 45.007951726273106 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 91.4777538802895, + "signed_pairwise_delta_percent": 59.43122170260607 + } + ], + "real_relative_margin_best_vs_second": 0.0007414165586019372, + "real_top3_configs": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns16" + ], + "real_winner": "tp4_mns64", + "regret": 0.0, + "sim_winner": "tp4_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-trace-pd/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-pd high", + "config_uniform_scale": 80.27356282971952, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 6.647450003760477, + "log_ratio_spread_real_top3": 2.2919615656812473, + "missing_configs": [], + "n_inv_mixed": 2, + "n_inv_mns": 1, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 95.49977993455686, + "signed_pairwise_delta_percent": 35.24651168874489 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": true, + "real_pair_margin_percent": 95.063546989325, + "signed_pairwise_delta_percent": -123.55932905047267 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 95.64389799119368, + "signed_pairwise_delta_percent": -534.2205177871797 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 94.7705561639161, + "signed_pairwise_delta_percent": 109.67959518505891 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 49.422396437751175, + "signed_pairwise_delta_percent": 60.17959107930238 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": true, + "real_pair_margin_percent": 49.333707226264345, + "signed_pairwise_delta_percent": -52.766300168579974 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 45.7277624272319, + "signed_pairwise_delta_percent": -523.7691675731886 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 46.01935636714559, + "signed_pairwise_delta_percent": 130.52448258886798 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 1.0788909971033689, + "signed_pairwise_delta_percent": 85.67453763656644 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": true, + "real_pair_margin_percent": 0.015070148334617827, + "signed_pairwise_delta_percent": -143.52161893155832 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 414.6352679329693, + "signed_pairwise_delta_percent": -263.0584045824042 + } + ], + "real_relative_margin_best_vs_second": 0.00015071283938516977, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns16" + ], + "real_winner": "tp4_mns32", + "regret": 0.5797674051508896, + "sim_winner": "tp2_mns64", + "sim_winner_real_relative_loss": 0.5797674051508896, + "sources": [ + "q30-fixed-hi/fixed-pd/comparison.json", + "q30-fixed-hi/fixed-pd/real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -523.7691675731886, + "winner_pair_m_s_percent": 45.7277624272319, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-pd high", + "config_uniform_scale": 130.66346534057038, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 7.092751831801099, + "log_ratio_spread_real_top3": 2.2268269641383185, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 93.72928763800273, + "signed_pairwise_delta_percent": 173.8072062575841 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 93.44021170742349, + "signed_pairwise_delta_percent": 21.330865292892188 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 93.88144073535756, + "signed_pairwise_delta_percent": -444.53250480136165 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 93.24983507093198, + "signed_pairwise_delta_percent": 244.41427013677844 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 49.20897259013651, + "signed_pairwise_delta_percent": 196.79655252285477 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 49.34236949428588, + "signed_pairwise_delta_percent": 90.24518667005819 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 44.600953337070045, + "signed_pairwise_delta_percent": -435.31434147282096 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 44.846370322374355, + "signed_pairwise_delta_percent": 264.74267837874817 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.23892035198680692, + "signed_pairwise_delta_percent": 222.6826964138318 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.06833719734037036, + "signed_pairwise_delta_percent": 142.72331929981317 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 470.42622660647976, + "signed_pairwise_delta_percent": -185.91602808451228 + } + ], + "real_relative_margin_best_vs_second": 0.0006836055252286144, + "real_top3_configs": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns16" + ], + "real_winner": "tp4_mns64", + "regret": 0.5620663582204115, + "sim_winner": "tp2_mns64", + "sim_winner_real_relative_loss": 0.5620663582204115, + "sources": [ + "q30-fixed-hi/fixed-pd/comparison.json", + "q30-fixed-hi/fixed-pd/real-audit.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -435.31434147282096, + "winner_pair_m_s_percent": 44.600953337070045, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-pd high", + "config_uniform_scale": 6.313105858122109, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 1.1809570890904728, + "log_ratio_spread_real_top3": 0.9568100689791217, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 30.94382797908889, + "signed_pairwise_delta_percent": 73.27671364429281 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 29.57530277436313, + "signed_pairwise_delta_percent": 95.68100689791214 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 31.449801879589334, + "signed_pairwise_delta_percent": 102.52772025706145 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 31.054240651015142, + "signed_pairwise_delta_percent": 43.97849942435047 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 54.259465439535774, + "signed_pairwise_delta_percent": 6.807202298146797 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 56.81232707946078, + "signed_pairwise_delta_percent": 22.71082561174641 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 38.68791163842329, + "signed_pairwise_delta_percent": 59.286451340060744 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 36.9319004785833, + "signed_pairwise_delta_percent": -2.2053184202783296 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 37.66973031153608, + "signed_pairwise_delta_percent": -15.567988651985809 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 36.885145529252576, + "signed_pairwise_delta_percent": 4.4100686169577905 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 37.88435252075483, + "signed_pairwise_delta_percent": 22.735748881633384 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.34413815100360146, + "real_top3_configs": [ + "tp4_mns8", + "tp1_mns32", + "tp1_mns16" + ], + "real_winner": "tp4_mns8", + "regret": 0.0, + "sim_winner": "tp4_mns8", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-fixed-hi/fixed-pd/comparison.json", + "q30-fixed-hi/fixed-pd/real-audit.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-pd high", + "config_uniform_scale": 5.885039491382665, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 1.2232223312082269, + "log_ratio_spread_real_top3": 0.9511712499899991, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 32.76189094601604, + "signed_pairwise_delta_percent": 71.90461644842226 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 31.623627006928796, + "signed_pairwise_delta_percent": 95.11712499899991 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 33.72600206831461, + "signed_pairwise_delta_percent": 103.22068421989188 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 32.79701086217899, + "signed_pairwise_delta_percent": 42.366270249175436 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 70.38082644168276, + "signed_pairwise_delta_percent": -8.955970516126776 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 70.213016980584, + "signed_pairwise_delta_percent": 10.564547864752338 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 41.23602295916777, + "signed_pairwise_delta_percent": 60.90742373262441 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 40.39726148279639, + "signed_pairwise_delta_percent": -5.603938757169219 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 41.49621341698774, + "signed_pairwise_delta_percent": -19.101548900930826 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 40.815568423340856, + "signed_pairwise_delta_percent": 1.6982357807242388 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 41.453178330956014, + "signed_pairwise_delta_percent": 23.186909212748937 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.3719543690892244, + "real_top3_configs": [ + "tp4_mns8", + "tp1_mns32", + "tp1_mns16" + ], + "real_winner": "tp4_mns8", + "regret": 0.0, + "sim_winner": "tp4_mns8", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-fixed-hi/fixed-pd/comparison.json", + "q30-fixed-hi/fixed-pd/real-audit.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-pd high", + "config_uniform_scale": 18.86746837736289, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 2.2653893343616147, + "log_ratio_spread_real_top3": 1.1146207125801464, + "missing_configs": [], + "n_inv_mixed": 2, + "n_inv_mns": 1, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mns-axis", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 1.276181452945976, + "signed_pairwise_delta_percent": 84.93667289106678 + }, + { + "axis_vs_real_winner": "none", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 1.7465468803864743, + "signed_pairwise_delta_percent": -39.82153729011294 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 1.3041295093334115, + "signed_pairwise_delta_percent": 143.94402186459692 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 19.530214414414008, + "signed_pairwise_delta_percent": 41.253386419289285 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": true, + "real_pair_margin_percent": 21.947794000268598, + "signed_pairwise_delta_percent": -34.6922795296003 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 4.610302806179068, + "signed_pairwise_delta_percent": -78.71974507782036 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 2.974254325999045, + "signed_pairwise_delta_percent": 113.33759030038964 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 1.5202495868550074, + "signed_pairwise_delta_percent": 33.73219440118163 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_mns32", + "inversion": true, + "real_pair_margin_percent": 0.7244237698333954, + "signed_pairwise_delta_percent": -26.525398366947872 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_mns64", + "inversion": true, + "real_pair_margin_percent": 1.6862244572033922, + "signed_pairwise_delta_percent": -82.59491157156452 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 113.12888745520367, + "signed_pairwise_delta_percent": -22.515920958436265 + } + ], + "real_relative_margin_best_vs_second": 0.007270540664867875, + "real_top3_configs": [ + "tp1_mns32", + "tp4_mns32", + "tp1_mns16" + ], + "real_winner": "tp1_mns32", + "regret": 0.017005214686575565, + "sim_winner": "tp4_mns64", + "sim_winner_real_relative_loss": 0.017005214686575565, + "sources": [ + "q30-fixed-hi/fixed-pd/comparison.json", + "q30-fixed-hi/fixed-pd/real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -82.59491157156452, + "winner_pair_m_s_percent": 1.6862244572033922, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-pd high", + "config_uniform_scale": 24.706368987570023, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 2.74200205248491, + "log_ratio_spread_real_top3": 1.7249105689293742, + "missing_configs": [], + "n_inv_mixed": 2, + "n_inv_mns": 1, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mns-axis", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 1.058967777283412, + "signed_pairwise_delta_percent": 110.6461541912706 + }, + { + "axis_vs_real_winner": "none", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 1.925759576793605, + "signed_pairwise_delta_percent": -63.428490447993504 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 1.0044647579152355, + "signed_pairwise_delta_percent": 172.49105689293737 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 33.27033924347824, + "signed_pairwise_delta_percent": 54.397757467940764 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 33.10740332058363, + "signed_pairwise_delta_percent": -32.26152641791503 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 5.3478186788827395, + "signed_pairwise_delta_percent": -101.70914835555358 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 4.530104156426464, + "signed_pairwise_delta_percent": 140.29978254043905 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 3.5336243843482795, + "signed_pairwise_delta_percent": 59.4958948992397 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_mns32", + "inversion": true, + "real_pair_margin_percent": 2.849041083001832, + "signed_pairwise_delta_percent": -8.454232788907941 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_mns64", + "inversion": true, + "real_pair_margin_percent": 3.5041851262986916, + "signed_pairwise_delta_percent": -92.1011626014233 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 156.6309394860069, + "signed_pairwise_delta_percent": -37.29992401503031 + } + ], + "real_relative_margin_best_vs_second": 0.010095264385685976, + "real_top3_configs": [ + "tp1_mns32", + "tp1_mns8", + "tp1_mns16" + ], + "real_winner": "tp1_mns32", + "regret": 0.05493398001893279, + "sim_winner": "tp2_mns64", + "sim_winner_real_relative_loss": 0.05493398001893279, + "sources": [ + "q30-fixed-hi/fixed-pd/comparison.json", + "q30-fixed-hi/fixed-pd/real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -101.70914835555358, + "winner_pair_m_s_percent": 5.3478186788827395, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-po high", + "config_uniform_scale": 0.964228039140724, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.2748488109291796, + "log_ratio_spread_real_top3": 0.0031834782899782255, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 101.05074090634112, + "signed_pairwise_delta_percent": -27.19082062892747 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 101.04075067896244, + "signed_pairwise_delta_percent": -27.180830401548782 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 101.23941070924307, + "signed_pairwise_delta_percent": -27.379490431829424 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 101.34480137033162, + "signed_pairwise_delta_percent": -27.484881092917966 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 51.753490275609735, + "signed_pairwise_delta_percent": -20.164334456356727 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 51.69449732837048, + "signed_pairwise_delta_percent": -20.10534150911747 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 50.79372465396658, + "signed_pairwise_delta_percent": -19.204568834713566 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 49.239894564408196, + "signed_pairwise_delta_percent": -17.650738745155188 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.3457059423188482, + "signed_pairwise_delta_percent": -0.3457059423188482 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.11778366602921966, + "signed_pairwise_delta_percent": -0.11778366602921966 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.3183478289978128, + "signed_pairwise_delta_percent": -0.3183478289978128 + } + ], + "real_relative_margin_best_vs_second": 0.0011785305823068803, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns8" + ], + "real_winner": "tp4_mns32", + "regret": 0.0034630419451076033, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.0034630419451076033, + "sources": [ + "q30-fixed-hi/fixed-po/comparison.json", + "q30-fixed-hi/fixed-po/real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.3457059423188482, + "winner_pair_m_s_percent": 0.3457059423188482, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-po high", + "config_uniform_scale": 0.9564015690834595, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.26887996664870883, + "log_ratio_spread_real_top3": 0.00304657671462559, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 100.62312189015677, + "signed_pairwise_delta_percent": -26.763201612741206 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 100.27398231390275, + "signed_pairwise_delta_percent": -26.414062036487184 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 100.66288725500065, + "signed_pairwise_delta_percent": -26.802966977585097 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 100.74791694228644, + "signed_pairwise_delta_percent": -26.887996664870883 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 51.29294809851041, + "signed_pairwise_delta_percent": -19.703792279257375 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 51.1818062042697, + "signed_pairwise_delta_percent": -19.59265038501667 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 50.60491466017445, + "signed_pairwise_delta_percent": -19.01575884092142 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 48.82452174557843, + "signed_pairwise_delta_percent": -17.235365926325404 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.5252213726458267, + "signed_pairwise_delta_percent": -0.5252213726458267 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.3046576714625563, + "signed_pairwise_delta_percent": -0.3046576714625563 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0066591234348748005, + "signed_pairwise_delta_percent": -0.0066591234348748005 + } + ], + "real_relative_margin_best_vs_second": 6.659345159421015e-05, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64" + ], + "real_winner": "tp4_mns32", + "regret": 0.005266030780421316, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.005266030780421316, + "sources": [ + "q30-fixed-hi/fixed-po/comparison.json", + "q30-fixed-hi/fixed-po/real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.5252213726458267, + "winner_pair_m_s_percent": 0.5252213726458267, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-po high", + "config_uniform_scale": 0.9639779985207279, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.2746145677630782, + "log_ratio_spread_real_top3": 0.0031816282464492696, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 101.02634481547024, + "signed_pairwise_delta_percent": -27.16642453805659 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 101.01671464824831, + "signed_pairwise_delta_percent": -27.156794370834657 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 101.21675100585334, + "signed_pairwise_delta_percent": -27.356830728439697 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 101.32137705372146, + "signed_pairwise_delta_percent": -27.461456776307813 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 51.73931792837375, + "signed_pairwise_delta_percent": -20.15016210912074 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 51.679741891211606, + "signed_pairwise_delta_percent": -20.090586071958594 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 50.77867236420423, + "signed_pairwise_delta_percent": -19.189516544951225 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 49.225955262342325, + "signed_pairwise_delta_percent": -17.636799443089313 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.3447193573234837, + "signed_pairwise_delta_percent": -0.3447193573234837 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.11768182780294648, + "signed_pairwise_delta_percent": -0.11768182780294648 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.31816282464494144, + "signed_pairwise_delta_percent": -0.31816282464494144 + } + ], + "real_relative_margin_best_vs_second": 0.0011775110003686695, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns64", + "tp4_mns8" + ], + "real_winner": "tp4_mns32", + "regret": 0.0034531419781376194, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.0034531419781376194, + "sources": [ + "q30-fixed-hi/fixed-po/comparison.json", + "q30-fixed-hi/fixed-po/real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.3447193573234837, + "winner_pair_m_s_percent": 0.3447193573234837, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-po high", + "config_uniform_scale": 0.9561532784377857, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.26866713457177865, + "log_ratio_spread_real_top3": 0.003023225018183584, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 100.59790659227713, + "signed_pairwise_delta_percent": -26.737986314861562 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 100.2500528083333, + "signed_pairwise_delta_percent": -26.390132530917743 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 100.63890275260015, + "signed_pairwise_delta_percent": -26.778982475184588 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 100.72663373459343, + "signed_pairwise_delta_percent": -26.86671345717787 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 51.27776494486162, + "signed_pairwise_delta_percent": -19.688609125608597 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 51.16674542969726, + "signed_pairwise_delta_percent": -19.577589610444228 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 50.5883392985462, + "signed_pairwise_delta_percent": -18.999183479293176 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 48.81412540227583, + "signed_pairwise_delta_percent": -17.224969583022798 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.5263180408757429, + "signed_pairwise_delta_percent": -0.5263180408757429 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.3023225018183476, + "signed_pairwise_delta_percent": -0.3023225018183476 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.005990797056166493, + "signed_pairwise_delta_percent": -0.005990797056166493 + } + ], + "real_relative_margin_best_vs_second": 5.990976507996848e-05, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64" + ], + "real_winner": "tp4_mns32", + "regret": 0.0052770552740579735, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.0052770552740579735, + "sources": [ + "q30-fixed-hi/fixed-po/comparison.json", + "q30-fixed-hi/fixed-po/real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.5263180408757429, + "winner_pair_m_s_percent": 0.5263180408757429, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-pd low", + "config_uniform_scale": 1.368098672630302, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.4377687692283213, + "log_ratio_spread_real_top3": 0.015340372114717726, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 87.54418992315985, + "signed_pairwise_delta_percent": -43.361639510708514 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 87.65226356157126, + "signed_pairwise_delta_percent": -43.46971314911994 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 87.95942733528346, + "signed_pairwise_delta_percent": -43.77687692283213 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 87.90596709065582, + "signed_pairwise_delta_percent": -43.723416678204494 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 44.441336244863145, + "signed_pairwise_delta_percent": -22.421306800693213 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 42.65776119983087, + "signed_pairwise_delta_percent": -20.63773175566094 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 41.25101894269539, + "signed_pairwise_delta_percent": -19.230989498525457 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 40.74382542815109, + "signed_pairwise_delta_percent": -18.72379598398116 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 1.5340372114717864, + "signed_pairwise_delta_percent": -1.5340372114717864 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 1.3397123950439613, + "signed_pairwise_delta_percent": -1.3397123950439613 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 2.2598323976525228, + "signed_pairwise_delta_percent": -2.2598323976525228 + } + ], + "real_relative_margin_best_vs_second": 0.013487267520547963, + "real_top3_configs": [ + "tp4_mns16", + "tp4_mns64", + "tp4_mns32" + ], + "real_winner": "tp4_mns16", + "regret": 0.0, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-expansion-lo/fixed-pd-comparison.json", + "q30-expansion-lo/fixed-pd-real-audit.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-pd low", + "config_uniform_scale": 1.4015522030046448, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.48740148963773433, + "log_ratio_spread_real_top3": 0.028938391079618264, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 83.24380892439706, + "signed_pairwise_delta_percent": -48.74014896377343 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 83.1240122706535, + "signed_pairwise_delta_percent": -48.620352310029865 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 83.23797700238447, + "signed_pairwise_delta_percent": -48.73431704176084 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 83.20129545481906, + "signed_pairwise_delta_percent": -48.69763549419543 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 41.904473317043724, + "signed_pairwise_delta_percent": -23.31237555807283 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 39.314739780466134, + "signed_pairwise_delta_percent": -20.722642021495243 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 38.33192154778196, + "signed_pairwise_delta_percent": -19.73982378881107 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 36.994448671903356, + "signed_pairwise_delta_percent": -18.402350912932466 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 3.2108167899433555, + "signed_pairwise_delta_percent": -3.2108167899433555 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 1.7450653265328064, + "signed_pairwise_delta_percent": -1.7450653265328064 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 2.8938391079618246, + "signed_pairwise_delta_percent": -2.8938391079618246 + } + ], + "real_relative_margin_best_vs_second": 0.01760380548676732, + "real_top3_configs": [ + "tp4_mns16", + "tp4_mns64", + "tp4_mns8" + ], + "real_winner": "tp4_mns16", + "regret": 0.0, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q30-expansion-lo/fixed-pd-comparison.json", + "q30-expansion-lo/fixed-pd-real-audit.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-pd low", + "config_uniform_scale": 4.951803445364717, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.5265893738689182, + "log_ratio_spread_real_top3": 0.0023698681934412047, + "missing_configs": [], + "n_inv_mixed": 6, + "n_inv_mns": 0, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": true, + "real_pair_margin_percent": 13.069596953822204, + "signed_pairwise_delta_percent": -52.50428401638987 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": true, + "real_pair_margin_percent": 13.062412988745601, + "signed_pairwise_delta_percent": -52.49710005131326 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 12.912704892995365, + "signed_pairwise_delta_percent": -52.347391955563026 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": true, + "real_pair_margin_percent": 13.224250324324174, + "signed_pairwise_delta_percent": -52.658937386891836 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": true, + "real_pair_margin_percent": 7.931511304661838, + "signed_pairwise_delta_percent": -38.89275046110161 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": true, + "real_pair_margin_percent": 9.285385999289323, + "signed_pairwise_delta_percent": -40.2466251557291 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 9.543865055930773, + "signed_pairwise_delta_percent": -40.50510421237055 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": true, + "real_pair_margin_percent": 9.455313972221232, + "signed_pairwise_delta_percent": -40.416553128661 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.7798181655007458, + "signed_pairwise_delta_percent": -0.7798181655007458 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.23698681934411808, + "signed_pairwise_delta_percent": -0.23698681934411808 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.039348303682490725, + "signed_pairwise_delta_percent": -0.039348303682490725 + } + ], + "real_relative_margin_best_vs_second": 0.0003935604614297983, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64" + ], + "real_winner": "tp4_mns32", + "regret": 0.13962124906187534, + "sim_winner": "tp1_mns16", + "sim_winner_real_relative_loss": 0.13962124906187534, + "sources": [ + "q30-expansion-lo/fixed-pd-comparison.json", + "q30-expansion-lo/fixed-pd-real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -52.50428401638987, + "winner_pair_m_s_percent": 13.069596953822204, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-pd low", + "config_uniform_scale": 4.949511590379519, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.5320917895477677, + "log_ratio_spread_real_top3": 0.0027149664654271266, + "missing_configs": [], + "n_inv_mixed": 6, + "n_inv_mns": 0, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": true, + "real_pair_margin_percent": 12.537285946185337, + "signed_pairwise_delta_percent": -52.8208410178207 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": true, + "real_pair_margin_percent": 12.710829179768771, + "signed_pairwise_delta_percent": -52.994384251404135 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 12.566252787665844, + "signed_pairwise_delta_percent": -52.8498078593012 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": true, + "real_pair_margin_percent": 12.925623883141398, + "signed_pairwise_delta_percent": -53.20917895477676 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": true, + "real_pair_margin_percent": 9.207508741892353, + "signed_pairwise_delta_percent": -40.867096818654424 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": true, + "real_pair_margin_percent": 9.12717383854323, + "signed_pairwise_delta_percent": -40.7867619153053 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 9.376587887539571, + "signed_pairwise_delta_percent": -41.03617596430164 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": true, + "real_pair_margin_percent": 9.357480041752579, + "signed_pairwise_delta_percent": -41.01706811851465 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 2.0112043752863755, + "signed_pairwise_delta_percent": -2.0112043752863755 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.2245770721418584, + "signed_pairwise_delta_percent": -0.2245770721418584 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.27149664654272265, + "signed_pairwise_delta_percent": -0.27149664654272265 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.002248294353297764, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64" + ], + "real_winner": "tp4_mns8", + "regret": 0.1335710369665437, + "sim_winner": "tp1_mns16", + "sim_winner_real_relative_loss": 0.1335710369665437, + "sources": [ + "q30-expansion-lo/fixed-pd-comparison.json", + "q30-expansion-lo/fixed-pd-real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -52.8208410178207, + "winner_pair_m_s_percent": 12.537285946185337, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-pd low", + "config_uniform_scale": 4.4225746146806655, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.5980564765480212, + "log_ratio_spread_real_top3": 0.0019192061388726422, + "missing_configs": [], + "n_inv_mixed": 6, + "n_inv_mns": 0, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": true, + "real_pair_margin_percent": 23.840206161907986, + "signed_pairwise_delta_percent": -59.60969731665489 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns32", + "inversion": true, + "real_pair_margin_percent": 23.856001111110555, + "signed_pairwise_delta_percent": -59.625492265857446 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 23.79706751418482, + "signed_pairwise_delta_percent": -59.56655866893173 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": true, + "real_pair_margin_percent": 24.036156500055224, + "signed_pairwise_delta_percent": -59.805647654802115 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": true, + "real_pair_margin_percent": 12.27084643218256, + "signed_pairwise_delta_percent": -41.25705263893593 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns32", + "inversion": true, + "real_pair_margin_percent": 13.179118203297904, + "signed_pairwise_delta_percent": -42.16532441005128 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 13.206003717586201, + "signed_pairwise_delta_percent": -42.19220992433957 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": true, + "real_pair_margin_percent": 13.059567610922084, + "signed_pairwise_delta_percent": -42.04577381767545 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.5379386855583932, + "signed_pairwise_delta_percent": -0.5379386855583932 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.19192061388725232, + "signed_pairwise_delta_percent": -0.19192061388725232 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.11294449835385692, + "signed_pairwise_delta_percent": -0.11294449835385692 + } + ], + "real_relative_margin_best_vs_second": 0.0011300830467204648, + "real_top3_configs": [ + "tp4_mns32", + "tp4_mns8", + "tp4_mns64" + ], + "real_winner": "tp4_mns32", + "regret": 0.2692193946565338, + "sim_winner": "tp1_mns16", + "sim_winner_real_relative_loss": 0.2692193946565338, + "sources": [ + "q30-expansion-lo/fixed-pd-comparison.json", + "q30-expansion-lo/fixed-pd-real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -59.60969731665489, + "winner_pair_m_s_percent": 23.840206161907986, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-pd low", + "config_uniform_scale": 4.421436305428993, + "expected_config_count": 12, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.5995516935141478, + "log_ratio_spread_real_top3": 0.0016899142191908467, + "missing_configs": [], + "n_inv_mixed": 6, + "n_inv_mns": 0, + "n_inv_tp": 2, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": true, + "real_pair_margin_percent": 22.897040620669692, + "signed_pairwise_delta_percent": -59.778209091748444 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": true, + "real_pair_margin_percent": 23.046064773358367, + "signed_pairwise_delta_percent": -59.92723324443712 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": true, + "real_pair_margin_percent": 22.841076346337463, + "signed_pairwise_delta_percent": -59.72224481741621 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": true, + "real_pair_margin_percent": 23.074000880336015, + "signed_pairwise_delta_percent": -59.95516935141476 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": true, + "real_pair_margin_percent": 12.569832422090965, + "signed_pairwise_delta_percent": -42.327734313472305 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": true, + "real_pair_margin_percent": 12.566770407129498, + "signed_pairwise_delta_percent": -42.32467229851083 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": true, + "real_pair_margin_percent": 12.592012573206555, + "signed_pairwise_delta_percent": -42.34991446458789 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": true, + "real_pair_margin_percent": 12.476154151010087, + "signed_pairwise_delta_percent": -42.23405604239143 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.7798712357940356, + "signed_pairwise_delta_percent": -0.7798712357940356 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.16899142191907698, + "signed_pairwise_delta_percent": -0.16899142191907698 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.11976959438291016, + "signed_pairwise_delta_percent": -0.11976959438291016 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.0011984134680460556, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32" + ], + "real_winner": "tp4_mns8", + "regret": 0.2573048300400054, + "sim_winner": "tp1_mns16", + "sim_winner_real_relative_loss": 0.2573048300400054, + "sources": [ + "q30-expansion-lo/fixed-pd-comparison.json", + "q30-expansion-lo/fixed-pd-real-audit.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -59.778209091748444, + "winner_pair_m_s_percent": 22.897040620669692, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 fixed-po low", + "config_uniform_scale": 1.2178864428064151, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.20880946208450285, + "log_ratio_spread_real_top3": 0.01086052419322281, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 85.73029954680493, + "signed_pairwise_delta_percent": -20.172637981772546 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 85.68328241018153, + "signed_pairwise_delta_percent": -20.12562084514915 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 86.43860777348267, + "signed_pairwise_delta_percent": -20.88094620845029 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 86.17619129616834, + "signed_pairwise_delta_percent": -20.618529731135947 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 41.62103939690888, + "signed_pairwise_delta_percent": -10.126432818195235 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 41.189828478080884, + "signed_pairwise_delta_percent": -9.69522189936724 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 38.78141976101907, + "signed_pairwise_delta_percent": -7.286813182305435 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 39.70095967038979, + "signed_pairwise_delta_percent": -8.20635309167615 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 1.4830480235035413, + "signed_pairwise_delta_percent": -1.4830480235035413 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.3264759887775993, + "signed_pairwise_delta_percent": -0.3264759887775993 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 1.0860524193222814, + "signed_pairwise_delta_percent": -1.0860524193222814 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.003270095020734498, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64" + ], + "real_winner": "tp4_mns8", + "regret": 0.014940997472395745, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.014940997472395745, + "sources": [ + "q30-expansion-lo/fixed-po-comparison.json", + "q30-expansion-lo/fixed-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -1.4830480235035413, + "winner_pair_m_s_percent": 1.4830480235035413, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-po low", + "config_uniform_scale": 1.1837027909165883, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.16051303629507518, + "log_ratio_spread_real_top3": 0.02273410040317836, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 80.70853515712606, + "signed_pairwise_delta_percent": -15.150873592066905 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 80.8156741203624, + "signed_pairwise_delta_percent": -15.258012555303235 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 81.60896519456668, + "signed_pairwise_delta_percent": -16.05130362950752 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 81.12861014030412, + "signed_pairwise_delta_percent": -15.570948575244959 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 37.49241217671676, + "signed_pairwise_delta_percent": -5.997805597996803 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 37.08952422311525, + "signed_pairwise_delta_percent": -5.5949176443952915 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 34.46421799160749, + "signed_pairwise_delta_percent": -2.9696114128875353 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 36.11912088646263, + "signed_pairwise_delta_percent": -4.624514307742672 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 3.527877494419683, + "signed_pairwise_delta_percent": -3.527877494419683 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 2.273410040317854, + "signed_pairwise_delta_percent": -2.273410040317854 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.6470456517583268, + "signed_pairwise_delta_percent": -0.6470456517583268 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.0064914351440450435, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32" + ], + "real_winner": "tp4_mns8", + "regret": 0.035908453871728696, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.035908453871728696, + "sources": [ + "q30-expansion-lo/fixed-po-comparison.json", + "q30-expansion-lo/fixed-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -3.527877494419683, + "winner_pair_m_s_percent": 3.527877494419683, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-po low", + "config_uniform_scale": 1.217293846920945, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.20839845449892702, + "log_ratio_spread_real_top3": 0.010876152767897662, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 85.68903876109678, + "signed_pairwise_delta_percent": -20.131377196064392 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 85.64280282131507, + "signed_pairwise_delta_percent": -20.08514125628268 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 86.3975070149251, + "signed_pairwise_delta_percent": -20.839845449892714 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 86.1361690595838, + "signed_pairwise_delta_percent": -20.578507494551413 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 41.599166472131735, + "signed_pairwise_delta_percent": -10.104559893418097 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 41.167571965772595, + "signed_pairwise_delta_percent": -9.672965387058952 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 38.758357485895, + "signed_pairwise_delta_percent": -7.263750907181354 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 39.67737178487652, + "signed_pairwise_delta_percent": -8.182765206162879 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 1.4785789280404857, + "signed_pairwise_delta_percent": -1.4785789280404857 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.3281651404981352, + "signed_pairwise_delta_percent": -0.3281651404981352 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 1.0876152767897598, + "signed_pairwise_delta_percent": -1.0876152767897598 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.003287041917935163, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns32", + "tp4_mns64" + ], + "real_winner": "tp4_mns8", + "regret": 0.014895639803871319, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.014895639803871319, + "sources": [ + "q30-expansion-lo/fixed-po-comparison.json", + "q30-expansion-lo/fixed-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -1.4785789280404857, + "winner_pair_m_s_percent": 1.4785789280404857, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 fixed-po low", + "config_uniform_scale": 1.1830544739129631, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.1602197871525023, + "log_ratio_spread_real_top3": 0.023698379402236264, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 80.67769169969341, + "signed_pairwise_delta_percent": -15.120030134634243 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 80.7830378068094, + "signed_pairwise_delta_percent": -15.225376241750244 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 81.5796402803094, + "signed_pairwise_delta_percent": -16.021978715250242 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 81.09692218428181, + "signed_pairwise_delta_percent": -15.539260619222638 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 37.48755808141785, + "signed_pairwise_delta_percent": -5.992951502697896 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 37.073974944994774, + "signed_pairwise_delta_percent": -5.579368366274817 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 34.45716604214575, + "signed_pairwise_delta_percent": -2.96255946342579 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 36.112698620380655, + "signed_pairwise_delta_percent": -4.618092041660699 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 3.527334206322636, + "signed_pairwise_delta_percent": -3.527334206322636 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 2.369837940223627, + "signed_pairwise_delta_percent": -2.369837940223627 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.6581754137676994, + "signed_pairwise_delta_percent": -0.6581754137676994 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.006603461479437911, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32" + ], + "real_winner": "tp4_mns8", + "regret": 0.03590282591969052, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.03590282591969052, + "sources": [ + "q30-expansion-lo/fixed-po-comparison.json", + "q30-expansion-lo/fixed-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -3.527334206322636, + "winner_pair_m_s_percent": 3.527334206322636, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-po low", + "config_uniform_scale": 0.6570740275710648, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.18223262538030371, + "log_ratio_spread_real_top3": 0.02603063729306998, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 3, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 90.72747001936281, + "signed_pairwise_delta_percent": -17.953572454869516 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 90.92536482192115, + "signed_pairwise_delta_percent": -18.151244664387878 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 90.84293846190077, + "signed_pairwise_delta_percent": -18.06852174527367 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 90.99716010252367, + "signed_pairwise_delta_percent": -18.223262538030372 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 46.24106247791962, + "signed_pairwise_delta_percent": -15.983259499872476 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 46.02969238739204, + "signed_pairwise_delta_percent": -15.772028622022393 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 43.35154374968354, + "signed_pairwise_delta_percent": -13.093873141924501 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 44.35514908862859, + "signed_pairwise_delta_percent": -14.09734611058145 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": true, + "real_pair_margin_percent": 3.137475730979236, + "signed_pairwise_delta_percent": -3.138149298412063 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": true, + "real_pair_margin_percent": 1.700370444058652, + "signed_pairwise_delta_percent": -1.7008263369350451 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": true, + "real_pair_margin_percent": 2.6023901618741747, + "signed_pairwise_delta_percent": -2.6030637293070016 + } + ], + "real_relative_margin_best_vs_second": 0.01714909028663425, + "real_top3_configs": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "regret": 0.03187213306057868, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.03187213306057868, + "sources": [ + "q30-expansion-lo/trace-po-comparison.json", + "q30-expansion-lo/trace-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -3.138149298412063, + "winner_pair_m_s_percent": 3.137475730979236, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 trace-po low", + "config_uniform_scale": 0.4780578821676308, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.17717267764722577, + "log_ratio_spread_real_top3": 0.0019733480856019003, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 102.02546049460537, + "signed_pairwise_delta_percent": -17.02774649052323 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 102.51013137549631, + "signed_pairwise_delta_percent": -17.51241737141418 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 102.71498176880472, + "signed_pairwise_delta_percent": -17.71726776472258 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 102.02298163614279, + "signed_pairwise_delta_percent": -17.025267632060647 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 53.38866226862059, + "signed_pairwise_delta_percent": -16.77030936999181 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 53.072230658401054, + "signed_pairwise_delta_percent": -16.453877759772272 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 51.17810724030649, + "signed_pairwise_delta_percent": -14.55975434167771 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 51.19432721693139, + "signed_pairwise_delta_percent": -14.57597431830261 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.37707949175865374, + "signed_pairwise_delta_percent": -0.37707949175865374 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.1973348085601952, + "signed_pairwise_delta_percent": -0.1973348085601952 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.19364778265184052, + "signed_pairwise_delta_percent": -0.19364778265184052 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.0019383540105721853, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32" + ], + "real_winner": "tp4_mns8", + "regret": 0.0037779133092612494, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.0037779133092612494, + "sources": [ + "q30-expansion-lo/trace-po-comparison.json", + "q30-expansion-lo/trace-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.37707949175865374, + "winner_pair_m_s_percent": 0.37707949175865374, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q30 trace-po low", + "config_uniform_scale": 0.6569411821071235, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.18204547796659498, + "log_ratio_spread_real_top3": 0.0260064332624374, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 3, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 90.70924994261264, + "signed_pairwise_delta_percent": -17.93535237811935 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 90.90683362884803, + "signed_pairwise_delta_percent": -18.132713471314766 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 90.82486083494068, + "signed_pairwise_delta_percent": -18.050444118313578 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 90.97844536115277, + "signed_pairwise_delta_percent": -18.204547796659476 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 46.2297703769945, + "signed_pairwise_delta_percent": -15.971967398947356 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 46.01838840420611, + "signed_pairwise_delta_percent": -15.760724638836459 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 43.33991907299428, + "signed_pairwise_delta_percent": -13.08224846523523 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 44.344292942369236, + "signed_pairwise_delta_percent": -14.08648996432209 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": true, + "real_pair_margin_percent": 3.1380840852501946, + "signed_pairwise_delta_percent": -3.138757652683021 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": true, + "real_pair_margin_percent": 1.6981488657060708, + "signed_pairwise_delta_percent": -1.698604758582464 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns8", + "inversion": true, + "real_pair_margin_percent": 2.599969758810901, + "signed_pairwise_delta_percent": -2.600643326243728 + } + ], + "real_relative_margin_best_vs_second": 0.017126493773631513, + "real_top3_configs": [ + "tp4_mns64", + "tp4_mns32", + "tp4_mns8" + ], + "real_winner": "tp4_mns64", + "regret": 0.03187841051786555, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.03187841051786555, + "sources": [ + "q30-expansion-lo/trace-po-comparison.json", + "q30-expansion-lo/trace-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -3.138757652683021, + "winner_pair_m_s_percent": 3.1380840852501946, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q30 trace-po low", + "config_uniform_scale": 0.4780257567661343, + "expected_config_count": 12, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.17668938090174957, + "log_ratio_spread_real_top3": 0.001510603131868793, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 12, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns16", + "inversion": false, + "real_pair_margin_percent": 101.97733324187712, + "signed_pairwise_delta_percent": -16.979619237794985 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns32", + "inversion": false, + "real_pair_margin_percent": 102.4618944001636, + "signed_pairwise_delta_percent": -17.464180396081463 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp1_mns64", + "inversion": false, + "real_pair_margin_percent": 102.66665209425709, + "signed_pairwise_delta_percent": -17.668938090174958 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp1_mns8", + "inversion": false, + "real_pair_margin_percent": 101.97521651914867, + "signed_pairwise_delta_percent": -16.977502515066533 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns16", + "inversion": false, + "real_pair_margin_percent": 53.34185487949904, + "signed_pairwise_delta_percent": -16.72350198087026 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns32", + "inversion": false, + "real_pair_margin_percent": 53.025317262729466, + "signed_pairwise_delta_percent": -16.406964364100684 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp2_mns64", + "inversion": false, + "real_pair_margin_percent": 51.13076854333851, + "signed_pairwise_delta_percent": -14.51241564470973 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp2_mns8", + "inversion": false, + "real_pair_margin_percent": 51.14883628784737, + "signed_pairwise_delta_percent": -14.530483389218585 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns16", + "inversion": false, + "real_pair_margin_percent": 0.33103325097428193, + "signed_pairwise_delta_percent": -0.33103325097428193 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns32", + "inversion": false, + "real_pair_margin_percent": 0.1510603131868839, + "signed_pairwise_delta_percent": -0.1510603131868839 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_mns64", + "inversion": false, + "real_pair_margin_percent": 0.1471039901586335, + "signed_pairwise_delta_percent": -0.1471039901586335 + }, + { + "axis_vs_real_winner": "none", + "config": "tp4_mns8", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.0014721224115223919, + "real_top3_configs": [ + "tp4_mns8", + "tp4_mns64", + "tp4_mns32" + ], + "real_winner": "tp4_mns8", + "regret": 0.003315817711349034, + "sim_winner": "tp4_mns16", + "sim_winner_real_relative_loss": 0.003315817711349034, + "sources": [ + "q30-expansion-lo/trace-po-comparison.json", + "q30-expansion-lo/trace-po-real-audit.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.33103325097428193, + "winner_pair_m_s_percent": 0.33103325097428193, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A0 fixed-pd", + "config_uniform_scale": 0.9905469229359986, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.060652013241479515, + "log_ratio_spread_real_top3": 0.05042015844107084, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 4.149707946615582, + "signed_pairwise_delta_percent": -5.042015844107099 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 5.172893426656431, + "signed_pairwise_delta_percent": -6.065201324147949 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.3735522251855693, + "signed_pairwise_delta_percent": -0.3735522251855693 + } + ], + "real_relative_margin_best_vs_second": 0.0037425080108803854, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.04237011756982145, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.04237011756982145, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -5.042015844107099, + "winner_pair_m_s_percent": 4.149707946615582, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-pd", + "config_uniform_scale": 1.0202314731471502, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.02402007999917405, + "log_ratio_spread_real_top3": 0.02402007999917405, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 1.0837798588092087, + "signed_pairwise_delta_percent": 2.1933999669826525 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 1.325627595184895, + "signed_pairwise_delta_percent": 1.9515522306069657 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.20860803293475833, + "signed_pairwise_delta_percent": -0.20860803293475833 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.002088257708717345, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.002088257708717345, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.002088257708717345, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.20860803293475833, + "winner_pair_m_s_percent": 0.20860803293475833, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A0 fixed-pd", + "config_uniform_scale": 3.0245083955601917, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.6396663007689712, + "log_ratio_spread_real_top3": 0.6391084715731847, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.6769338761256622, + "signed_pairwise_delta_percent": -0.6769338761256622 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 28.537073298091865, + "signed_pairwise_delta_percent": -63.96663007689711 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 28.481290378513215, + "signed_pairwise_delta_percent": -63.910847157318464 + } + ], + "real_relative_margin_best_vs_second": 0.006792302522137694, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.33025510615656906, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.33025510615656906, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -63.96663007689711, + "winner_pair_m_s_percent": 28.537073298091865, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-pd", + "config_uniform_scale": 3.06180896406935, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.6289288207236795, + "log_ratio_spread_real_top3": 0.628447602146478, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.2660356852052706, + "signed_pairwise_delta_percent": -0.2660356852052706 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 31.648007592775677, + "signed_pairwise_delta_percent": -62.84476021464781 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 31.696129450495825, + "signed_pairwise_delta_percent": -62.89288207236796 + } + ], + "real_relative_margin_best_vs_second": 0.002663898741543047, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.37228890038064755, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.37228890038064755, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -62.84476021464781, + "winner_pair_m_s_percent": 31.648007592775677, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-pd", + "config_uniform_scale": 2.912666802729827, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.6150853261890787, + "log_ratio_spread_real_top3": 0.6147276719238838, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.6988689401937037, + "signed_pairwise_delta_percent": -0.6988689401937037 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 26.754499952423572, + "signed_pairwise_delta_percent": -61.50853261890787 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 26.718734525904075, + "signed_pairwise_delta_percent": -61.47276719238837 + } + ], + "real_relative_margin_best_vs_second": 0.007013167281255672, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.3067524317507717, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.3067524317507717, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -61.50853261890787, + "winner_pair_m_s_percent": 26.754499952423572, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-pd", + "config_uniform_scale": 2.9547444028283523, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.6045691264386757, + "log_ratio_spread_real_top3": 0.6036185233190644, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.2938181405538578, + "signed_pairwise_delta_percent": -0.2938181405538578 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 29.686793800126072, + "signed_pairwise_delta_percent": -60.36185233190644 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 29.781854112087196, + "signed_pairwise_delta_percent": -60.45691264386757 + } + ], + "real_relative_margin_best_vs_second": 0.002942502091140886, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.3456375801249614, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.3456375801249614, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -60.36185233190644, + "winner_pair_m_s_percent": 29.686793800126072, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-po", + "config_uniform_scale": 0.9643054515322584, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.1213620172981578, + "log_ratio_spread_real_top3": 0.12020280158083751, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 5.708587111542836, + "signed_pairwise_delta_percent": -12.020280158083734 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 5.824508683274874, + "signed_pairwise_delta_percent": -12.13620172981577 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.20084412263250578, + "signed_pairwise_delta_percent": -0.20084412263250578 + } + ], + "real_relative_margin_best_vs_second": 0.002010459495370265, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.058746722254088946, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.058746722254088946, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -12.020280158083734, + "winner_pair_m_s_percent": 5.708587111542836, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-po", + "config_uniform_scale": 0.9406625738485695, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.0831623213185602, + "log_ratio_spread_real_top3": 0.07972272936918397, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 1.6605798903418276, + "signed_pairwise_delta_percent": -7.972272936918415 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 2.0045390852794442, + "signed_pairwise_delta_percent": -8.316232131856031 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.26132246488070326, + "signed_pairwise_delta_percent": -0.26132246488070326 + } + ], + "real_relative_margin_best_vs_second": 0.0026166420965441084, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.016744441542810407, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.016744441542810407, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -7.972272936918415, + "winner_pair_m_s_percent": 1.6605798903418276, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-po", + "config_uniform_scale": 0.9641676390687596, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.12135084408558959, + "log_ratio_spread_real_top3": 0.12019469213946486, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 5.707776167405595, + "signed_pairwise_delta_percent": -12.019469213946493 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 5.823391362018052, + "signed_pairwise_delta_percent": -12.13508440855895 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.20067628063206866, + "signed_pairwise_delta_percent": -0.20067628063206866 + } + ], + "real_relative_margin_best_vs_second": 0.0020087777023818365, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.05873813644442971, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.05873813644442971, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -12.019469213946493, + "winner_pair_m_s_percent": 5.707776167405595, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 fixed-po", + "config_uniform_scale": 0.9405407803694738, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.08316619889204555, + "log_ratio_spread_real_top3": 0.07973880510831316, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 1.6621874642547243, + "signed_pairwise_delta_percent": -7.973880510831312 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 2.004926842627968, + "signed_pairwise_delta_percent": -8.316619889204555 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.2619399460772448, + "signed_pairwise_delta_percent": -0.2619399460772448 + } + ], + "real_relative_margin_best_vs_second": 0.0026228330848965253, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.016760786592592503, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.016760786592592503, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -7.973880510831312, + "winner_pair_m_s_percent": 1.6621874642547243, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 trace-pd", + "config_uniform_scale": 1.4928001864415554, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 1.016421300063101, + "log_ratio_spread_real_top3": 1.016421300063101, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 314.42970560873886, + "signed_pairwise_delta_percent": 98.99891938297203 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 331.9449017039404, + "signed_pairwise_delta_percent": 101.64213000631013 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 348.43412520640095, + "signed_pairwise_delta_percent": 34.81502191302237 + } + ], + "real_relative_margin_best_vs_second": 22.203359079184416, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp4_ep1_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A0 trace-pd", + "config_uniform_scale": 1.5105914717935105, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.9648739788503272, + "log_ratio_spread_real_top3": 0.9648739788503272, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 357.1169800531145, + "signed_pairwise_delta_percent": 87.72634904159466 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 376.2523321691806, + "signed_pairwise_delta_percent": 96.48739788503273 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 392.34130165358084, + "signed_pairwise_delta_percent": 24.022421630469815 + } + ], + "real_relative_margin_best_vs_second": 34.55816479168436, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp4_ep1_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A0 trace-pd", + "config_uniform_scale": 1.340235629202675, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.6151851510539637, + "log_ratio_spread_real_top3": 0.43053455008809527, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 23.577777935433357, + "signed_pairwise_delta_percent": 42.06672378709799 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 15.538065881069413, + "signed_pairwise_delta_percent": 43.05345500880952 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 29.89518418107898, + "signed_pairwise_delta_percent": -18.465060096586843 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.16810252500411127, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A0 trace-pd", + "config_uniform_scale": 1.2809934431185706, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.4940487275442631, + "log_ratio_spread_real_top3": 0.2975515038921607, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 38.93167860985212, + "signed_pairwise_delta_percent": 24.34566988904276 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 23.332498967838475, + "signed_pairwise_delta_percent": 29.755150389216045 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 42.55321100556, + "signed_pairwise_delta_percent": -19.649722365210227 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.26279180695073645, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A0 trace-pd", + "config_uniform_scale": 1.414672147250406, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.6925792799032979, + "log_ratio_spread_real_top3": 0.6925792799032979, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 1, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 11.295399365368032, + "signed_pairwise_delta_percent": 50.146824333458675 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 9.087763981481082, + "signed_pairwise_delta_percent": 51.16511829066085 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 0.6135354983447174, + "signed_pairwise_delta_percent": -18.092809699668955 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.006154214824712234, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.006154214824712234, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.006154214824712234, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -18.092809699668955, + "winner_pair_m_s_percent": 0.6135354983447174, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 trace-pd", + "config_uniform_scale": 1.4729688476415597, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.7152768252374071, + "log_ratio_spread_real_top3": 0.7152768252374071, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 1, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 13.943039926940672, + "signed_pairwise_delta_percent": 50.65884104036849 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 10.455222177040667, + "signed_pairwise_delta_percent": 54.81170400182358 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 6.030894994380943, + "signed_pairwise_delta_percent": -16.715978521917112 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.062164651568153984, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.062164651568153984, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.062164651568153984, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -16.715978521917112, + "winner_pair_m_s_percent": 6.030894994380943, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 trace-po", + "config_uniform_scale": 0.753106573816306, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.36295220040710596, + "log_ratio_spread_real_top3": 0.3458080890155827, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 1, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 5.119345640134235, + "signed_pairwise_delta_percent": -34.58080890155828 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 6.802653056521642, + "signed_pairwise_delta_percent": -36.29522004071061 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 1.3657297298272386, + "signed_pairwise_delta_percent": -1.3745139177357077 + } + ], + "real_relative_margin_best_vs_second": 0.013750984200462835, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.07039370625209918, + "sim_winner": "tp4_ep1_mns64", + "sim_winner_real_relative_loss": 0.07039370625209918, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -36.29522004071061, + "winner_pair_m_s_percent": 6.802653056521642, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 trace-po", + "config_uniform_scale": 0.7665080250950821, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.3179742793323852, + "log_ratio_spread_real_top3": 0.3160683400465084, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 19.209826109044702, + "signed_pairwise_delta_percent": -31.606834004650835 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 19.400420037632387, + "signed_pairwise_delta_percent": -31.79742793323852 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.26044028368209854, + "signed_pairwise_delta_percent": -0.26044028368209854 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.002607797240047338, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.2117895828808376, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.2117895828808376, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -31.606834004650835, + "winner_pair_m_s_percent": 19.209826109044702, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 trace-po", + "config_uniform_scale": 0.7530456732974741, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.3629311886734452, + "log_ratio_spread_real_top3": 0.34578727672972004, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 1, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 5.1172644115479695, + "signed_pairwise_delta_percent": -34.57872767297201 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 6.800551883155552, + "signed_pairwise_delta_percent": -36.29311886734452 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 1.3654615828579657, + "signed_pairwise_delta_percent": -1.3742457707664346 + } + ], + "real_relative_margin_best_vs_second": 0.0137482658615673, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.07037121566091509, + "sim_winner": "tp4_ep1_mns64", + "sim_winner_real_relative_loss": 0.07037121566091509, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -36.29311886734452, + "winner_pair_m_s_percent": 6.800551883155552, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A0 trace-po", + "config_uniform_scale": 0.7664791140961692, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.3178774778616069, + "log_ratio_spread_real_top3": 0.3159715282630199, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": true, + "real_pair_margin_percent": 19.20014493069586, + "signed_pairwise_delta_percent": -31.59715282630199 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": true, + "real_pair_margin_percent": 19.390739890554553, + "signed_pairwise_delta_percent": -31.787747786160686 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.2516272531701875, + "signed_pairwise_delta_percent": -0.2516272531701875 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.0025194410024500513, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.21167227304868486, + "sim_winner": "tp4_ep1_mns128", + "sim_winner_real_relative_loss": 0.21167227304868486, + "sources": [ + "q235-fourcase-a0/comparison.json" + ], + "winner_deciding_axis": "mixed", + "winner_match": false, + "winner_pair_delta_s_percent": -31.59715282630199, + "winner_pair_m_s_percent": 19.20014493069586, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 fixed-pd", + "config_uniform_scale": 0.8052174107907835, + "expected_config_count": 4, + "failure": false, + "fragile_success": true, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.2684736063166279, + "log_ratio_spread_real_top3": 0.2684736063166279, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 4.149707946615582, + "signed_pairwise_delta_percent": 26.47380840647722 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 5.172893426656431, + "signed_pairwise_delta_percent": 25.450622926436374 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.3735522251855693, + "signed_pairwise_delta_percent": -0.3735522251855693 + } + ], + "real_relative_margin_best_vs_second": 0.0037425080108803854, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 fixed-pd", + "config_uniform_scale": 0.8369995512232968, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.30454826263257295, + "log_ratio_spread_real_top3": 0.30454826263257295, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 1.0837798588092087, + "signed_pairwise_delta_percent": 30.246218230322537 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 1.325627595184895, + "signed_pairwise_delta_percent": 30.004370493946848 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.20860803293475833, + "signed_pairwise_delta_percent": -0.20860803293475833 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.002088257708717345, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.002088257708717345, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.002088257708717345, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.20860803293475833, + "winner_pair_m_s_percent": 0.20860803293475833, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 fixed-pd", + "config_uniform_scale": 2.608078839444179, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.6005626377134611, + "log_ratio_spread_real_top3": 0.6000048085176745, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.6769338761256622, + "signed_pairwise_delta_percent": -0.6769338761256622 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 28.537073298091865, + "signed_pairwise_delta_percent": -60.056263771346096 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 28.481290378513215, + "signed_pairwise_delta_percent": -60.00048085176745 + } + ], + "real_relative_margin_best_vs_second": 0.006792302522137694, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.33025510615656906, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.33025510615656906, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -60.056263771346096, + "winner_pair_m_s_percent": 28.537073298091865, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 fixed-pd", + "config_uniform_scale": 2.586955736602833, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.6116904181892778, + "log_ratio_spread_real_top3": 0.6112091996120763, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.2660356852052706, + "signed_pairwise_delta_percent": -0.2660356852052706 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 31.648007592775677, + "signed_pairwise_delta_percent": -61.12091996120764 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 31.696129450495825, + "signed_pairwise_delta_percent": -61.169041818927795 + } + ], + "real_relative_margin_best_vs_second": 0.002663898741543047, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.37228890038064755, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.37228890038064755, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -61.12091996120764, + "winner_pair_m_s_percent": 31.648007592775677, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 fixed-pd", + "config_uniform_scale": 2.50822061424693, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.5825805162281401, + "log_ratio_spread_real_top3": 0.5822228619629451, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.6988689401937037, + "signed_pairwise_delta_percent": -0.6988689401937037 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 26.754499952423572, + "signed_pairwise_delta_percent": -58.25805162281401 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 26.718734525904075, + "signed_pairwise_delta_percent": -58.222286196294526 + } + ], + "real_relative_margin_best_vs_second": 0.007013167281255672, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns64" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.3067524317507717, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.3067524317507717, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -58.25805162281401, + "winner_pair_m_s_percent": 26.754499952423572, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 fixed-pd", + "config_uniform_scale": 2.494675463839052, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.5929354020035993, + "log_ratio_spread_real_top3": 0.591984798883988, + "missing_configs": [], + "n_inv_mixed": 1, + "n_inv_mns": 0, + "n_inv_tp": 1, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "none", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 0.2938181405538578, + "signed_pairwise_delta_percent": -0.2938181405538578 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 29.686793800126072, + "signed_pairwise_delta_percent": -59.1984798883988 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 29.781854112087196, + "signed_pairwise_delta_percent": -59.29354020035993 + } + ], + "real_relative_margin_best_vs_second": 0.002942502091140886, + "real_top3_configs": [ + "tp4_ep1_mns128", + "tp4_ep1_mns64", + "tp8_ep8_mns128" + ], + "real_winner": "tp4_ep1_mns128", + "regret": 0.3456375801249614, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.3456375801249614, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "tp-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -59.1984798883988, + "winner_pair_m_s_percent": 29.686793800126072, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 fixed-po", + "config_uniform_scale": 0.7759575081370588, + "expected_config_count": 4, + "failure": false, + "fragile_success": true, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.24794491019938592, + "log_ratio_spread_real_top3": 0.24794491019938592, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 5.708587111542836, + "signed_pairwise_delta_percent": 24.593646897306066 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 5.824508683274874, + "signed_pairwise_delta_percent": 24.477725325574028 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.20084412263250578, + "signed_pairwise_delta_percent": -0.20084412263250578 + } + ], + "real_relative_margin_best_vs_second": 0.002010459495370265, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 fixed-po", + "config_uniform_scale": 0.7569325524828211, + "expected_config_count": 4, + "failure": false, + "fragile_success": true, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.2890297658333315, + "log_ratio_spread_real_top3": 0.2890297658333315, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 1.6605798903418276, + "signed_pairwise_delta_percent": 28.641654118452443 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 2.0045390852794442, + "signed_pairwise_delta_percent": 28.297694923514825 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.26132246488070326, + "signed_pairwise_delta_percent": -0.26132246488070326 + } + ], + "real_relative_margin_best_vs_second": 0.0026166420965441084, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 fixed-po", + "config_uniform_scale": 0.7758466131757196, + "expected_config_count": 4, + "failure": false, + "fragile_success": true, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.24795134122075396, + "log_ratio_spread_real_top3": 0.24795134122075396, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 5.707776167405595, + "signed_pairwise_delta_percent": 24.594457841443305 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 5.823391362018052, + "signed_pairwise_delta_percent": 24.47884264683085 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.20067628063206866, + "signed_pairwise_delta_percent": -0.20067628063206866 + } + ], + "real_relative_margin_best_vs_second": 0.0020087777023818365, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 fixed-po", + "config_uniform_scale": 0.7568345476811307, + "expected_config_count": 4, + "failure": false, + "fragile_success": true, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.28901986490616804, + "log_ratio_spread_real_top3": 0.28901986490616804, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 1.6621874642547243, + "signed_pairwise_delta_percent": 28.640046544539548 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 2.004926842627968, + "signed_pairwise_delta_percent": 28.297307166166302 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.2619399460772448, + "signed_pairwise_delta_percent": -0.2619399460772448 + } + ], + "real_relative_margin_best_vs_second": 0.0026228330848965253, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 trace-pd", + "config_uniform_scale": 1.2167509036113942, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 1.5440051366971033, + "log_ratio_spread_real_top3": 1.5440051366971033, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 314.42970560873886, + "signed_pairwise_delta_percent": 150.7153663041929 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 331.9449017039404, + "signed_pairwise_delta_percent": 154.40051366971034 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 348.43412520640095, + "signed_pairwise_delta_percent": 106.41277848941169 + } + ], + "real_relative_margin_best_vs_second": 22.203359079184416, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp4_ep1_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 trace-pd", + "config_uniform_scale": 1.218869500962875, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 1.511462353051269, + "log_ratio_spread_real_top3": 1.511462353051269, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 357.1169800531145, + "signed_pairwise_delta_percent": 142.91720964148476 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 376.2523321691806, + "signed_pairwise_delta_percent": 151.14623530512694 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 392.34130165358084, + "signed_pairwise_delta_percent": 101.89467468744536 + } + ], + "real_relative_margin_best_vs_second": 34.55816479168436, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp4_ep1_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 trace-pd", + "config_uniform_scale": 1.3379683648861502, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.46724919425743383, + "log_ratio_spread_real_top3": 0.28229702985939353, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 23.577777935433357, + "signed_pairwise_delta_percent": 26.996972943866705 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 15.538065881069413, + "signed_pairwise_delta_percent": 28.229702985939365 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 29.89518418107898, + "signed_pairwise_delta_percent": -18.495216439804036 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.16810252500411127, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 trace-pd", + "config_uniform_scale": 1.220753300748686, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": false, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.4532917510074562, + "log_ratio_spread_real_top3": 0.2226509791024174, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "tpot_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 38.93167860985212, + "signed_pairwise_delta_percent": 15.892799069061242 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 23.332498967838475, + "signed_pairwise_delta_percent": 22.265097910241728 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 42.55321100556, + "signed_pairwise_delta_percent": -23.064077190503873 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.26279180695073645, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp4_ep1_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.0, + "sim_winner": "tp8_ep8_mns64", + "sim_winner_real_relative_loss": 0.0, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "none", + "winner_match": true, + "winner_pair_delta_s_percent": 0.0, + "winner_pair_m_s_percent": 0.0, + "winner_pair_relation": "match", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 trace-pd", + "config_uniform_scale": 1.430237446084041, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.487848464399216, + "log_ratio_spread_real_top3": 0.487848464399216, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 1, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 11.295399365368032, + "signed_pairwise_delta_percent": 30.727731611159037 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 9.087763981481082, + "signed_pairwise_delta_percent": 32.85538085186 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 0.6135354983447174, + "signed_pairwise_delta_percent": -15.929465588061584 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.006154214824712234, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.006154214824712234, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.006154214824712234, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -15.929465588061584, + "winner_pair_m_s_percent": 0.6135354983447174, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 trace-pd", + "config_uniform_scale": 1.4917712567044108, + "expected_config_count": 4, + "failure": true, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "HOLDS", + "log_ratio_spread_all": 0.49679676904370246, + "log_ratio_spread_real_top3": 0.49679676904370246, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 1, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 13.943039926940672, + "signed_pairwise_delta_percent": 29.838662224128264 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 10.455222177040667, + "signed_pairwise_delta_percent": 32.30699826893181 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": true, + "real_pair_margin_percent": 6.030894994380943, + "signed_pairwise_delta_percent": -17.372678635438426 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.062164651568153984, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns64" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.062164651568153984, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.062164651568153984, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -17.372678635438426, + "winner_pair_m_s_percent": 6.030894994380943, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 trace-po", + "config_uniform_scale": 0.5481032351618105, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.3220111524478986, + "log_ratio_spread_real_top3": 0.3220111524478986, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 1, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "ttft_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 5.119345640134235, + "signed_pairwise_delta_percent": 30.823724522814285 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 6.802653056521642, + "signed_pairwise_delta_percent": 29.107141724120687 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 1.3657297298272386, + "signed_pairwise_delta_percent": -1.3773907219755823 + } + ], + "real_relative_margin_best_vs_second": 0.013750984200462835, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.013750984200462835, + "sim_winner": "tp8_ep8_mns64", + "sim_winner_real_relative_loss": 0.013750984200462835, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -1.3773907219755823, + "winner_pair_m_s_percent": 1.3657297298272386, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 trace-po", + "config_uniform_scale": 0.5678289728094312, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.31919899251598766, + "log_ratio_spread_real_top3": 0.31919899251598766, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "ttft_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 19.209826109044702, + "signed_pairwise_delta_percent": 31.659458967916663 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 19.400420037632387, + "signed_pairwise_delta_percent": 31.468865039328982 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.26044028368209854, + "signed_pairwise_delta_percent": -0.26044028368209854 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.002607797240047338, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.002607797240047338, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.002607797240047338, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.26044028368209854, + "winner_pair_m_s_percent": 0.26044028368209854, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + }, + { + "case": "q235 A1 trace-po", + "config_uniform_scale": 0.548058912389237, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.32202928326406854, + "log_ratio_spread_real_top3": 0.32202928326406854, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 1, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 0, + "n_tie_tp": 0, + "objective": "e2e_mean_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 5.1172644115479695, + "signed_pairwise_delta_percent": 30.825805751400548 + }, + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 6.800551883155552, + "signed_pairwise_delta_percent": 29.109242897486777 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns64", + "inversion": true, + "real_pair_margin_percent": 1.3654615828579657, + "signed_pairwise_delta_percent": -1.3771225750063092 + } + ], + "real_relative_margin_best_vs_second": 0.0137482658615673, + "real_top3_configs": [ + "tp8_ep8_mns128", + "tp8_ep8_mns64", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns128", + "regret": 0.0137482658615673, + "sim_winner": "tp8_ep8_mns64", + "sim_winner_real_relative_loss": 0.0137482658615673, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -1.3771225750063092, + "winner_pair_m_s_percent": 1.3654615828579657, + "winner_pair_relation": "reversal", + "winner_pair_strict_reversal": true + }, + { + "case": "q235 A1 trace-po", + "config_uniform_scale": 0.5678075555479308, + "expected_config_count": 4, + "failure": false, + "fragile_success": false, + "h_scale_condition_spread_gt_log_margin": true, + "h_scale_verdict": "COUNTEREXAMPLE", + "log_ratio_spread_all": 0.3192076739943569, + "log_ratio_spread_real_top3": 0.3192076739943569, + "missing_configs": [], + "n_inv_mixed": 0, + "n_inv_mns": 0, + "n_inv_tp": 0, + "n_tie_mixed": 0, + "n_tie_mns": 1, + "n_tie_tp": 0, + "objective": "e2e_p90_ms", + "paired_config_count": 4, + "pairwise_vs_real_winner": [ + { + "axis_vs_real_winner": "mixed", + "config": "tp4_ep1_mns128", + "inversion": false, + "real_pair_margin_percent": 19.20014493069586, + "signed_pairwise_delta_percent": 31.669140146265505 + }, + { + "axis_vs_real_winner": "tp-axis", + "config": "tp4_ep1_mns64", + "inversion": false, + "real_pair_margin_percent": 19.390739890554553, + "signed_pairwise_delta_percent": 31.478545186406816 + }, + { + "axis_vs_real_winner": "mns-axis", + "config": "tp8_ep8_mns128", + "inversion": false, + "real_pair_margin_percent": 0.2516272531701875, + "signed_pairwise_delta_percent": -0.2516272531701875 + }, + { + "axis_vs_real_winner": "none", + "config": "tp8_ep8_mns64", + "inversion": false, + "real_pair_margin_percent": 0.0, + "signed_pairwise_delta_percent": 0.0 + } + ], + "real_relative_margin_best_vs_second": 0.0025194410024500513, + "real_top3_configs": [ + "tp8_ep8_mns64", + "tp8_ep8_mns128", + "tp4_ep1_mns128" + ], + "real_winner": "tp8_ep8_mns64", + "regret": 0.0025194410024500513, + "sim_winner": "tp8_ep8_mns128", + "sim_winner_real_relative_loss": 0.0025194410024500513, + "sources": [ + "q235-ablation-a1/comparison.json" + ], + "winner_deciding_axis": "mns-axis", + "winner_match": false, + "winner_pair_delta_s_percent": -0.2516272531701875, + "winner_pair_m_s_percent": 0.2516272531701875, + "winner_pair_relation": "sim-tie", + "winner_pair_strict_reversal": false + } + ], + "schema": "frontier-split-rootcause-decomposition-v2", + "summary": { + "case_count": 14, + "data_gap_count": 0, + "failure_count": 23, + "h_scale_counterexample_count": 40, + "h_scale_counterexamples": [ + { + "case": "q30 trace-pd", + "objective": "ttft_mean_ms" + }, + { + "case": "q30 trace-pd", + "objective": "ttft_p90_ms" + }, + { + "case": "q30 trace-pd", + "objective": "e2e_mean_ms" + }, + { + "case": "q30 trace-pd", + "objective": "e2e_p90_ms" + }, + { + "case": "q30 fixed-pd high", + "objective": "tpot_mean_ms" + }, + { + "case": "q30 fixed-pd high", + "objective": "tpot_p90_ms" + }, + { + "case": "q30 fixed-pd high", + "objective": "e2e_mean_ms" + }, + { + "case": "q30 fixed-po high", + "objective": "ttft_mean_ms" + }, + { + "case": "q30 fixed-po high", + "objective": "ttft_p90_ms" + }, + { + "case": "q30 fixed-po high", + "objective": "e2e_mean_ms" + }, + { + "case": "q30 fixed-po high", + "objective": "e2e_p90_ms" + }, + { + "case": "q30 fixed-pd low", + "objective": "ttft_mean_ms" + }, + { + "case": "q30 fixed-pd low", + "objective": "ttft_p90_ms" + }, + { + "case": "q30 fixed-po low", + "objective": "ttft_mean_ms" + }, + { + "case": "q30 fixed-po low", + "objective": "ttft_p90_ms" + }, + { + "case": "q30 fixed-po low", + "objective": "e2e_mean_ms" + }, + { + "case": "q30 fixed-po low", + "objective": "e2e_p90_ms" + }, + { + "case": "q30 trace-po low", + "objective": "ttft_mean_ms" + }, + { + "case": "q30 trace-po low", + "objective": "ttft_p90_ms" + }, + { + "case": "q30 trace-po low", + "objective": "e2e_mean_ms" + }, + { + "case": "q30 trace-po low", + "objective": "e2e_p90_ms" + }, + { + "case": "q235 A0 fixed-pd", + "objective": "ttft_mean_ms" + }, + { + "case": "q235 A0 fixed-pd", + "objective": "ttft_p90_ms" + }, + { + "case": "q235 A0 fixed-po", + "objective": "ttft_p90_ms" + }, + { + "case": "q235 A0 fixed-po", + "objective": "e2e_p90_ms" + }, + { + "case": "q235 A0 trace-pd", + "objective": "tpot_mean_ms" + }, + { + "case": "q235 A0 trace-pd", + "objective": "tpot_p90_ms" + }, + { + "case": "q235 A0 trace-pd", + "objective": "e2e_mean_ms" + }, + { + "case": "q235 A1 fixed-pd", + "objective": "ttft_mean_ms" + }, + { + "case": "q235 A1 fixed-pd", + "objective": "ttft_p90_ms" + }, + { + "case": "q235 A1 fixed-po", + "objective": "ttft_mean_ms" + }, + { + "case": "q235 A1 fixed-po", + "objective": "ttft_p90_ms" + }, + { + "case": "q235 A1 fixed-po", + "objective": "e2e_mean_ms" + }, + { + "case": "q235 A1 fixed-po", + "objective": "e2e_p90_ms" + }, + { + "case": "q235 A1 trace-pd", + "objective": "tpot_mean_ms" + }, + { + "case": "q235 A1 trace-pd", + "objective": "e2e_mean_ms" + }, + { + "case": "q235 A1 trace-po", + "objective": "ttft_mean_ms" + }, + { + "case": "q235 A1 trace-po", + "objective": "ttft_p90_ms" + }, + { + "case": "q235 A1 trace-po", + "objective": "e2e_mean_ms" + }, + { + "case": "q235 A1 trace-po", + "objective": "e2e_p90_ms" + } + ], + "row_count": 70, + "winner_mismatch_count": 47 + } +} diff --git a/runs/frontier-split-rootcause-v0/results/decomposition.md b/runs/frontier-split-rootcause-v0/results/decomposition.md new file mode 100644 index 0000000..bac3454 --- /dev/null +++ b/runs/frontier-split-rootcause-v0/results/decomposition.md @@ -0,0 +1,301 @@ +# Frontier good/bad split decomposition + +All 70 rows are recomputed from frozen per-config JSON values. Lower is better for every objective. `scale` is the geometric mean of `sim/real`; spreads are shown as `100 ×` log-space width, while `real margin` is the ordinary relative difference `100 × (real_second / real_best - 1)`. H-SCALE compares the log-space spread with `log1p(real margin)`, not directly with the ordinary relative margin. The real top-3 neighborhood is selected independently for every row. Directional `m_s` and `delta_s` are already expressed in log-percentage points. + +- Rows / case surfaces: 70 / 14. +- Failures (`regret > 5%`): 23. +- Exact winner mismatches: 47. +- H-SCALE counterexamples: 40. + +## Unified decomposition + +| case | objective | sim winner | real winner | regret | scale | spread all | spread real top-3 | real margin (relative) | sim-winner real loss | failure | H-SCALE | deciding axis | m_s | delta_s | n_inv_tp | n_inv_mns | n_inv_mixed | fragile success | +|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---:|---:|---:|---:|---:|---| +| q30 trace-pd | ttft_mean_ms | tp4_mns64 | tp4_mns64 | 0.0% | 45.900x | 414.0% | 73.5% | 52.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 trace-pd | ttft_p90_ms | tp4_mns64 | tp4_mns64 | 0.0% | 43.384x | 386.3% | 136.8% | 67.6% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 trace-pd | tpot_mean_ms | tp4_mns8 | tp4_mns8 | 0.0% | 4.799x | 34.2% | 11.9% | 30.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 trace-pd | tpot_p90_ms | tp4_mns8 | tp4_mns8 | 0.0% | 4.291x | 34.9% | 10.9% | 32.9% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 trace-pd | e2e_mean_ms | tp4_mns64 | tp4_mns64 | 0.0% | 8.263x | 109.9% | 50.1% | 6.6% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 trace-pd | e2e_p90_ms | tp4_mns64 | tp4_mns64 | 0.0% | 8.066x | 123.0% | 70.3% | 0.1% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes | +| q30 fixed-pd high | ttft_mean_ms | tp2_mns64 | tp4_mns32 | 58.0% | 80.274x | 664.7% | 229.2% | 0.0% | 58.0% | yes | HOLDS | mixed | 45.7% | -523.8% | 2 | 1 | 2 | no | +| q30 fixed-pd high | ttft_p90_ms | tp2_mns64 | tp4_mns64 | 56.2% | 130.663x | 709.3% | 222.7% | 0.1% | 56.2% | yes | HOLDS | tp-axis | 44.6% | -435.3% | 2 | 0 | 0 | no | +| q30 fixed-pd high | tpot_mean_ms | tp4_mns8 | tp4_mns8 | 0.0% | 6.313x | 118.1% | 95.7% | 34.4% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 fixed-pd high | tpot_p90_ms | tp4_mns8 | tp4_mns8 | 0.0% | 5.885x | 122.3% | 95.1% | 37.2% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 fixed-pd high | e2e_mean_ms | tp4_mns64 | tp1_mns32 | 1.7% | 18.867x | 226.5% | 111.5% | 0.7% | 1.7% | no | COUNTEREXAMPLE | mixed | 1.7% | -82.6% | 2 | 1 | 2 | no | +| q30 fixed-pd high | e2e_p90_ms | tp2_mns64 | tp1_mns32 | 5.5% | 24.706x | 274.2% | 172.5% | 1.0% | 5.5% | yes | HOLDS | mixed | 5.3% | -101.7% | 1 | 1 | 2 | no | +| q30 fixed-po high | ttft_mean_ms | tp4_mns16 | tp4_mns32 | 0.3% | 0.964x | 27.5% | 0.3% | 0.1% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no | +| q30 fixed-po high | ttft_p90_ms | tp4_mns16 | tp4_mns32 | 0.5% | 0.956x | 26.9% | 0.3% | 0.0% | 0.5% | no | COUNTEREXAMPLE | mns-axis | 0.5% | -0.5% | 0 | 0 | 0 | no | +| q30 fixed-po high | e2e_mean_ms | tp4_mns16 | tp4_mns32 | 0.3% | 0.964x | 27.5% | 0.3% | 0.1% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no | +| q30 fixed-po high | e2e_p90_ms | tp4_mns16 | tp4_mns32 | 0.5% | 0.956x | 26.9% | 0.3% | 0.0% | 0.5% | no | COUNTEREXAMPLE | mns-axis | 0.5% | -0.5% | 0 | 0 | 0 | no | +| q30 fixed-pd low | ttft_mean_ms | tp4_mns16 | tp4_mns16 | 0.0% | 1.368x | 43.8% | 1.5% | 1.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 fixed-pd low | ttft_p90_ms | tp4_mns16 | tp4_mns16 | 0.0% | 1.402x | 48.7% | 2.9% | 1.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q30 fixed-pd low | tpot_mean_ms | tp1_mns16 | tp4_mns32 | 14.0% | 4.952x | 52.7% | 0.2% | 0.0% | 14.0% | yes | HOLDS | mixed | 13.1% | -52.5% | 2 | 0 | 6 | no | +| q30 fixed-pd low | tpot_p90_ms | tp1_mns16 | tp4_mns8 | 13.4% | 4.950x | 53.2% | 0.3% | 0.2% | 13.4% | yes | HOLDS | mixed | 12.5% | -52.8% | 2 | 0 | 6 | no | +| q30 fixed-pd low | e2e_mean_ms | tp1_mns16 | tp4_mns32 | 26.9% | 4.423x | 59.8% | 0.2% | 0.1% | 26.9% | yes | HOLDS | mixed | 23.8% | -59.6% | 2 | 0 | 6 | no | +| q30 fixed-pd low | e2e_p90_ms | tp1_mns16 | tp4_mns8 | 25.7% | 4.421x | 60.0% | 0.2% | 0.1% | 25.7% | yes | HOLDS | mixed | 22.9% | -59.8% | 2 | 0 | 6 | no | +| q30 fixed-po low | ttft_mean_ms | tp4_mns16 | tp4_mns8 | 1.5% | 1.218x | 20.9% | 1.1% | 0.3% | 1.5% | no | COUNTEREXAMPLE | mns-axis | 1.5% | -1.5% | 0 | 0 | 0 | no | +| q30 fixed-po low | ttft_p90_ms | tp4_mns16 | tp4_mns8 | 3.6% | 1.184x | 16.1% | 2.3% | 0.6% | 3.6% | no | COUNTEREXAMPLE | mns-axis | 3.5% | -3.5% | 0 | 0 | 0 | no | +| q30 fixed-po low | e2e_mean_ms | tp4_mns16 | tp4_mns8 | 1.5% | 1.217x | 20.8% | 1.1% | 0.3% | 1.5% | no | COUNTEREXAMPLE | mns-axis | 1.5% | -1.5% | 0 | 0 | 0 | no | +| q30 fixed-po low | e2e_p90_ms | tp4_mns16 | tp4_mns8 | 3.6% | 1.183x | 16.0% | 2.4% | 0.7% | 3.6% | no | COUNTEREXAMPLE | mns-axis | 3.5% | -3.5% | 0 | 0 | 0 | no | +| q30 trace-po low | ttft_mean_ms | tp4_mns16 | tp4_mns64 | 3.2% | 0.657x | 18.2% | 2.6% | 1.7% | 3.2% | no | COUNTEREXAMPLE | mns-axis | 3.1% | -3.1% | 0 | 3 | 0 | no | +| q30 trace-po low | ttft_p90_ms | tp4_mns16 | tp4_mns8 | 0.4% | 0.478x | 17.7% | 0.2% | 0.2% | 0.4% | no | COUNTEREXAMPLE | mns-axis | 0.4% | -0.4% | 0 | 0 | 0 | no | +| q30 trace-po low | e2e_mean_ms | tp4_mns16 | tp4_mns64 | 3.2% | 0.657x | 18.2% | 2.6% | 1.7% | 3.2% | no | COUNTEREXAMPLE | mns-axis | 3.1% | -3.1% | 0 | 3 | 0 | no | +| q30 trace-po low | e2e_p90_ms | tp4_mns16 | tp4_mns8 | 0.3% | 0.478x | 17.7% | 0.2% | 0.1% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no | +| q235 A0 fixed-pd | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 4.2% | 0.991x | 6.1% | 5.0% | 0.4% | 4.2% | no | COUNTEREXAMPLE | tp-axis | 4.1% | -5.0% | 1 | 0 | 1 | no | +| q235 A0 fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.2% | 1.020x | 2.4% | 2.4% | 0.2% | 0.2% | no | COUNTEREXAMPLE | mns-axis | 0.2% | -0.2% | 0 | 0 | 0 | no | +| q235 A0 fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 33.0% | 3.025x | 64.0% | 63.9% | 0.7% | 33.0% | yes | HOLDS | tp-axis | 28.5% | -64.0% | 1 | 0 | 1 | no | +| q235 A0 fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 37.2% | 3.062x | 62.9% | 62.8% | 0.3% | 37.2% | yes | HOLDS | tp-axis | 31.6% | -62.8% | 1 | 0 | 1 | no | +| q235 A0 fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 30.7% | 2.913x | 61.5% | 61.5% | 0.7% | 30.7% | yes | HOLDS | tp-axis | 26.8% | -61.5% | 1 | 0 | 1 | no | +| q235 A0 fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 34.6% | 2.955x | 60.5% | 60.4% | 0.3% | 34.6% | yes | HOLDS | tp-axis | 29.7% | -60.4% | 1 | 0 | 1 | no | +| q235 A0 fixed-po | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 5.9% | 0.964x | 12.1% | 12.0% | 0.2% | 5.9% | yes | HOLDS | tp-axis | 5.7% | -12.0% | 1 | 0 | 1 | no | +| q235 A0 fixed-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 1.7% | 0.941x | 8.3% | 8.0% | 0.3% | 1.7% | no | COUNTEREXAMPLE | tp-axis | 1.7% | -8.0% | 1 | 0 | 1 | no | +| q235 A0 fixed-po | e2e_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 5.9% | 0.964x | 12.1% | 12.0% | 0.2% | 5.9% | yes | HOLDS | tp-axis | 5.7% | -12.0% | 1 | 0 | 1 | no | +| q235 A0 fixed-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | 1.7% | 0.941x | 8.3% | 8.0% | 0.3% | 1.7% | no | COUNTEREXAMPLE | tp-axis | 1.7% | -8.0% | 1 | 0 | 1 | no | +| q235 A0 trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.493x | 101.6% | 101.6% | 2220.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A0 trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.511x | 96.5% | 96.5% | 3455.8% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A0 trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.340x | 61.5% | 43.1% | 16.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A0 trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.281x | 49.4% | 29.8% | 26.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A0 trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.6% | 1.415x | 69.3% | 69.3% | 0.6% | 0.6% | no | COUNTEREXAMPLE | mns-axis | 0.6% | -18.1% | 0 | 1 | 0 | no | +| q235 A0 trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 6.2% | 1.473x | 71.5% | 71.5% | 6.2% | 6.2% | yes | HOLDS | mns-axis | 6.0% | -16.7% | 0 | 1 | 0 | no | +| q235 A0 trace-po | ttft_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | 7.0% | 0.753x | 36.3% | 34.6% | 1.4% | 7.0% | yes | HOLDS | mixed | 6.8% | -36.3% | 1 | 1 | 1 | no | +| q235 A0 trace-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | 21.2% | 0.767x | 31.8% | 31.6% | 0.3% | 21.2% | yes | HOLDS | mixed | 19.2% | -31.6% | 1 | 0 | 1 | no | +| q235 A0 trace-po | e2e_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | 7.0% | 0.753x | 36.3% | 34.6% | 1.4% | 7.0% | yes | HOLDS | mixed | 6.8% | -36.3% | 1 | 1 | 1 | no | +| q235 A0 trace-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | 21.2% | 0.766x | 31.8% | 31.6% | 0.3% | 21.2% | yes | HOLDS | mixed | 19.2% | -31.6% | 1 | 0 | 1 | no | +| q235 A1 fixed-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.805x | 26.8% | 26.8% | 0.4% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes | +| q235 A1 fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.2% | 0.837x | 30.5% | 30.5% | 0.2% | 0.2% | no | COUNTEREXAMPLE | mns-axis | 0.2% | -0.2% | 0 | 0 | 0 | no | +| q235 A1 fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 33.0% | 2.608x | 60.1% | 60.0% | 0.7% | 33.0% | yes | HOLDS | tp-axis | 28.5% | -60.1% | 1 | 0 | 1 | no | +| q235 A1 fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 37.2% | 2.587x | 61.2% | 61.1% | 0.3% | 37.2% | yes | HOLDS | tp-axis | 31.6% | -61.1% | 1 | 0 | 1 | no | +| q235 A1 fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 30.7% | 2.508x | 58.3% | 58.2% | 0.7% | 30.7% | yes | HOLDS | tp-axis | 26.8% | -58.3% | 1 | 0 | 1 | no | +| q235 A1 fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | 34.6% | 2.495x | 59.3% | 59.2% | 0.3% | 34.6% | yes | HOLDS | tp-axis | 29.7% | -59.2% | 1 | 0 | 1 | no | +| q235 A1 fixed-po | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.776x | 24.8% | 24.8% | 0.2% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes | +| q235 A1 fixed-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.757x | 28.9% | 28.9% | 0.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes | +| q235 A1 fixed-po | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.776x | 24.8% | 24.8% | 0.2% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes | +| q235 A1 fixed-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 0.757x | 28.9% | 28.9% | 0.3% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | yes | +| q235 A1 trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.217x | 154.4% | 154.4% | 2220.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A1 trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | 0.0% | 1.219x | 151.1% | 151.1% | 3455.8% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A1 trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.338x | 46.7% | 28.2% | 16.8% | 0.0% | no | COUNTEREXAMPLE | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A1 trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | 0.0% | 1.221x | 45.3% | 22.3% | 26.3% | 0.0% | no | HOLDS | none | 0.0% | +0.0% | 0 | 0 | 0 | no | +| q235 A1 trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.6% | 1.430x | 48.8% | 48.8% | 0.6% | 0.6% | no | COUNTEREXAMPLE | mns-axis | 0.6% | -15.9% | 0 | 1 | 0 | no | +| q235 A1 trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 6.2% | 1.492x | 49.7% | 49.7% | 6.2% | 6.2% | yes | HOLDS | mns-axis | 6.0% | -17.4% | 0 | 1 | 0 | no | +| q235 A1 trace-po | ttft_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | 1.4% | 0.548x | 32.2% | 32.2% | 1.4% | 1.4% | no | COUNTEREXAMPLE | mns-axis | 1.4% | -1.4% | 0 | 1 | 0 | no | +| q235 A1 trace-po | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.3% | 0.568x | 31.9% | 31.9% | 0.3% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no | +| q235 A1 trace-po | e2e_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns128 | 1.4% | 0.548x | 32.2% | 32.2% | 1.4% | 1.4% | no | COUNTEREXAMPLE | mns-axis | 1.4% | -1.4% | 0 | 1 | 0 | no | +| q235 A1 trace-po | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | 0.3% | 0.568x | 31.9% | 31.9% | 0.3% | 0.3% | no | COUNTEREXAMPLE | mns-axis | 0.3% | -0.3% | 0 | 0 | 0 | no | + +## H-SCALE verdict + +H-SCALE is not sufficient as stated; the following rows are counterexamples to the biconditional: + +All 40 observed counterexamples are non-failures with spread > `log1p(margin)`; there are 0 failures with spread <= `log1p(margin)`. Thus the threshold is necessary for the observed failures but not sufficient to predict them. + +| case | objective | failure | spread real top-3 | real margin | counterexample type | +|---|---|---|---:|---:|---| +| q30 trace-pd | ttft_mean_ms | no | 73.5% | 52.8% | non-failure with spread > log1p(margin) | +| q30 trace-pd | ttft_p90_ms | no | 136.8% | 67.6% | non-failure with spread > log1p(margin) | +| q30 trace-pd | e2e_mean_ms | no | 50.1% | 6.6% | non-failure with spread > log1p(margin) | +| q30 trace-pd | e2e_p90_ms | no | 70.3% | 0.1% | non-failure with spread > log1p(margin) | +| q30 fixed-pd high | tpot_mean_ms | no | 95.7% | 34.4% | non-failure with spread > log1p(margin) | +| q30 fixed-pd high | tpot_p90_ms | no | 95.1% | 37.2% | non-failure with spread > log1p(margin) | +| q30 fixed-pd high | e2e_mean_ms | no | 111.5% | 0.7% | non-failure with spread > log1p(margin) | +| q30 fixed-po high | ttft_mean_ms | no | 0.3% | 0.1% | non-failure with spread > log1p(margin) | +| q30 fixed-po high | ttft_p90_ms | no | 0.3% | 0.0% | non-failure with spread > log1p(margin) | +| q30 fixed-po high | e2e_mean_ms | no | 0.3% | 0.1% | non-failure with spread > log1p(margin) | +| q30 fixed-po high | e2e_p90_ms | no | 0.3% | 0.0% | non-failure with spread > log1p(margin) | +| q30 fixed-pd low | ttft_mean_ms | no | 1.5% | 1.3% | non-failure with spread > log1p(margin) | +| q30 fixed-pd low | ttft_p90_ms | no | 2.9% | 1.8% | non-failure with spread > log1p(margin) | +| q30 fixed-po low | ttft_mean_ms | no | 1.1% | 0.3% | non-failure with spread > log1p(margin) | +| q30 fixed-po low | ttft_p90_ms | no | 2.3% | 0.6% | non-failure with spread > log1p(margin) | +| q30 fixed-po low | e2e_mean_ms | no | 1.1% | 0.3% | non-failure with spread > log1p(margin) | +| q30 fixed-po low | e2e_p90_ms | no | 2.4% | 0.7% | non-failure with spread > log1p(margin) | +| q30 trace-po low | ttft_mean_ms | no | 2.6% | 1.7% | non-failure with spread > log1p(margin) | +| q30 trace-po low | ttft_p90_ms | no | 0.2% | 0.2% | non-failure with spread > log1p(margin) | +| q30 trace-po low | e2e_mean_ms | no | 2.6% | 1.7% | non-failure with spread > log1p(margin) | +| q30 trace-po low | e2e_p90_ms | no | 0.2% | 0.1% | non-failure with spread > log1p(margin) | +| q235 A0 fixed-pd | ttft_mean_ms | no | 5.0% | 0.4% | non-failure with spread > log1p(margin) | +| q235 A0 fixed-pd | ttft_p90_ms | no | 2.4% | 0.2% | non-failure with spread > log1p(margin) | +| q235 A0 fixed-po | ttft_p90_ms | no | 8.0% | 0.3% | non-failure with spread > log1p(margin) | +| q235 A0 fixed-po | e2e_p90_ms | no | 8.0% | 0.3% | non-failure with spread > log1p(margin) | +| q235 A0 trace-pd | tpot_mean_ms | no | 43.1% | 16.8% | non-failure with spread > log1p(margin) | +| q235 A0 trace-pd | tpot_p90_ms | no | 29.8% | 26.3% | non-failure with spread > log1p(margin) | +| q235 A0 trace-pd | e2e_mean_ms | no | 69.3% | 0.6% | non-failure with spread > log1p(margin) | +| q235 A1 fixed-pd | ttft_mean_ms | no | 26.8% | 0.4% | non-failure with spread > log1p(margin) | +| q235 A1 fixed-pd | ttft_p90_ms | no | 30.5% | 0.2% | non-failure with spread > log1p(margin) | +| q235 A1 fixed-po | ttft_mean_ms | no | 24.8% | 0.2% | non-failure with spread > log1p(margin) | +| q235 A1 fixed-po | ttft_p90_ms | no | 28.9% | 0.3% | non-failure with spread > log1p(margin) | +| q235 A1 fixed-po | e2e_mean_ms | no | 24.8% | 0.2% | non-failure with spread > log1p(margin) | +| q235 A1 fixed-po | e2e_p90_ms | no | 28.9% | 0.3% | non-failure with spread > log1p(margin) | +| q235 A1 trace-pd | tpot_mean_ms | no | 28.2% | 16.8% | non-failure with spread > log1p(margin) | +| q235 A1 trace-pd | e2e_mean_ms | no | 48.8% | 0.6% | non-failure with spread > log1p(margin) | +| q235 A1 trace-po | ttft_mean_ms | no | 32.2% | 1.4% | non-failure with spread > log1p(margin) | +| q235 A1 trace-po | ttft_p90_ms | no | 31.9% | 0.3% | non-failure with spread > log1p(margin) | +| q235 A1 trace-po | e2e_mean_ms | no | 32.2% | 1.4% | non-failure with spread > log1p(margin) | +| q235 A1 trace-po | e2e_p90_ms | no | 31.9% | 0.3% | non-failure with spread > log1p(margin) | + +## S0b directional conclusions + +Directional pair statistics remove the S0 range statistic's ambiguity by measuring every config against the real winner. Strict inversions use the frozen-value test `real_c > real_w and sim_c < sim_w`; simulator ties are not counted as inversions. + +### (a) Which config axis decides each material failure? + +| case | objective | real winner → sim winner | deciding axis | m_s | delta_s | -delta_s | regret | +|---|---|---|---|---:|---:|---:|---:| +| q30 fixed-pd high | ttft_mean_ms | tp4_mns32 → tp2_mns64 | mixed | 45.7% | -523.8% | 523.8% | 58.0% | +| q30 fixed-pd high | ttft_p90_ms | tp4_mns64 → tp2_mns64 | tp-axis | 44.6% | -435.3% | 435.3% | 56.2% | +| q30 fixed-pd high | e2e_p90_ms | tp1_mns32 → tp2_mns64 | mixed | 5.3% | -101.7% | 101.7% | 5.5% | +| q30 fixed-pd low | tpot_mean_ms | tp4_mns32 → tp1_mns16 | mixed | 13.1% | -52.5% | 52.5% | 14.0% | +| q30 fixed-pd low | tpot_p90_ms | tp4_mns8 → tp1_mns16 | mixed | 12.5% | -52.8% | 52.8% | 13.4% | +| q30 fixed-pd low | e2e_mean_ms | tp4_mns32 → tp1_mns16 | mixed | 23.8% | -59.6% | 59.6% | 26.9% | +| q30 fixed-pd low | e2e_p90_ms | tp4_mns8 → tp1_mns16 | mixed | 22.9% | -59.8% | 59.8% | 25.7% | +| q235 A0 fixed-pd | tpot_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 28.5% | -64.0% | 64.0% | 33.0% | +| q235 A0 fixed-pd | tpot_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 31.6% | -62.8% | 62.8% | 37.2% | +| q235 A0 fixed-pd | e2e_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 26.8% | -61.5% | 61.5% | 30.7% | +| q235 A0 fixed-pd | e2e_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 29.7% | -60.4% | 60.4% | 34.6% | +| q235 A0 fixed-po | ttft_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns128 | tp-axis | 5.7% | -12.0% | 12.0% | 5.9% | +| q235 A0 fixed-po | e2e_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns128 | tp-axis | 5.7% | -12.0% | 12.0% | 5.9% | +| q235 A0 trace-pd | e2e_p90_ms | tp8_ep8_mns64 → tp8_ep8_mns128 | mns-axis | 6.0% | -16.7% | 16.7% | 6.2% | +| q235 A0 trace-po | ttft_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns64 | mixed | 6.8% | -36.3% | 36.3% | 7.0% | +| q235 A0 trace-po | ttft_p90_ms | tp8_ep8_mns64 → tp4_ep1_mns128 | mixed | 19.2% | -31.6% | 31.6% | 21.2% | +| q235 A0 trace-po | e2e_mean_ms | tp8_ep8_mns128 → tp4_ep1_mns64 | mixed | 6.8% | -36.3% | 36.3% | 7.0% | +| q235 A0 trace-po | e2e_p90_ms | tp8_ep8_mns64 → tp4_ep1_mns128 | mixed | 19.2% | -31.6% | 31.6% | 21.2% | +| q235 A1 fixed-pd | tpot_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 28.5% | -60.1% | 60.1% | 33.0% | +| q235 A1 fixed-pd | tpot_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 31.6% | -61.1% | 61.1% | 37.2% | +| q235 A1 fixed-pd | e2e_mean_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 26.8% | -58.3% | 58.3% | 30.7% | +| q235 A1 fixed-pd | e2e_p90_ms | tp4_ep1_mns128 → tp8_ep8_mns128 | tp-axis | 29.7% | -59.2% | 59.2% | 34.6% | +| q235 A1 trace-pd | e2e_p90_ms | tp8_ep8_mns64 → tp8_ep8_mns128 | mns-axis | 6.0% | -17.4% | 17.4% | 6.2% | + +### (b) Where do TP-axis and MNS-axis inversions and boundary ties concentrate? + +`n_inv_*` counts sum strict `(real winner, c)` pair inversions across all objectives in each case; a row can contain more than one inverted pair. `n_tie_*` counts cover exact simulator ties only for the simulator-selected winner versus the real winner. Ties remain decision-boundary winner-label mismatches, not strict reversals. + +| case | n_inv_tp | n_inv_mns | n_inv_mixed | n_tie_tp | n_tie_mns | n_tie_mixed | +|---|---:|---:|---:|---:|---:|---:| +| q30 trace-pd | 0 | 0 | 0 | 0 | 0 | 0 | +| q30 fixed-pd high | 7 | 3 | 6 | 0 | 0 | 0 | +| q30 fixed-po high | 0 | 0 | 0 | 0 | 4 | 0 | +| q30 fixed-pd low | 8 | 0 | 24 | 0 | 0 | 0 | +| q30 fixed-po low | 0 | 0 | 0 | 0 | 4 | 0 | +| q30 trace-po low | 0 | 6 | 0 | 0 | 2 | 0 | +| q235 A0 fixed-pd | 5 | 0 | 5 | 0 | 1 | 0 | +| q235 A0 fixed-po | 4 | 0 | 4 | 0 | 0 | 0 | +| q235 A0 trace-pd | 0 | 2 | 0 | 0 | 0 | 0 | +| q235 A0 trace-po | 4 | 2 | 4 | 0 | 0 | 0 | +| q235 A1 fixed-pd | 4 | 0 | 4 | 0 | 1 | 0 | +| q235 A1 fixed-po | 0 | 0 | 0 | 0 | 0 | 0 | +| q235 A1 trace-pd | 0 | 2 | 0 | 0 | 0 | 0 | +| q235 A1 trace-po | 0 | 2 | 0 | 0 | 2 | 0 | +| **total** | **32** | **17** | **47** | **0** | **14** | **0** | + +- Among strict inversions, TP-axis cases are most concentrated in: q30 fixed-pd low (8), q30 fixed-pd high (7), q235 A0 fixed-pd (5), q235 A0 fixed-po (4), q235 A0 trace-po (4). +- Among strict inversions, MNS-axis cases are most concentrated in: q30 trace-po low (6), q30 fixed-pd high (3), q235 A0 trace-pd (2), q235 A0 trace-po (2), q235 A1 trace-pd (2). +- The eight Q235 A0/A1 Fixed-PD failures are all tp-axis. This does not conflict with the A0/MNS64 state-drift diagnosis, but does not constitute additional state-drift evidence for A1/MNS128. +- Q30 admission evidence does not map one-to-one to an MNS-axis deciding pair: high-pressure Fixed-PD TTFT mean is **mixed** and p90 is **tp-axis**. This is an axis/mechanism mismatch, not a direct causal contradiction: changing TP changes offered arrival pressure, while the diagnosed nonlinear event is crossing the MNS admission cap. It rules out the simpler claim that an MNS-threshold mechanism must appear as an mns-axis winner pair. +- All 14 exact simulator winner ties are mns-axis. Including these boundary errors gives MNS **17 + 14 = 31**, comparable to TP's **32** strict inversions. + +The exact ties below show that Frontier is completely insensitive to MNS changes within the corresponding case/objective cell: + +| case | objective | real winner ↔ simulator-selected tied winner (MNS pair) | +|---|---|---| +| q30 fixed-po high | ttft_mean_ms | tp4_mns32 ↔ tp4_mns16 | +| q30 fixed-po high | ttft_p90_ms | tp4_mns32 ↔ tp4_mns16 | +| q30 fixed-po high | e2e_mean_ms | tp4_mns32 ↔ tp4_mns16 | +| q30 fixed-po high | e2e_p90_ms | tp4_mns32 ↔ tp4_mns16 | +| q30 fixed-po low | ttft_mean_ms | tp4_mns8 ↔ tp4_mns16 | +| q30 fixed-po low | ttft_p90_ms | tp4_mns8 ↔ tp4_mns16 | +| q30 fixed-po low | e2e_mean_ms | tp4_mns8 ↔ tp4_mns16 | +| q30 fixed-po low | e2e_p90_ms | tp4_mns8 ↔ tp4_mns16 | +| q30 trace-po low | ttft_p90_ms | tp4_mns8 ↔ tp4_mns16 | +| q30 trace-po low | e2e_p90_ms | tp4_mns8 ↔ tp4_mns16 | +| q235 A0 fixed-pd | ttft_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 | +| q235 A1 fixed-pd | ttft_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 | +| q235 A1 trace-po | ttft_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 | +| q235 A1 trace-po | e2e_p90_ms | tp8_ep8_mns64 ↔ tp8_ep8_mns128 | + +### (c) How many exact-winner successes are margin-robust? + +Among exact winner matches, **17** are margin-robust (`margin >= 1%`) and **6** are fragile (`margin < 1%`). Separately, 14 winner-label mismatches come from exact simulator ties and sit on the decision boundary rather than being strict inversions. + +| fragile-success case | objective | real margin | +|---|---|---:| +| q30 trace-pd | e2e_p90_ms | 0.1% | +| q235 A1 fixed-pd | ttft_mean_ms | 0.4% | +| q235 A1 fixed-po | ttft_mean_ms | 0.2% | +| q235 A1 fixed-po | ttft_p90_ms | 0.3% | +| q235 A1 fixed-po | e2e_mean_ms | 0.2% | +| q235 A1 fixed-po | e2e_p90_ms | 0.3% | + +## Cross-checks + +| check | observed | frozen expectation | result | +|---|---:|---:|---| +| q30 high-pressure fixed-pd TTFT mean regret | 58.0% | 58.0% | PASS | +| q235 A0 fixed-pd TPOT mean regret | 33.0% | 33.0% | PASS | +| q235 A1 fixed-pd TPOT mean regret | 33.0% | 33.0% | PASS | +| q30 trace-pd all-objective regret | 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0% | 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0% | PASS | +| q235 Trace-PO TTFT p90 A0 to A1 regret | 21.2%, 0.3% | 21.2%, 0.3% | PASS | +| q235 Fixed-PD TPOT/E2E regrets unchanged from A0 to A1 | 33.0%, 37.2%, 30.7%, 34.6%, 33.0%, 37.2%, 30.7%, 34.6% | 33.0%, 37.2%, 30.7%, 34.6%, 33.0%, 37.2%, 30.7%, 34.6% | PASS | + +## Data gaps + +No per-config sim/real pairing gaps were found in the requested surfaces. + +## 异常 + +- 无。重算 winner/regret 与 frozen JSON 一致,硬性交叉核对均通过。 + +## Figure + +`margin-vs-residual.png` uses x = `100 × log1p(real best-vs-second relative margin)` and y = `100 ×` real-top-3 log-ratio spread. Both axes are logarithmic because the real margins span six orders of magnitude. Green means exact winner match; red means winner mismatch. The diagonal is the H-SCALE threshold, while failure itself is defined by regret, not by exact winner match. + +`decision-pair-axis.png` is a mechanism census, not a hypothesis test: for a strict winner reversal, `-delta_s > m_s` is algebraically equivalent to the mismatch. The frozen surfaces also contain deterministic winner-label mismatches caused by exact simulator ties; those points lie on `-delta_s = m_s` and are not counted as strict inversions. Color identifies the winner-deciding config axis and marker area is proportional to regret. + +## A0 vs A1 对照 + +A1 repairs the Q235 Trace-PO p90 ranking error but leaves the four large Fixed-PD TPOT/E2E regrets unchanged. + +| case / objective | A0 regret | A1 regret | +|---|---:|---:| +| trace-po / ttft_p90_ms | 21.2% | 0.3% | +| fixed-pd / tpot_mean_ms | 33.0% | 33.0% | +| fixed-pd / tpot_p90_ms | 37.2% | 37.2% | +| fixed-pd / e2e_mean_ms | 30.7% | 30.7% | +| fixed-pd / e2e_p90_ms | 34.6% | 34.6% | + +Thus Trace-PO TTFT p90 changes **21.2% → 0.3%**, whereas Fixed-PD remains **33.0% / 37.2% / 30.7% / 34.6%** in both A0 and A1. + +## 已知分量归因摘要 + +### Q30 high-pressure Fixed-PD TTFT: first-scheduling wait vs execution-side TTFT + +`queue-free TTFT` is the execution-side TTFT after removing first-scheduling wait; in the frozen diagnostic, simulated TTFT is their sum (up to floating-point rounding). + +| config | sim TPOT / real | sim slots / MNS | real slots upper / MNS | first-scheduling wait | queue-free TTFT | sim TTFT | real TTFT | real waiting max | +|---|---:|---:|---:|---:|---:|---:|---:|---:| +| tp2_mns64 | 8.09x | 58.0/64 | 7.5/64 | 38.2 ms | 191.0 ms | 229.3 ms | 194.6 ms | 0 | +| tp4_mns16 | 3.83x | 54.5/16 | 14.6/16 | 64906.3 ms | 134.7 ms | 65041.0 ms | 124.5 ms | 0 | +| tp4_mns32 | 4.67x | 65.9/32 | 14.5/32 | 27180.7 ms | 135.6 ms | 27316.3 ms | 123.2 ms | 0 | +| tp4_mns64 | 5.61x | 79.8/64 | 14.7/64 | 6367.3 ms | 136.7 ms | 6504.0 ms | 123.2 ms | 0 | + +For TP4/MNS32 minus TP2/MNS64, the observed simulator contrast is **+27087.0 ms**; removing first-scheduling wait changes it to **-55.4 ms**, aligned with the real contrast **-71.4 ms**. This localizes the Q30 high-pressure TTFT reversal to admission queueing, while the upstream service-time overprediction remains unattributed in the frozen data. + +### Q235 Fixed-PD: own-composition vs exact-state contrast + +Positive TP8−TP4 means TP8 is slower and therefore agrees with the real TP4 winner. + +| config | sim own decode batch mean | real exact decode batch mean | real token-weighted iteration | exact-state matched sim | coverage | +|---|---:|---:|---:|---:|---:| +| tp4_ep1_mns64 | 13.537 | 3.890 | 18.4710 ms | 34.2575 ms | 100.0% | +| tp8_ep8_mns64 | 18.938 | 10.921 | 27.5887 ms | 45.1603 ms | 100.0% | + +| component | own-composition TP8−TP4 | exact-state TP8−TP4 | +|---|---:|---:| +| total | -20.0663 ms | +10.9028 ms | +| moe_compute | -32.4544 ms | -1.9774 ms | +| attention | -7.4619 ms | -2.1304 ms | +| tp_dp_communication | -6.5711 ms | -6.4718 ms | +| ep_communication | +6.2354 ms | +6.5355 ms | +| moe_routing | -0.2862 ms | +0.1384 ms | +| dense_mlp_compute | -0.1231 ms | -0.0308 ms | +| pipeline_communication | +0.0000 ms | +0.0000 ms | +| runtime_overhead | +0.0000 ms | +0.0000 ms | + +The total contrast flips from **-20.0663 ms** on Frontier's own composition to **+10.9028 ms** at exact real state, a **+30.9690 ms** state-composition shift. The exact real iteration contrast is **+9.1177 ms** and the frozen real TPOT contrast is **+6.9500 ms**. This supports closed-loop state/composition drift for Q235 Fixed-PD; it does not by itself attribute every remaining conditional execution residual. diff --git a/runs/frontier-split-rootcause-v0/results/margin-vs-residual.png b/runs/frontier-split-rootcause-v0/results/margin-vs-residual.png new file mode 100644 index 0000000..f4e9396 Binary files /dev/null and b/runs/frontier-split-rootcause-v0/results/margin-vs-residual.png differ