Record code-trace canary fidelity gates

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.gitignore

@@ -19,3 +19,25 @@ runs/**/*.jsonl
 .ruff_cache/
 # Recovered dash1 interaction-run stores (100 MB raw tune logs, kept on disk only)
 recovered-stores/
+
+# Local reference material and accidental shell output.
+/AITuner系统优化与挑战.pdf
+/16
+/docs/assets/simulator-fidelity/*.svg
+
+# Generated experiment state. Protocols, analysis code, compact result tables,
+# and frozen manifests remain tracked next to these directories.
+/runs/frontier-phase-factorial-v0/fleet-artifacts*/
+/runs/frontier-phase-factorial-v0/fleet-state*/
+/runs/frontier-phase-factorial-v0/invalid-overlap-*/
+/runs/frontier-phase-factorial-v0/simulator-smoke/
+/runs/frontier-phase-factorial-v0/simulator-*/cache
+/runs/frontier-phase-factorial-v0/simulator-*/runs/
+/runs/frontier-phase-factorial-v0/simulator-*/traces/
+/runs/frontier-phase-factorial-v0/results/final/qwen30-prefill-ranking.png
+/runs/frontier-qwen30-vllm020-profile-v1/comparison/
+/runs/frontier-qwen30-vllm020-profile-v1/fleet-artifacts/
+/runs/frontier-qwen30-vllm020-profile-v1/fleet-state/
+/runs/frontier-multicase-sufficiency-v1/fleet-artifacts/
+/runs/frontier-multicase-sufficiency-v1/fleet-state/
+/runs/frontier-multicase-sufficiency-v1/frontier-smoke-failure/

.research/experiments/frontier-split-rootcause-s0-20260720.md

@@ -0,0 +1,60 @@
+# 实验 S0：good/bad case 分裂的统一分解（margin vs differential residual）
+
+> **状态：** 已完成（2026-07-20；含 S0b 方向化修正与一轮 strict review 修复）
+>
+> 用户指令：核心要务是分析为什么部分 case 下 Frontier work、部分不 work 的 system 根因；本 card 是该诊断 campaign 的第一个 slice，仅使用 frozen artifacts，零 GPU 成本。人工 review 由用户的直接指令（"只有做好这个分析我们才能推进下一步"）满足。
+
+## Claim 与决策
+
+- **Parent claim：** ongoing.md H2——误差机制是 action-conditioned residual；本实验把它细化为"分裂从哪来"。
+- **现象（已冻结）：** 同一 best-effort Frontier 栈上，Q30/Q235 的 Trace-PD 与多数 PO 面 selection 近优，而 Fixed-PD 面 14–58% regret；失败 objective 随负载档切换（Q30 低压 TPOT/E2E 全反、高压 TTFT 56–58%）；A1 collective profile 修复了 Q235 Trace-PO p90 21.2%→0.3% 但对 Fixed-PD 33% 完全无效（本 card frozen-inputs/q235-ablation-a1）。
+- **Competing hypotheses：**
+  - **H-SCALE：** 分裂完全由「config-differential residual vs 真机 decision margin」的关系解释：good case 的 sim/real 比值跨 config 近似均匀（乘性偏移，argmin 不变），bad case 的比值跨 config 分散且超过 margin。workload shape 本身不需要出现在解释里。
+  - **H-THRESH：** 绝对 service-time 高估近似均匀，但与离散机制（MNS admission cap、MoE token-bucket、graph bucket）交互后被转换为 config-differential 误差；fixed uniform workload 把所有请求同步到同一 state 轨迹，使阈值交叉对整个 cell 相干生效；trace 的长度/到达异质性把阈值效应摊平。
+  - **H-STATE：** 失败由 simulator 闭环 batch state 分布漂移主导（Q235：sim decode batch 13.5 vs real 3.9 + B4→B5 profile cliff 正反馈）；即使打破 workload 同步性，闭环漂移仍可翻转排序。
+  - 三者关系：H-SCALE 是现象层（必要条件），H-THRESH/H-STATE 是 differential residual 的两种产生机制，可共存但可判别（见事前预测）。
+- **事前预测：**
+  - H-SCALE 成立 ⟺ 对每个 case×objective，failure 恰好发生在「top 邻域 log-ratio spread > log1p(真机相对 margin)」处（两侧同为 log-space 尺度），无反例。
+  - H-THRESH 独有：bad case 的 differential 误差集中于阈值语义分量（first-scheduling wait、bucket 跳变段），且 sim-only 反事实（去阈值/加 jitter）恢复排序——Q30 高压 TTFT 的 admission 反事实已支持一例。
+  - H-STATE 独有：差异化误差在去掉阈值分量后仍在 execution 项内（Q235 Fixed-PD 的 own-composition −20.07 vs exact-state +10.90 已支持一例）。
+- **判定规则：** S0 只裁决 H-SCALE 与「分量定位」（queue vs execution）；H-THRESH/H-STATE 的干预判别属 S1+（sim-only 反事实）与 GPU 实验（需另行 review）。若 H-SCALE 出现反例（good case 有 spread>margin 仍选对，或 bad case spread<margin），必须原样报告，不得平滑。
+
+## Setup
+
+- **输入（全部 frozen，runs/frontier-split-rootcause-v0/frozen-inputs/）：** q30-trace-pd（graph-piecewise comparison）、q30-fixed-hi（fixed-pd/fixed-po 高压面）、q30-expansion-lo（低压 fixed-pd/fixed-po/trace-po）、q235-fourcase-a0、q235-ablation-a1（+provenance）、q235-state-diag、q30-admission-diag。来源 cpfs 路径与 SHA 见各目录内 manifest/launch 记录。
+- **计算（每 case×objective）：** per-config 比值 r_c=sim_c/real_c；config-uniform scale=geomean(r_c)；differential residual=log-ratio spread（全 surface 与真机 top-3 邻域各一）；真机 relative margin（best 与 2nd-best、best 与 sim-winner 的真机值差）；failure flag=regret>5%；H-SCALE 检验=failure ⟺ 邻域 spread>log1p(margin)（review 修正：初版直接以 ln 差比较普通 relative margin，尺度不一致；修正后 70 行 verdict 不变）。
+- **分量定位：** q30-admission-diag 提供 TTFT=first-scheduling wait+prefill execution 分解；q235-state-diag 提供 own-composition vs exact-state contrast；把这些已知分量证据合并进统一表。
+- **交叉核对：** 重算的 regret 必须与各 frozen comparison.md 表一致（抽查 58.0%、33.0%、0.0%）；A0 vs A1 的 Q235 对比必须复现 trace-po p90 21.2%→0.3%、fixed-pd 四项不变。
+
+## 预期产物与 review
+
+- runs/frontier-split-rootcause-v0/analyze_split_decomposition.py（只读 frozen-inputs，确定性输出）
+- runs/frontier-split-rootcause-v0/results/decomposition.{json,md}：统一表，每行 case×objective，列出 winner、regret、scale、spread（全/邻域）、margin、H-SCALE verdict、已知分量归因
+- runs/frontier-split-rootcause-v0/results/margin-vs-residual.png：x=真机 margin，y=邻域 differential residual，点色=selection 对错；H-SCALE 成立则对错点被对角线分离
+- 人工验收：编排者亲自重跑脚本、抽查交叉核对数字、亲自查看渲染图
+
+## 复现信息
+
+- **Code：** AITuner branch feature/sim，自 HEAD 18c0b25 起；worker/reviewer job-id 见下方「Review 与 provenance 补记」；脚本与产物随本 card 同一 commit 入库（含 frozen-inputs 本地拷贝）。
+- **Environment：** 本地 workstation，CPU-only，python3+matplotlib；不访问远端。
+- **已知 deviation：** frozen-inputs 是 cpfs 原件的本地拷贝（scp，2026-07-20）；q30-expansion-lo 的低压 Fixed-PD 面已被高压面取代为 primary，本分析将两档并列为独立观测，不混合。
+
+## 结果
+
+- **观察事实：**
+  - 70 行（14 个 case surface）全部算出，无数据缺口；四组硬性交叉核对通过；连续运行产物 SHA 一致。
+  - **H-SCALE 判为必要非充分**：23 个 material failure（regret>5%）全部满足「top-3 邻域 log spread > log1p(margin)」，无一例失败发生在 residual 小于 margin 处；但另有 40/70 行同样满足该条件却均非 material failure（其中仅 16 行 exact winner match，其余 24 行是小 regret 的 winner 错位——14 个 MNS 精确 tie 与 10 个 strict reversal）——无方向 spread 不携带决策信息。
+  - **S0b 方向化后的机制普查**：23 个 material failure 的 winner-deciding pair 分布为 tp-axis 11、mixed 10、mns-axis 2（Q235 A0/A1 Fixed-PD 的 8 个 TPOT/E2E failure 全为 tp-axis；Q30 Fixed-PD 高低压为 tp/mixed）；仅有的 2 个 mns-axis failure 是 Q235 A0/A1 Trace-PD E2E p90（regret 6.2%，勉强越过 5% 门槛）。trace 面严格反序中 tp-axis 为 0（q30 trace-pd 三轴全 0）。
+  - **A1 对照的轴分解**：serving-matched collective profile 把 Q235 两个 PO 面的 tp-axis 反序从 4/4 清零（trace-po p90 regret 21.2%→0.3%），但 Fixed-PD 仅 5→4、四项 regret 一位小数不动——prefill 路径的 TP-differential 误差源=collective profile（可修），decode 耦合的 TP-differential 误差另有来源。
+  - **MNS 不敏感缺陷**：14 个 winner-label mismatch 是 simulator 逐位相等的 tie，全部 mns-axis（如 q30 fixed-po 的 MNS16↔32、q235 fixed-pd 的 MNS64↔128）；tie 计入后 MNS 边界误差 31 与 TP 严格反序 32 相当，但 MNS 侧 regret 小。
+  - **成功的鲁棒性**：23 个 exact-winner success 中 17 个 margin-robust（margin≥1%），6 个 fragile（含 q30 trace-pd E2E p90 的 0.1% margin 与 q235 A1 fixed-po 四项）。
+  - **面级 scale 对照**：prefill-only 面 geomean scale 0.96–1.37×（绝对预测基本准确），含 decode 的面 4.3–130×——绝对误差灾难集中于 decode。
+- **异常：** 无数据异常。strict review（FAIL：3 Major/1 Minor）指出 H-SCALE 尺度混用（log spread vs relative margin）、tie 轴普查缺失、card 状态过期、Q235 一致性表述过强；全部修复，修复后 70 行 verdict 逐行不变。
+- **含义：** 分裂的现象层解释是「margin 保护 + config-differential 误差」。机制层上，21/23 个 material failure 由 TP/mixed pair 决定，且所有大 regret（≥13%）failure 都发生在含 decode 的面上：其中 Q235 Fixed-PD 有 state-drift 直接证据、Q30 高压 TTFT 有 admission 反事实证据，而 **Q30 低压 TPOT/E2E 反转的机制尚未诊断**（S1 目标）。「decode 耦合的 TP-differential 误差是主要载体」是当前最强归纳，不是对全部 failure 的已证机制归因；2 个 mns-axis 边缘 failure（6.2%）在该归纳之外。trace 面成功伴随「TP 反序为零 + TP margin 宽」，但「误差小」与「margin 宽」谁是主因仍未判——这正是 H-THRESH vs H-STATE 的判别缺口。轴标签与机制不一一对应（Q30 admission 是 MNS 阈值机制但 deciding pair 为 tp/mixed，因 TP 改变到达压力）。
+- **Claim update：** H2（action-conditioned residual）supported 且被细化：residual 的决策相关分量集中在 TP 轴、由 decode 状态耦合产生；H-SCALE 降级为必要条件；H-THRESH/H-STATE 保持 competing，待 S1 判别。
+- **下一步：** S1（sim-only 反事实：Q30 低压 Fixed-PD TPOT 反转的分量定位——这是唯一无机制解释的 material failure；fixed workload jitter 判别 H-THRESH vs H-STATE）；GPU 判别实验（加压 Trace-PD、jittered Fixed-PD 真机面，dash1–4）另行出 card 供 review。
+
+## Review 与 provenance 补记
+
+- S0 worker：codex `task-mrsn1s9c-z6ha0w`；S0b：`task-mrsnjzn6-k18x4n`（resume）；strict reviewer（fresh 只读）：`task-mrsocmlb-386btc`（verdict FAIL）；修复轮：`task-mrsop06n-pwxo0b`（fresh writable）。编排者独立验收：脚本重跑、SHA 比对、两图目视检查。
+- 产物 SHA（修复后）：decomposition.json `81ea56b2…`、decomposition.md `4d19af54…`、margin-vs-residual.png `df2110c4…`、decision-pair-axis.png `2e787301…`。

.research/experiments/frontier-workload-regime-taxonomy-20260720.md

@@ -0,0 +1,183 @@
+# Frontier workload-regime taxonomy
+
+- Date: 2026-07-20
+- Status: proposed; awaiting review before workload generation or GPU runs
+- Scope: explain when Frontier preserves the real-system config ranking, rather than merely comparing Fixed with Trace
+
+## Claim under test
+
+Frontier reliability is controlled by three quantities:
+
+1. the latency-model residual between simulator and real execution;
+2. the closed-loop gain from timing to scheduler state (batch, MoE routing, CUDA-graph bucket, MNS occupancy, admission/KV pressure);
+3. the real decision margin between configurations.
+
+For a config pair `a,b`, define
+
+```text
+D_real(a,b) = log L_real(a) - log L_real(b)
+delta(a,b)  = [log L_sim(a)-log L_real(a)]
+              - [log L_sim(b)-log L_real(b)]
+slack(a,b)  = sign(D_real) * [D_real + delta]
+```
+
+`slack < 0` means the simulator reverses the real pairwise ordering. The primary hypothesis is that reversals occur when simulator and real execution land on different sides of a scheduler-state knee, or when the real decision margin is too small to absorb the differential residual. `Fixed` and `Trace` are not themselves the causal classes.
+
+## Existing evidence motivating the experiment
+
+- Q30 Trace-PD preserves all six objective winners, but many pairwise residuals oppose the real winner. Its success is therefore often margin protection, not zero residual.
+- Q235 Trace-PD preserves TTFT/TPOT winners but misses E2E p90 by 6.2%; Trace is not universally safe.
+- Q30/Q235 Fixed-PD decode objectives show negative minimum signed slack and 13--37% regret.
+- In Q30 low-load Fixed-PD, Frontier's batch-1 TP ordering is correct, while the closed-loop simulator increases TP4's effective batch and changes the MoE cost enough to reverse the ordering. This identifies a concrete state knee, but does not yet establish a general rule.
+
+## Workload families
+
+All comparisons use the same request multiset where applicable, the same total observation window, and the same normalized offered decode load
+
+```text
+rho = request_rate * E[output_tokens] / measured_reference_decode_capacity.
+```
+
+This avoids equating equal request rates with equal load.
+
+| ID | Shape / request lengths | Arrival process | Prefix/session state | Isolated effect |
+|---|---|---|---|---|
+| W0 | short fixed `2048 -> 128` | uniform | off | known low-residence failure anchor |
+| W1 | trace-mean fixed ISL/OSL | uniform | off | homogeneous baseline |
+| W2 | trace-mean fixed ISL/OSL | trace timestamps | off | arrival burst only |
+| W3 | exact trace ISL/OSL multiset | uniform | off | length heterogeneity only |
+| W4 | exact trace ISL/OSL multiset | trace timestamps | off | length + burst |
+| W5 | exact trace prompts/ISL/OSL | uniform | exact prefix/session identity | prefix state without burst |
+| W6 | exact trace prompts/ISL/OSL | trace timestamps | exact prefix/session identity | full production trace |
+
+Prefix is intentionally a nested factor: enabling a synthetic prefix graph on fixed identical requests would introduce a different workload rather than isolate production prefix reuse. Therefore this is not presented as a full `2^3` factorial.
+
+## Load sweep and expected patterns
+
+Simulator discovery sweep: `rho in {0.05, 0.25, 0.50, 0.90, 1.20}`. The points mean deep low load, light batching, moderate batching, capacity knee, and overload; their request rates are derived independently for every workload family.
+
+| Pattern | Observable state | Prediction for Frontier |
+|---|---|---|
+| P1 singleton-linear | real and sim stay below the first batch/graph knee | works if the batch-1 operator ordering is correct |
+| P2 knee-straddling | real and sim occupy opposite sides of a batch/MoE/graph/MNS knee | fails systematically; Fixed-PD is the current example |
+| P3 same-side batched | both systems cross the same knee and remain below admission pressure | works if batch-conditioned operator ordering is correct |
+| P4 capacity/admission aligned | both systems are governed by the same capacity bottleneck | TTFT/config winner may work despite large absolute error; E2E/MNS can remain fragile |
+| P5 heterogeneity-smoothed | broad lengths reduce coherent threshold occupancy at matched `rho` | may work; this is a hypothesis, not an established explanation |
+| P6 burst-sensitive | same request multiset, but transient queue/MNS occupancy differs | mean ranking may work while TTFT/E2E tail ranking fails |
+| P7 prefix-state-sensitive | hit/eviction and reused-token distributions differ | TTFT ranking fails unless prefix-state transitions are modeled; decode TPOT may remain stable |
+| P8 decision-boundary | real config margin is comparable to run variance/residual | fragile; an exact winner match is not reliable evidence |
+
+## Hypotheses and distinguishing tests
+
+### H1: state-regime hypothesis (primary)
+
+I believe config-ranking failures occur when the latency residual moves a workload across a scheduler-state knee, because the residual is then amplified into a different batch/resource trajectory. I will verify this by checking whether signed-slack zero crossings co-locate with measured real/simulator state-knee crossings.
+
+### H2: heterogeneity-smoothing hypothesis
+
+I believe length heterogeneity can reduce coherent threshold amplification, because requests reach scheduler boundaries at dispersed times. I will verify it with W1 vs W3 and W2 vs W4 at matched `rho`, requiring a smaller real/sim state-distribution gap rather than merely a correct winner.
+
+### H3: bottleneck/margin-protection alternative
+
+Trace success may instead be explained entirely by a large real decision margin or a shared capacity bottleneck. This hypothesis wins over H2 if W3/W4 do not reduce state-distribution error after matching load and margin, while ranking correctness remains predicted by margin alone.
+
+### H4: burst and prefix are independent failure channels
+
+I believe arrival bursts primarily affect waiting/admission and tail TTFT/E2E, whereas prefix mismatch primarily affects prefill/TTFT state. I will verify this with W1/W2, W3/W4, and W3/W5 paired comparisons.
+
+## Configuration and model scope
+
+Discovery uses Qwen30B because its 12-cell `TP x MNS` surface already has simulator and real anchors:
+
+- TP: `{1, 2, 4}`
+- MNS: `{8, 16, 32, 64}`
+- objectives: mean/p90 TTFT, TPOT, E2E
+
+Qwen235B is a held-out confirmation, not pooled into discovery:
+
+- existing four feasible TP/MNS configurations;
+- only the workload/load patterns that discriminate H1--H4 after Q30 converges.
+
+## Measurements
+
+End-to-end:
+
+- completed/failed requests and achieved request/token rate;
+- TTFT, TPOT, E2E mean/p50/p90/p95;
+- config regret, pairwise agreement, signed decision slack;
+- run-to-run winner stability.
+
+Closed-loop state:
+
+- prefill/decode batch-size histograms and time-weighted batch;
+- Running/Waiting distributions and admission delay;
+- MNS active-token occupancy and KV/context pressure;
+- CUDA-graph bucket residency and fallback frequency;
+- prefix hit/reused-token/eviction distributions for W5/W6.
+
+## Decision rules
+
+A workload/load region is:
+
+- **reliable** if regret is at most 5%, pairwise agreement is at least 0.8 at two adjacent load points, and the winner is stable across confirmation trials;
+- **fragile** if regret is at most 5% but the real margin overlaps run uncertainty, or a small rate/timing perturbation changes the winner;
+- **failed** if regret exceeds 5% or a decision-critical pair has negative signed slack;
+- **mechanistically explained by H1** only if the ranking transition co-locates with an observed state-regime transition. Correlation with the Fixed/Trace label is insufficient.
+
+H2 is supported only if the heterogeneous member of a matched pair reduces state-distribution error and shifts the failure boundary in repeated trials. A correct winner alone does not support smoothing.
+
+## Execution plan after review
+
+1. Materialize W0--W6 with one manifest recording request multiset, arrival timestamps, prefix identity, rate contract, and hashes.
+2. Run the simulator sweep across `rho` and the Q30 surface; emit a per-stage state ledger.
+3. Select real-machine pilot points only around the predicted knees plus one safe-side control. Use guard configs `TP1/MNS64`, `TP4/MNS8`, and `TP4/MNS64`; add `TP2/MNS32` only if the transition is not bracketed.
+4. Use only `dash1`, `dash2`, `dash3`, and `dash4`, each verified as an 8×H20 host. `dash0` is excluded from probing, synchronization, and execution. Pin one independent experiment group to each host so at most four groups run in parallel; do not split one trial across hosts.
+5. Run one pilot trial per selected point. Confirm only hypothesis-discriminating points with three fresh-server trials and rotated order.
+6. Apply the resulting classifier unchanged to the Q235 held-out cases.
+
+Provisional four-way allocation after the simulator identifies the discriminating points:
+
+| Host | Experiment group | Primary contrast |
+|---|---|---|
+| dash1 | homogeneous controls | W0/W1 across safe side and first knee |
+| dash2 | arrival effect | W1 vs W2 and W3 vs W4 |
+| dash3 | length heterogeneity | W1 vs W3 and W2 vs W4 |
+| dash4 | prefix/full trace | W3 vs W5 and W4 vs W6 |
+
+The groups are logical queues, not permanent ownership: if a host probe fails, that host is excluded and its group waits or moves to another permitted idle host. Cross-host latency values are not pooled until a common canary config verifies that host effects are within run uncertainty.
+
+No GPU run is authorized by this card yet. The review decision is whether the workload decomposition and decision rules are sufficient to implement the materializer and launch Phase 1.
+
+## Expected figure
+
+The accompanying mock figure is schematic, not data. Panel A shows the state knee that real and simulator trajectories may cross at different loads. Panel B shows the corresponding minimum signed decision slack; a negative value denotes a ranking reversal. The claim is supported only if measured zero crossings and state knees align across workload families.
+
+## Risks and controls
+
+- Equal `rho` does not guarantee equal prefill pressure; report both prefill and decode offered work and stratify if necessary.
+- Full-trace overload can collapse all configs to similarly poor latency. Such points identify a capacity-limited region but cannot validate fine-grained ranking.
+- MNS ties and censored/failed requests can create false winners; exclude invalid cells before calculating regret and report the exclusion.
+- One trace cannot establish generality. The initial result is a mechanism boundary for this trace/model/hardware, followed by held-out Q235 validation.
+
+## Execution log
+
+### 2026-07-20: materialization and simulator launch
+
+- Code baseline: `feature/sim@157bf36` for the valid v4 sweep.
+- Hosts probed: `dash1`, `dash2`, `dash3`, `dash4`; each exposed 8 NVIDIA H20 GPUs with 0 MiB used at probe time. `dash0` was not probed or used.
+- Source cohort: 129 Q30 Trace-PD requests. The private artifact supplies exact prompts, lengths, outputs, timestamps, sessions, and runtime block identities; the simulator projection retains only the first `floor(ISL/16)` complete block identities.
+- Materialized: 35 cases = W0--W6 × `rho {0.05,0.25,0.50,0.90,1.20}`. Audit passed request count, exact decode offered load, empirical arrival rate, prefix block count, and prefix-off empty identity vectors.
+- Simulator smoke: W0 / `rho=0.05` / TP4-MNS64 completed 129/129. Simulator TTFT mean/p90 was 109.81/124.16 ms and TPOT mean/p90 was 36.26/36.79 ms. This is a harness check, not real-system fidelity evidence.
+- Invalid attempts retained for audit: v1 had a Bash argument-expansion error; v2 mixed multiple workload families into a runner that requires strictly increasing anchors from one family; v3 exposed a scikit-learn cache-version mismatch. None is used as scientific evidence.
+- Valid v4 controls: isolated output/predictor cache per TP/prefix group; scikit-learn 1.9.0 matching the predictor cache format; per-family five-point runner invocations; stage batch ledger enabled; TP1 exempted from the collective fallback gate because a single rank has no all-reduce.
+- Active v4 allocation: dash1=TP1 prefix off/on, dash2=TP2 prefix off/on, dash3=TP4 prefix off, dash4=TP4 prefix on. The four fleet jobs are running from fresh `sim-v4` output roots. First-process audit found the explicit isolated `--metrics_config_cache_dir` on all hosts and zero cross-version warnings.
+- First valid v4 tranche: 16/16 observed cells completed, each with 129 requests, request metrics, and a stage-batch ledger; no traceback, fallback, or version warning was found. The tranche covers all five W0 load points at TP1/TP2/TP4-MNS8 plus the first W5 prefix points at TP4-MNS8.
+- Early load-boundary observation: W0 at `rho=0.05` is low-latency for TP4-MNS8 (simulator TTFT mean 109.25 ms) but already queues for TP1-MNS8 (25.70 s); at `rho=0.25`, even TP4-MNS8 reaches 27.13 s mean TTFT. Because `rho` normalizes decode tokens only, high-rate short-output W0 also raises prefill and active-sequence pressure. These points map the overload boundary and are not eligible as reasonable-latency real pilots.
+- Real-runtime gate: a stock vLLM 0.20.0 environment passed import/H20 checks but used CUDA 13.0, so it is excluded from comparison with the historical CUDA 12.9 baseline. The replacement environment `vllm-0.20.0-cu129-workload-regime-v2` passes `vllm CLI=0.20.0+cu129`, torch `2.11.0+cu129`, CUDA runtime 12.9, H20 visibility, and all 179 package dependency checks. The first CPFS install used file copies and was stopped after download because it was still copying roughly 7 GB after 12 minutes; its incomplete directory is retained with an `invalid-copy-incomplete` suffix, while v2 uses same-filesystem hardlinks from the validated cache.
+- Load-contract correction: the original Fixed-PD surface held request rate per GPU constant, so global arrival rate scaled with TP. The v4 sweep holds global arrival rate constant and is retained as the control that isolates service-topology changes. A matched per-GPU sweep is now required to reproduce the original closed-loop intervention: TP1/TP2/TP4 receive `1x/2x/4x` global arrival rate at the same per-GPU `rho`.
+- Per-GPU low-load materialization: 105 cases = W0--W6 × `rho {0.0025,0.005,0.01,0.02,0.05}` × TP `{1,2,4}` were generated under `traces-per-gpu-low`. Audit passed 105 unique paths, 129 public/private rows per case, digests, arrival alignment, and exact `global_rate / TP = per_gpu_rate`. W0 `rho=0.01` is 0.239375 req/s/GPU, bracketing the original 0.215 req/s/GPU Fixed-PD point with `rho=0.005`.
+- The per-GPU sweep writes to a separate `sim-per-gpu-v1` result root but reuses the completed v4 predictor cache for the same TP/prefix/config. Predictor cache provenance is explicit in every surface manifest; workload results and state ledgers are never shared.
+- `wait_and_dispatch_per_gpu.sh` is active locally as a serial gate. It requires all four exact v4 run directories to contain `finished_at` and exit code zero before probing dash1--dash4 and dispatching the four per-GPU jobs; it does not launch a second sweep while v4 is still consuming CPU.
+- A first materialization attempt rounded both `rho=0.005` and `rho=0.01` to the same `rho0p01` directory. Digest validation stopped before simulator launch; the invalid directories were retained with an `invalid-rho-label-collision` suffix. The label function now preserves up to 12 significant digits and has a regression test.
+
+Current decision: finish the v4 fixed-global-rate control, then reuse its trained predictors for the low-load per-GPU sweep before selecting discriminating real-machine pilot points. No real latency result from vLLM 0.20.2 will be compared with the historical vLLM 0.20.0 baseline until the runtime-version gate is resolved.

.research/experiments/qwen235-fixed-pd-state-matched-diagnosis-20260719.md

@@ -0,0 +1,45 @@
+# 实验：Qwen235 Fixed-PD state-matched decode diagnosis
+
+> **状态：** 已批准，执行中
+>
+> 本 card 记录 Qwen235 Fixed-PD 在真实 collective profile 后仍保留 30%+ selection regret 的下一层判因实验。
+
+## Claim 与决策
+
+- **Parent claim：** Qwen235 Fixed-PD 的错误排序来自 action-conditioned decode residual，而不是缺失的 TP8 all-reduce profile。
+- **目的：** 区分 simulator 错在 state distribution，还是相同 state 下的 conditional execution-time composition。
+- **Competing hypotheses：** H1：Frontier 生成的 decode batch/context/graph state 与真机不同；H2：state 对齐后 Frontier 仍把 TP8/EP8 预测得更快，误差位于 MoE/EP、graph 或 attention 的 conditional stage model。
+- **事前预测：** 真机 config contrast 为 `TP8-TP4=+6.95 ms/token`，A1 simulator 为 `-20.07 ms/token`。若 H1 成立，用真实 state 重加权后 contrast 应翻正；若 H2 成立，matched-state contrast 仍为负。
+- **判定规则：** 先比较 frozen-real coarse state 与 full-ledger Frontier state。state 明显不匹配则补 iteration telemetry；state 支持重叠且 matched-state predictor 仍反序，才进入 stage breakdown。任何 stage 只有在 measured substitution 能使 winner 翻转时才称为 decision-bearing root cause。
+
+## Setup
+
+- **自变量：** state source（frozen real / Frontier）；后续 matched-state replay 中固定 decode batch、context-length、graph bucket 与 routing load。
+- **控制变量：** Qwen235 FP8、vLLM 0.20.0、H20、Fixed-PD 4096→256、0.2 req/s/GPU、MBT8192、MNS64、TP4/EP1 与 TP8/EP8、Frontier commit、r2 operator profiles、A1 measured collective CSV 全部冻结。
+- **选择 MNS64：** A1 中 MNS64/128 的 TTFT/TPOT/E2E 完全相同；先去掉不提供判别力的重复维度。
+- **第一阶段：** CPU-only 重放原 A1 commands，只打开 `frontier_stage_batch_ledger` 与 individual batch metrics；验证 request metrics 与原 A1 bitwise/score 等价。真机先复用 3 次 frozen server logs 的 10 秒 Running/Waiting/KV samples，明确标为 coarse proxy，不冒充 per-iteration batch。
+- **第二阶段触发条件：** coarse proxy 不足以判断或 state mismatch 显著时，短窗口重跑真机并采集 per-iteration `decode_batch_size/context_length_hist/cudagraph bucket`；否则进入 matched-state whole-decode-step。
+- **Metrics：** decode batch/token distribution、prefill fraction、scheduler steps/s、graph bucket/padding、queue/KV proxy；configuration contrast `TP8-TP4`；stage measured-substitution 后的 winner。
+
+## 预期产物与 review
+
+- **预期数据：** 两个 Frontier full-ledger replays；三次真机日志的 coarse state summary；state overlap/reweighting verdict；必要时的 short-window iteration telemetry。
+- **Figure prototype：** `../../runs/frontier-fidelity-envelope-v1/qwen235-state-matched-diagnosis-mock.png`。左图对比 real/sim state；右图展示 H1 与 H2 下 matched-state contrast 的可区分方向。全部数值标为 schematic/mock。
+- **人工 review：** 已批准（用户在分析方案后要求“推进”）。
+- **Review 意见：** 先做最便宜的 state audit，不直接启动完整 Nsight sweep；每一步只在能改变下一决策时升级证据成本。
+
+## 复现信息
+
+- **Code：** AITuner `feature/sim`；运行 commit 待冻结。Frontier `6e8e0d845bceff11b0b62cb29df3a1a93411fdd4`。
+- **Environment：** dash0；Frontier replay CPU-only；后续真机才使用 4/8×H20。
+- **输入：** `/home/admin/cpfs/wjh/aituner/qwen235-collective-profile-ablation-20260719-r1/sim/fixed-pd` 与 frozen real campaign `/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/real/fixed-pd`。
+- **产物路径：** `/home/admin/cpfs/wjh/aituner/qwen235-fixed-pd-state-diagnosis-20260719-r1`。
+- **已知 deviation：** frozen real logs 的 Running 指标是 10 秒采样的 active-request proxy，不是 scheduler iteration ledger；不能单独支持 matched-state causal claim。
+
+## 结果
+
+- **观察事实：** 待运行。
+- **异常：** 待运行。
+- **含义：** 待运行。
+- **Claim update：** unchanged
+- **下一步：** 运行两项 CPU state replay 与 coarse real-state analysis。

.research/experiments/qwen235-tp8-collective-profile-ablation-20260719.md

@@ -0,0 +1,65 @@
+# 实验 EXP-SIMFID-Q235-CC-TP8：真实 TP4/TP8 collective profile 消融
+
+> **状态：** review 通过，执行中
+>
+> 本 card 是 SHA、command、config、log 等 provenance 的唯一归宿；本轮只重跑 simulator，不重跑已经冻结的 48 个真机 trial。
+
+## Claim 与决策
+
+- **Parent claim：** Qwen235 Fixed-PD 的 30%+ TPOT/E2E selection regret，是否主要由 TP8 collective profile 缺失及 TP4 profile 与真实 serving backend 不匹配造成。
+- **目的：** 支持或反驳 mechanism hypothesis；不是用同一 workload 的 E2E calibration 修正 simulator。
+- **Competing hypotheses：**
+  - H1：collective profile coverage/backend mismatch 是排序反转的必要主因。换成与真机 serving 一致的 TP4/TP8 实测 profile 后，Frontier 的 Fixed-PD TPOT winner 从 TP8 翻到 TP4，mean/p90 TPOT 与 E2E selection regret 降到 10% 以内。
+  - H2：collective mismatch 只解释部分误差。换 profile 后 TP8 仍是 Frontier winner，Fixed-PD TPOT/E2E regret 仍超过 10%；下一主因应定位 decode batch/state-conditioned MoE composition。
+- **事前预测：** 当前 Frontier 在 Fixed-PD 上预测 TP4/TP8 mean TPOT 为 87.77/61.59 ms，TP8 有 26.19 ms 优势；真实 TP4/TP8 为 21.04/27.99 ms。若新的 TP4/TP8 collective profile 使这个 26.19 ms 的 simulator margin 反转，则支持 H1；若不能，则支持 H2。
+- **判定规则：** 只以 frozen simulator rerun 的 winner 与真实 frozen surface 计算 selection regret。绝对 latency ratio 作为 secondary metric，不用它替代 selection verdict。
+
+## Setup
+
+- **自变量：**
+  - A0：当前 `measured-allreduce.csv`；TP4 是 Qwen30 hidden=2048 的旧实测，且 profiler 只检查 FlashInfer 可用、没有证明每个 payload 的实际 dispatch；TP8 无行并静默 analytical fallback。
+  - A1：Qwen235 serving-matched piecewise collective profile；TP4/TP8 都在 dash0 H20、vLLM 0.20.0 commit `88d34c640...` 上实测。Frozen server logs 证明真机同时使用 `disable_custom_all_reduce=true` 与 FlashInfer-TRTLLM `allreduce_rms` fusion；profile 对 fusion-eligible payload 测同一 FlashInfer communicator，对阈值外 payload 测真实 PyNCCL/symmetric fallback。
+- **控制变量：** Frontier commit、Qwen235 operator profiles、runtime contract、四类 frozen traces、候选配置、MNS/MBT、prefix policy、real results 与分析脚本全部不变。
+- **System context：** Qwen3-235B-A22B-FP8；vLLM 0.20.0+cu129；dash0 8×H20；`{TP4/EP1, TP8/EP8} × MNS{64,128}`；MBT=8192；Frontier piecewise graph path。
+- **Workload 或 trace：** 重跑四类 simulator surface：Fixed-PD 4096→256 @ 0.2 req/s/GPU、Fixed-PO 4096→1、Trace-PD、Trace-PO；每 cell 沿用原 129-request trace。Fixed-PD 是 primary，另外三类检查 profile 替换是否引入新的 selection regression。
+- **Profile protocol：** payload 覆盖所有 Qwen235 decode graph buckets（1--256，含真实 capture sizes）、fusion 阈值两侧 `{63,64,65}` / `{255,256,257}`，以及 512--8192 prefill sizes；每个 TP 与 payload 先 warmup，再保留 3×20 个 per-rank CUDA-event samples。raw JSON 记录实际 backend dispatch、fusion byte limit、world size、dtype、payload bytes、GPU/runtime/commit 与 source hashes。TP4/TP8 使用相同 payload grid；不把 microbenchmark 直接当作 E2E 结论。
+- **Profile contract：** H20/SM90 上 vLLM 0.20 的 fusion limit 是 TP4 2 MiB、TP8 0.5 MiB，即 Q235 BF16 hidden=4096 时分别为 256/64 tokens。simulator runner 启动前解析 CSV，要求所选 configs 的每个 `TP>1` 都有有限、正值的 measured rows；缺覆盖立即失败。结果 manifest 写入 CSV SHA-256、TP coverage、row counts 与 piecewise backend 集合。决策实验禁止 analytical fallback。
+- **Baselines：** A0 current Frontier、A1 measured-profile Frontier、frozen real hardware surface。
+- **Metrics：** profile latency median/p90 与跨 rank spread；simulated mean/p90 TTFT/TPOT/E2E；winner、selection regret、tau-b（可定义时）；每个 TP 的 measured-profile hit/fallback counters。
+
+## 预期产物与 review
+
+- **预期数据：** TP4/TP8 raw collective JSON；materialized Frontier CSV + manifest；四类 A1 simulator surface；A0/A1/real comparison JSON/Markdown；profile cost ledger。
+- **Figure prototype：** `../../runs/frontier-fidelity-envelope-v1/qwen235-collective-ablation-mock.png`；左图固定真实与 A0 TPOT，并为 A1 留待测 series；右图明确“winner flip→0% regret / unchanged→33% regret”的判定。它回答 profile 修复是否足以改变配置选择。
+- **人工 review：** 通过（2026-07-19，用户明确要求“推进实验”）。
+- **Review 意见：** 保留 frozen real surface，只补真实 TP4/TP8 profile 后重跑 simulator；每个 simulator 实验必须使用真实 profile，缺失 coverage 或运行时 analytical fallback 立即失败。
+
+## Benchmark design audit（experiment-design-review）
+
+| Crime | Verdict | Severity | Evidence | Fix / gate |
+|---|---|---|---|---|
+| 用 microbenchmark 代替 E2E | PASS | — | collective profile 只作为自变量；结论来自完整 simulator surface 对 frozen real surface 的 selection regret | 保留 A0/A1/real 三方结果 |
+| calibration set 等于 evaluation set | PASS | — | A1 只测 collective operator，不使用 real E2E latency 拟合参数 | 禁止 E2E scale/calibration |
+| selective benchmarking | PASS | — | primary Fixed-PD 外，同时重跑另外三类 workload | 报告所有 16 个 simulator cells |
+| 缺失平台/版本 | PASS | — | raw/manifest 绑定 H20、vLLM commit、model、backend 与 hashes | 任一 provenance 缺失则 profile 不可采纳 |
+| 缺失方差 | NEEDS EVIDENCE | Major | 尚未执行 profile repeats | raw artifact 必须保留 per-rank repeated samples并报告 spread |
+| profile 覆盖静默降级 | FAIL（A0） | Blocking | TP8 无 measured rows，Frontier 使用 analytical fallback | A1 runner fail-fast；fallback count 必须为 0 |
+| backend/fusion 阈值未对齐 | FAIL（A0） | Blocking | 真机日志启用 FlashInfer `allreduce_rms`；vLLM 源码规定 H20 TP4/TP8 fusion limit 为 2/0.5 MiB，旧 CSV 未记录这条 piecewise contract | A1 在阈值两侧实测并记录每行 dispatch |
+
+**总体建议：** 已批准执行；coverage gate、backend match 与 provenance gate 任一不通过则 Block。
+
+## 复现信息
+
+- **Code：** AITuner branch `feature/sim`；本 card 创建时 HEAD `f4a75aa8e400ead4eb6d305178192e85940c6de6`，后续运行 commit 待填。vLLM source commit `88d34c6409e9fb3c7b8ca0c04756f061d2099eb1`。
+- **Environment：** dash0 8×H20；`/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1`；model `/home/admin/cpfs/wjh/models/Qwen/Qwen3-235B-A22B-FP8`。
+- **产物路径：** 待 review 后冻结；不得覆盖旧 campaign `/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1`。
+- **已知 deviation：** `--disable-custom-all-reduce` 只关闭 vLLM custom AR，不关闭编译器的 FlashInfer `allreduce_rms` fusion。真机 TP4/TP8 日志都显示自动选择 `trtllm` workspace；旧 TP4 profiler 没有记录/执行真实 fusion-limit piecewise dispatch。A1 因此必须同时重测 TP4 与 TP8，不能只追加 TP8 行。
+- **执行异常：** 首次 simulator launch 误把 frozen operator profile root 写成 r1；原 A0 campaign 实际使用 r2。四个 Fixed-PD cells 因缺少 `attn_decode_in_mixed` predictor 均 fail-fast，未产生可用 metric。command diff 确认后停止后续运行，恢复 r2 并从 failed cells 重新执行；这些失败不计入 A1 surface。
+
+## 结果
+
+- **观察事实：** 待运行。
+- **异常：** 待运行。
+- **含义：** 待运行。
+- **Claim update：** unchanged
+- **下一步：** 依次完成 collective profiling、profile materialization、CPU simulator rerun 与 analysis。

.research/experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md

@@ -0,0 +1,46 @@
+# 实验：Qwen30 Fixed-PD TTFT admission diagnosis
+
+> **状态：** 已完成
+>
+> 用户要求分析 Frontier 在 Qwen30 Fixed-PD 高压 case 上 56--58% TTFT
+> selection regret 的根因，并给出简洁结论。
+
+## Claim 与决策
+
+- **Parent claim：** Frontier 在 capacity knee 附近的配置排序是否会因 state transition error 失效。
+- **目的：** 区分 TP2/TP4 conditional prefill-time 错误、mixed-step composition 错误与 admission queue feedback。
+- **Competing hypotheses：** H1：Frontier 把 TP4 prefill execution 相对 TP2 算慢；H2：decode service time 的绝对误差使 `arrival_rate × residence_time` 越过 MNS cap，首次调度等待被阈值放大；H3：即使固定 admission state，mixed prefill/decode composition 仍反序。
+- **事前预测：** H1 下去掉 queue 后 TP2 仍有更低 prefill time；H2 下去掉 queue 后 TP4 恢复更快，且只有 simulator 的 required concurrency 超过 MNS；H3 下 state-matched stage contrast 仍支持 TP2。
+- **判定规则：** 只有 state ledger/scorer 等价、queue counterfactual 和真机 Running/Waiting 同时支持时才归因 H2；否则保留 H1/H3 并补最小 telemetry。
+
+## Setup
+
+- **自变量：** config 为 Frontier winner `TP2/MNS64`、real mean winner `TP4/MNS32` 与 real p90 winner `TP4/MNS64`。
+- **控制变量：** Qwen3-30B-A3B BF16、community vLLM 0.20、H20、Fixed-PD 4096→256、1.125 req/s/GPU、MBT8192、piecewise graph、原 measured profiles/collectives 和原 257-request traces全部冻结。
+- **Workload：** uniform open-loop arrival；global rate 随 TP 为 2.25/4.5 req/s；prefix cache off；每个真机 cell 三次 fresh-server。
+- **Baselines：** 完整 12-cell frozen real/sim surface；真机三轮 pooled metrics。
+- **Metrics：** TTFT=`first scheduling delay + prefill execution`；request execution/residence time；`arrival_rate × service_time` 相对 MNS；Running/Waiting；stage ledger composition。
+
+## 预期产物与 review
+
+- **预期数据：** 三个 scorer-equivalent Frontier state replays；service/admission decomposition；H1--H3 verdict。
+- **Figure prototype：** `../../runs/frontier-fidelity-envelope-v1/qwen30-fixed-pd-ttft-admission-mock.png`；左图区分 TTFT execution 与 queue，右图显示 required concurrency 是否跨越 MNS。
+- **人工 review：** 已批准（用户要求直接分析清楚该 case）。
+- **Review 意见：** 先复用 existing artifacts 和 CPU replay；只有现有真机 periodic queue proxy 不足时才增加 GPU telemetry。
+
+## 复现信息
+
+- **Code：** AITuner `2970f74d`；Frontier `deadc4a321f0baaa534c6ebd17f974123733cdc2`。
+- **Environment：** dash0；Frontier replay CPU-only，GPU visibility disabled。
+- **输入：** `/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1`。
+- **产物路径：** `/home/admin/cpfs/wjh/aituner/qwen30-fixed-pd-ttft-diagnosis-20260719-r1`。
+- **已知 deviation：** 原真机日志只有 10 秒 periodic Running/Waiting，没有 per-iteration ledger；它可验证 steady queue 是否积压，但不用于细粒度 stage timing。
+
+## 结果
+
+- **观察事实：** Frontier 把 TP2/MNS64 与 TP4/MNS64 的 TPOT 分别高估 8.09× 与 5.61×。按 frozen request execution time 计算，TP2 需要 58.0 个并发槽，未超过 MNS64；TP4 需要 79.8 个，超过 MNS64。真机 TP4 的 E2E-based required-slot upper bound 只有 14.7，MNS16/32/64 三组 periodic logs 的 Waiting max 均为 0。
+- **关键反事实：** observed Frontier 中 `TP4/MNS32 - TP2/MNS64` TTFT 为 `+27087.0 ms`；减去每请求的 first-scheduling delay 后变为 `-55.4 ms`，与真机 `-71.4 ms` 同方向。TP4/MNS32 的 27.3 秒 simulated TTFT 中，27.18 秒来自首次调度前等待，而不是 prefill execution。
+- **异常：** 计划中的 state-ledger replay 会重新训练 frozen no-cache predictors；8 分钟后仍停留在 predictor training。由于原 request metrics 已精确提供 TTFT=first-scheduling wait+prefill execution，且 MNS sweep 已构成 controlled intervention，继续 ledger 不改变判定，故主动停止；partial output 保留在产物根目录但不进入结果。
+- **含义：** H2 supported；H1/H3 对“TP topology 排序从哪里被反转”均 rejected。Frontier 的无排队 prefill 仍预测 TP4 比 TP2 快，错误由 decode service-time 绝对高估使 TP4 独自跨过 admission cap，再经 queue feedback 放大产生。该反事实恢复的是 TP4 topology 方向，不声称恢复 exact MNS winner；现有真机日志也不能把最初的 service-time overprediction 继续归因到某个单独 operator。
+- **Claim update：** supported
+- **下一步：** 若研究问题升级为“为什么 TPOT 绝对值高估 4--8×”，需增加真机 stage timing/overlap 证据；它不是解释本次 TTFT selection reversal 所必需。

.research/ongoing.md

@@ -0,0 +1,79 @@
+# AITuner 研究当前状态
+
+> 2026-07-17｜写给未参与项目的读者，可直接作为 presentation 讲稿。历史过程与复现信息见 `../runs/*/` 各 experiment card、`../docs/` 各 campaign 文档。
+>
+> **2026-07-19 update：** Qwen235 Fixed-PD 的错误排序在 exact real state composition 下已经翻正，主因是 simulator closed-loop batch state，而不是 collective。Qwen30 Fixed-PD 的 56--58% TTFT regret 也已定位：Frontier 将 decode service time 高估 4--8×，使 TP4 的 modeled concurrency 越过 MNS admission cap并产生虚假排队；去掉该等待后 Frontier 与真机都判定 TP4 topology 更快。详见 [`experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md`](experiments/qwen30-fixed-pd-ttft-admission-diagnosis-20260719.md)。
+>
+> **2026-07-20 update：** 对全部 14 个 frozen case surface（70 个 case×objective）做了统一的 margin-vs-residual 分解与方向化机制普查（[`experiments/frontier-split-rootcause-s0-20260720.md`](experiments/frontier-split-rootcause-s0-20260720.md)）。三个要点：(1) 「residual 超过 margin」是失败的必要条件但远非充分——good/bad 分裂不能用无方向误差量解释；(2) 23 个 material failure 的 winner-deciding pair 中 21 个落在 TP 轴或 mixed（其余 2 个是 6.2% regret 的边缘 mns-axis case），trace 面的 TP 反序为零；A1 measured collective 把 Qwen235 两个 prefill-only 面的 TP 反序清零（trace-PO p90 regret 21.2%→0.3%）却对 Fixed-PD 完全无效——prefill 路径的 TP 差异化误差源是 collective profile（可修），decode 耦合的 TP 差异化误差是当前所有 material failure 的载体；(3) 「Fixed-PD 失败因为高压」被否证：失败 Fixed-PD 的真机 in-flight（14.05）低于全对的 Trace-PD（38.69），且低压 Fixed-PD 同样失败、失败 objective 随负载切换。另有次要缺陷：14 个 winner 错位来自 simulator 对 MNS 逐位不敏感的精确 tie。
+>
+> **2026-07-20 root-cause update：** Q30 低压 Fixed-PD 的 exact stage ledger 关闭了最后一个未解释的 material failure。相同 batch=1 state 下 Frontier full predictor 给 TP4 `18.3515 ms/step`、TP1 `19.3506 ms/step`，方向正确；但 per-GPU 固定到达率使 cluster arrival 随 TP 增长，叠加 decode residence 高估后，TP4 在 simulator 内自激到 time-weighted batch `3.0437`（96.13% decode 时间 batch≥3），own-state step 变为 `28.1712 ms`。其中相对 batch=1 的 `+9.8197 ms` 有 `+8.9297 ms` 来自 batch-conditioned MoE，collective 仅 `+0.0121 ms`。因此 Fixed-PD 的根因不是“固定 workload”或“高压力”本身，而是 **execution-time residual 进入离散事件时钟后改变 future scheduler state；该 state 再通过 MoE/profile/graph 或 MNS admission 非线性放大，形成 action-dependent signed residual 并穿过 decision margin**。Q30 低压是平滑 state-feedback，Q30 高压是跨 MNS cap 的 threshold amplification，Q235 是 composition drift；三者为同一闭环机制族。
+>
+> **2026-07-20 load-audit update：** Trace-PD overload 不是 Fixed/Trace good-bad 分裂的统一解释。旧 Q30 Trace-PD decode offered/observed-peak throughput≈`1.00×`、peak Running/Waiting=`47/0`；降到 `0.10 req/s/GPU` 后 TTFT `245.95/685.51 → 228.14/835.38 ms`（mean/p90，不出现 tail collapse），TPOT `13.18/15.39 → 7.91/8.90 ms`。旧 Q235 则是 `3.44×` 明确过载、peak=`116/3`；降到 `0.035 req/s/GPU` 后 TTFT `1141.54/2616.69 → 478.14/1347.75 ms`，TPOT `61.89/78.62 → 24.00/28.49 ms`。旧 surface 仍有 `417×/32.6×` mean-TTFT spread，否定“所有配置一样差”。八 case baseline 与 claim boundary 见 [`experiments/frontier-eightcase-load-audit-20260720.md`](experiments/frontier-eightcase-load-audit-20260720.md)。
+
+## 一眼看懂
+
+- **Topic / problem：** LLM serving 的自动、低成本配置调优（AITuner）。当前主线问题：用 simulator 给部署配置（并行度、批量上限等）排序，什么时候可信？需要补多少真机证据？算上这些成本还划算吗？
+- **Central claim：** simulator 要能帮助配置调优，必须先满足 scheduler transition 的 liveness/coverage，再满足「配置相关残差小于真机 decision margin」；前者决定 capacity 是否有定义，后者决定排序是否正确。(ID: C0)
+- **当前结论：** 早先 35 个 trace stall 不是 Frontier scheduler liveness failure：adapter 为不满 16-token 的 prefix block 错误生成了 cache identity，Frontier 又没有 fail-fast。修正为完整 block、使用真实 graph buckets/KV blocks 和 `piecewise`/`KERNEL_ONLY` profile 后，Qwen30 Trace-PD 的全部 12 个 cell 完成 129/129 request，Frontier 对 TTFT/TPOT/E2E 的 6 个 argmin 均与三次 fresh-server 真机一致；但绝对 latency 仍高估 4--511×。这只证明一个 MoE Trace-PD surface 的 selection fidelity，不能外推到 prefill-only、fixed workload 或 235B。
+- **最大 uncertainty / risk：** 根因已收敛，且 overload 已被排除为统一解释，但可信域边界仍未画清：trace 面的 heterogeneity 是否让 closed-loop state residual 变小，还是当前 success 主要由 capacity/MNS margin 保护？两个降载点只建立 reference-config latency baseline，不能证明新负载下全 surface 仍选对。
+- **下一项 critical action：** 不再做无锚点的 jitter 猜测；保持 request shape 不变，在预测的 MoE/MNS knee 两侧做小规模 rate sweep，并用少量真机 state/batch anchor 验证 `λR(B)` fixed point。成功标准是同时预测 state-regime、排名与 knee，而不只是某个点的 regret。
+- **停止条件：** T1 出 verdict 且成本账本建立后：pass 且摊销论证成立 → 转向「sim 剪枝 + 真机终选」的 hybrid 机制设计；fail → 转入失败机制归因；两条路都无 insight 增量 → 收敛写作。
+
+## 核心概念
+
+- **Frontier：** 本项目使用的 simulator，属 Vidur 系（直接使用 vidur backend），加自研 FP8/MoE/EP/decode-profile 兼容补丁。
+- **Regret：** 按 simulator 排序选配置，相对真机最优配置的性能损失百分比（以每 GPU capacity 计）。primary metric：排序选对则 regret=0。
+- **τ-b（Kendall tau-b）：** simulator 排序与真机排序的秩相关，1 = 完全一致，−1 = 完全反序，0 = 无关；tie-aware。
+- **Decision margin：** 真机上头部配置之间的性能差距，即 simulator 误差的容忍带。
+- **Action-differential residual：** simulator 误差中随配置（action）不同而不同的部分。Why needed：所有配置统一偏移不影响排序，只有差异化残差才可能穿过 margin 改变选择——这解释了「绝对误差 33%」与「排序全对」为何可以同时成立。
+- **Capacity bracket：** 真机 anchor 为候选配置的 capacity 划出的上下界；「bracket 不反转」指未测的负载点不可能推翻 top 选择。
+- **Decision-valid coverage：** simulator 能从初始状态推进到所有请求完成，并为 config×workload cell 产生合法 SLO metric 的比例。若 reachable nonterminal state 没有 enabled transition/future event，capacity 与 rank 都没有定义，不能把该 cell 当作 infeasible。
+- **Workload realism 阶梯：** prefill-only（无 decode）→ fixed-shape mixed（固定输入输出长度的混合负载）→ trace-faithful mixed（生产 trace 忠实回放）。fidelity 结论不能向更高一级外推。
+
+## Claim 层级
+
+- **Central claim：** 见「一眼看懂」。(ID: C0)
+  - **Subclaim：** zero-shot 排序失败是真实现象。(ID: C1；supported)
+    - 30B 纯 profile 驱动的 regret 为 25.63%（τ-b=0）；另一 throughput-proxy 评测口径下为 30.46%。Boundary：均发生在 capacity-point + SLO-gated selection——恰是 Vidur 论文自己声明预测误差会爆炸、评测刻意回避的 regime（见 claim map）。
+  - **Subclaim：** 少量结构化的真机证据可以恢复低 regret 排序。(ID: C2)
+    - **Hypothesis（decision-bearing）：** trace-faithful 回放下，同栈 profile + 真机 KV capacity + 兼容补丁、且不做逐案例端到端校准的 Frontier，能满足 gate：regret ≤5% ∧ τ-b ≥0.8 ∧ bracket 不反转。(ID: H1；weakened)
+      - **Supporting：** 235B prefill-only regret=0；235B fixed-shape mixed 的 top set 全中；30B 加 per-TP 校准后 regret 0.76%（但这是外部端到端 scale 给出的上界，不是原生 profile 保真度）。
+      - **Counterevidence：** 修正 prefix trace contract 后的 TP2/MNS16 `none`-graph run 完成但 p50 TPOT 约 96 ms，真机为约 14 ms；然而该比较尚未对齐 real vLLM 的 `FULL_AND_PIECEWISE` graph path。
+      - **下一项 discriminative experiment：** 补齐 `KERNEL_ONLY` graph family并以 `piecewise` 重跑相同 trace；若 full surface 仍错，graph omission 不再是可用解释。
+    - **Hypothesis（机制，active）：** 误差机制是 action-conditioned residual——执行状态的转移（并行拓扑、kernel family、graph mode、batch 组成）使按算子 profile 的组合预测跨配置不可复合；残差大于 margin 时排序失败。(ID: H2；supported，已细化)
+      - **Supporting：** 三个 TP 档的端到端校准系数为 0.72/0.47/0.35，残差确实随配置剧烈变化；235B 的批量上限交互预测错误但被 2× margin 容忍；30B prefill-only 在低负载近似对齐、饱和后按 TP 反向放大，最终 τ-b=−1。
+      - **细化（2026-07-20 统一普查）：** 决策相关的残差分量集中在 TP 轴且由 decode 状态耦合产生——prefill-only 面的绝对 scale 仅 0.96–1.37× 且 measured collective 即可清除其 TP 反序，而含 decode 的面 scale 4.3–130×、全部 material failure 都由 TP/mixed pair 决定。「residual>margin」只是必要条件；失败还需要残差对准 winner-deciding pair。
+      - **机制 verdict（2026-07-20）：** closed-loop state drift 是根因，离散阈值是其放大器而非 competing explanation。Q30 低压 exact ledger 显示同 state 的 TP 方向正确，但 TP4 被模拟 residence 反馈推到 batch 3--4，MoE step 增长后反序；Q30 高压进一步跨过 MNS admission cap；Q235 换成 exact real composition 后排序翻正。下一步从“找根因”转为测量 state-regime/knee 的可信边界。
+  - **Subclaim：** 成本论证只有在摊销前提下成立。(ID: C3)
+    - **Hypothesis（active）：** 每个 model×硬件×runtime 的一次性对齐成本，摊销到大配置面、频繁重调（引擎版本 churn 的频率证据见 claim map）或禁止在线实验的场景后，低于重复真机调优。(ID: H3；untested——分母已实测，分子未入账)
+      - **下一步：** 建 cost ledger（见「下一步」）。
+
+## 当前 critical experiment
+
+- **Question：** 生产 trace 忠实回放（prefix 打开、原始到达时间与会话结构）下，best-effort Frontier 能否满足 low-regret gate？
+- **为什么现在做：** 这是 H1 的判决实验；所有已完成的机制分解都在人工 workload 上，不能替代这个 verdict。
+- **当前状态：** Trace-PD 的 graph-aligned surface 已通过原负载 selection gate，但绝对 latency 不通过 calibration；Fixed-PD 的 failure 已定位为 closed-loop state drift。两个降载 Trace-PD anchor 已通过完成率/admission/backlog gate，下一步需要 full surface rate sweep 才能检验 ranking 是否跨 load regime 保持。
+- **Result → decision：** 若其它 surface 排序失败，保留 Trace-PD success 为条件化 envelope，并按 fixed/trace/prefill/decode 的差异定位 state composition；若都通过，才扩大到 Q235 或寻找 simulator 已解决范围之外的新问题。
+- **Experiment card：** [`../runs/frontier-fidelity-envelope-v1/experiment-card.md`](../runs/frontier-fidelity-envelope-v1/experiment-card.md)
+
+## Key evidence（最多 3 条）
+
+- **E1（否证「prefill-only 是充分 easy condition」，支持 H2）：** 30B BF16、去掉 decode/prefix/混合 batch 后，真机最优是 TP4（8 vs 7 req/s/GPU），simulator 却把 TP4 排最差（6 vs 8）：top set 无交集，regret 12.5%，τ-b=−1。产物：`../runs/frontier-phase-factorial-v0/results/final/`（dash0，12.07 H20-GPUh）。
+- **E2（统一机制普查：material failure 全部由 decode 耦合的 TP 差异化误差决定，支持 H2 细化）：** 对 14 个 frozen surface、70 个 case×objective 的方向化分解显示：23 个 material failure 中 21 个由 TP/mixed pair 决定（仅 2 个 6.2% 边缘 mns-axis case）、trace 面 TP 反序为零；measured collective（A1）把 Qwen235 两个 prefill-only 面的 TP 反序清零（trace-PO p90 regret 21.2%→0.3%）但对 Fixed-PD 的 33% 无效；「residual>margin」仅为失败的必要条件。产物：[`../runs/frontier-split-rootcause-v0/results/`](../runs/frontier-split-rootcause-v0/results/decomposition.md)（实验 card：[`experiments/frontier-split-rootcause-s0-20260720.md`](experiments/frontier-split-rootcause-s0-20260720.md)）。
+- **E3（closed-loop state 是 Fixed-PD 根因，而非同 state predictor 反序）：** Q30 低压相同 batch=1 state 下 TP4 比 TP1 快约 1.00 ms/step，但 TP4 own state 的 time-weighted batch=3.0437，使 step 增加 9.8197 ms（其中 MoE +8.9297 ms）并反序；Q235 用 exact real composition 重放也把 TP8−TP4 从错向 −20.07 ms 翻为正确 +10.90 ms。Q30 高压再由 MNS cap 将同族 state/residence 误差放大成约 27 s 排队。产物：[`experiments/frontier-split-rootcause-s1-20260720.md`](experiments/frontier-split-rootcause-s1-20260720.md)。
+
+## 下一步（最多 3 项）
+
+- [ ] **画可信域边界：** 固定 request shape，在预测的 MoE/MNS knee 两侧做最小 rate sweep；只在判别点补真机 batch/state anchor，验证 `B≈min(MNS, λR(B))` 是否同时解释 state 与 ranking。
+- [ ] **Q235 portability gate：** 先验证 vLLM0.20 TP4/TP8 FP8 runtime 和 deadc4a profile provenance，再决定是否允许其 Fixed-P sweep。
+- [ ] **建 cost ledger：** parent H3；完成标准 = 每 case 一行（profiling GPU-h、补丁工时、校准探测、sim CPU-h），与已实测的真机调优成本同表，随每个 case 更新。
+
+## Blocker 或 anomaly
+
+- **当前运行状态：** 八 case load audit 的新增真机 run 已完成；未启动 full-surface rate sweep，避免把两个 single-config anchor 外推成 ranking claim。自 2026-07-20 起，本任务只允许使用 `dash1`--`dash4`（每台 8×H20、最多四组并行）；`dash0` 保留给其他同事，不做 probe、同步或运行。
+- **Anomaly（保留）：** 235B pilot 中 simulator 把 10/34 个 anchor 误判为不可行——false-infeasible 是 H1 的主要威胁模式，T1 分析时须单独报告。
+- **平台边界（已更新）：** 历史结果仍来自其各自 card 记录的平台，不改写 provenance；后续实验平台切换为 `dash1`--`dash4`。跨主机比较前必须跑相同 canary 并量化 host effect。fixed-shape pilot 的主 SLO（TPOT 40ms）无判别力，150ms 是事后明示的敏感性分析，不得写成盲选的 primary。
+
+## Related work
+
+- Claim map：[`../docs/simulator-claim-map-20260716.md`](../docs/simulator-claim-map-20260716.md)。核心缺口：capacity-point + SLO-gated selection 的 regret 无人用真机 ground-truth 面验证过；alignment 成本无人与真机调优成本放进同一张表比较。

AGENTS.md

@@ -1,10 +1,11 @@
 # Project Operating Notes
 
-## Remote experiment host
+## Remote experiment hosts
 
-- Default experiment machine: `dash0`.
-- Hardware expectation: 8 NVIDIA H20 GPUs.
-- SSH check: use `ssh dash0` before scheduling or debugging remote runs.
+- Experiment machines: `dash1`, `dash2`, `dash3`, and `dash4`.
+- Do not use or probe `dash0`; it is reserved for other users.
+- Hardware expectation: 8 NVIDIA H20 GPUs per host.
+- Before scheduling, probe only `dash1`--`dash4` and confirm all eight GPUs are idle and healthy.
 - Remote project path: `/home/admin/cpfs/wjh/aituner/aituner`.
 - If remote downloads are slow or fail, start the proxy from the remote `wjh`
   home directory with `./auto_proxy.sh`, then run downloads in a shell where
@@ -13,7 +14,8 @@
 
 ## Local/remote sync workflow
 
-- Treat this local repository and the `dash0` repository as the same project checkout.
+- Treat this local repository and the `dash1`--`dash4` repositories as the same project checkout.
 - Synchronize code through Git using `commit`, `push`, and `pull`.
-- For remote experiments, commit local changes, push to `origin`, then pull on `dash0` in `/home/admin/cpfs/wjh/aituner/aituner` before running.
+- For remote experiments, commit local changes, push to `origin`, then pull on each assigned host in `/home/admin/cpfs/wjh/aituner/aituner` before running.
+- Up to four independent 8-GPU experiment groups may run in parallel, one group per host; pin every job explicitly to one of `dash1`--`dash4`.
 - Do not ask for the remote host or project path again unless the user explicitly changes them.

docs/simulator-claim-map-20260716.md

@@ -0,0 +1,75 @@
+# Simulator-for-config-tuning related-work claim map
+
+日期：2026-07-16。目的：为「Frontier/Vidur-class simulator 能否低成本解决 config tuning」这条主线建立 related-work 边界。Vidur 与 LLMServingSim 的条目基于原文（PDF 全文核读）；SimAI 基于论文页与摘要口径。每项按 Context / Claim / Assumption / Mechanism / Evidence / Boundary / 与本 project 的关系提取。
+
+## Vidur（MLSys 2024，arXiv:2405.05465）
+
+| 维度 | 内容 |
+|---|---|
+| Context | MSR India。首个面向 LLM inference 的大规模模拟器。Motivation 与我们一致：config search 复杂度 O(\|M\|·\|T\|)，且 optimal config 是 (model, trace) 的函数——Fig 1b 显示跨 trace misconfiguration 代价最高 2×。 |
+| Claim | (a) request-level 预测误差 <9%：static trace 上 P95 normalized execution latency 误差 ≤3.33%（4 模型 × 3 trace），dynamic trace 在 **85% capacity** 负载下误差 <5%。(b) Vidur-Search 用约 1 小时 96-core CPU（$9.93/h）为 LLaMA2-70B 找到最优 config，对比 deployment-based exploration 估算 42K GPU-hours ≈ $218K。(c) what-if 全量探索 $125 模拟成本 vs 估算 $1.14M 真机成本。 |
+| Assumption | operator runtime 可由单 GPU profiling + 小型 ML 估计器（random forest）插值；prefill attention 可用等效单序列 sqrt(Σp_i²) 近似；decode attention runtime 只依赖总 KV 读量而非 per-request context 分布；LLM 架构同质（小算子集合跨模型共享）。 |
+| Mechanism | 声明式 model spec → 算子三分类（token-level / sequence-level / communication）→ 单 GPU CUPTI profiling → RF runtime estimator → event-driven simulator + 三层 hierarchical scheduler（支持 vLLM/Orca+/Sarathi-Serve/FasterTransformer/LightLLM 策略）→ Vidur-Search 对每个 config 二分搜索 max QPS（判据 P99 scheduling delay <5s），目标 QPS/dollar。 |
+| Evidence | LLaMA2-7B/70B、InternLM-20B、Qwen-72B（全 dense）；Azure A100/H100 4-GPU pairwise-NVLink 节点；Chat-1M / Arxiv-4K / BWB-4K trace，总长截断到 4096 tokens。 |
+| Boundary | **作者明示**：接近 capacity point 时小误差会因排队失控放大，故 fidelity 评测停在 85% capacity。**结构性**：无 MoE、无 FP8/量化、无 prefix-cache reuse（多轮对话按独立请求处理）、无 speculative decoding（列为 future work）、PP 仅同步、长上下文未覆盖（4K 截断）。metric 口径为 normalized execution latency（static 排除 scheduling delay）。**最关键**：sim 选出的 config 在真机 ground-truth surface 上的 selection regret 从未被验证；42K GPU-h/$218K 是反事实估算，分母是穷举式 exploration 而非 strong sequential tuner。 |
+| 与本 project 的关系 | Frontier 即 Vidur-class（代码直接使用 vidur backend）+ 我们的 FP8/MoE/EP/decode-profile patches。我们的所有实验恰好工作在 Vidur 声明误差爆炸并回避的 regime（capacity point + SLO gate），补的正是它缺的 selection-regret ground truth。我们的 zero-shot 失败（25–30% regret）与其 <9% 不矛盾——不同 metric、不同 load regime、不同 stack alignment，论文必须主动写明这一点。其 Fig 1b 的 workload-conditioned 结论与我们 P4 sign-flip、P6 churn 互为独立佐证 → 支持 retune 频率 / amortization 论证（C3）。 |
+
+## LLMServingSim（IISWC 2024，arXiv:2408.05499）
+
+| 维度 | 内容 |
+|---|---|
+| Context | KAIST。scale-out LLM serving 的 HW/SW co-simulation，面向 NPU/PIM/异构加速器设计探索，基于 ASTRA-sim。 |
+| Claim | 对真实 multi-GPU vLLM serving 平均误差 14.7% 且「趋势一致」；比 mNPUsim/GeneSys/NeuPIMs 快 34.7–491×（摘要口径 91.5×）。 |
+| Assumption | iteration-level 模拟 + decoder-block 冗余复用（编译一个 block 复制展开、attention/非 attention 分离）可在可行时间内保持足够精度；硬件行为可由可插拔 accelerator compiler+simulator 栈表达（GeneSys 原型）。 |
+| Mechanism | 逐 iteration：scheduler（iteration-level batching、KV paging、operator mapping）→ per-device 硬件模拟 → graph converter（Chakra）→ ASTRA-sim 网络级模拟 → 循环。 |
+| Evidence | 与 multi-GPU vLLM 真机对照，变量为 LLM 架构、并行方案、NPU 数量、异构度；报告平均误差与趋势一致性。 |
+| Boundary | 定位是硬件/系统设计空间探索，不是 engine-knob config tuning；validation 口径是 trend-following，无 SLO-gated capacity、无 selection-regret；14.7% 平均误差大于典型 config 间 capacity margin（我们 12-cell 面上 top-2 差距 0.76%），故该精度不足以支撑近邻 config 选择。 |
+| 与本 project 的关系 | 说明「模拟保 trend」是社区通行 validation 标准；「trend ≠ selection」这一缺口对它同样成立。不构成直接 baseline，但在 related work 中界定我们评测口径（selection regret at capacity point）的必要性。 |
+
+## SimAI（NSDI 2025，Alibaba，aliyun/SimAI）
+
+| 维度 | 内容 |
+|---|---|
+| Context | 大规模 LLM **training** 的架构设计与参数调优模拟，生产背景（Alibaba Cloud）。 |
+| Claim | 各测试场景平均 98.1% 与真实结果对齐；为 host 设计与参数设置提供生产可用 guidance。 |
+| Assumption | training 过程可由 framework + kernel computation + collective communication 的选择性高保真集成复现。 |
+| Mechanism | 高保真集成三层栈 + 多线程加速 + lock-free global context sharing。 |
+| Evidence | 与生产 training 场景对齐（论文口径，未逐一核读实验细节）。 |
+| Boundary | training-only。training iteration 均匀、batch 组成静态——恰是 Vidur 指出 inference 所缺的性质，因此 98.1% 不可外推到 serving capacity point。 |
+| 与本 project 的关系 | 「simulator 指导 infra 决策」的工业先例与动机背书；不与 serving config tuning claim 竞争。引用价值在 motivation，不在 evaluation 对照。 |
+
+## Frontier（本 project 被测对象，非 related work）
+
+内部 Vidur-class 实现（vidur backend）+ 本 project 的 FP8/MoE tuning-key、Qwen MoE serving plan、TP/EP-aware cache key、critical-lane、decode/true-mixed profile 补丁。我们全部 fidelity 结论限定于该实现与已声明的 patch 集；见 `simulator-fidelity.md`。
+
+## Consensus / disagreement / uncovered regime
+
+**Consensus（三方一致或与我们互证）：**
+
+1. operator/iteration profile + 调度复合的模拟器，在中低负载下能达到 5–15% latency 误差；模拟成本比真机低数个数量级。
+2. optimal config 是 (model, workload) 的函数，misconfiguration 代价可达 ~2×（Vidur Fig 1b；我们 P4 pattern sign-flip 与 P6 engine-churn 独立复证）。
+
+**Disagreement：** 无直接冲突数字。我们的 zero-shot 失败与 Vidur <9% 处于不同 metric/regime，论文需主动解释，防止被误读为矛盾或重复。
+
+**Uncovered regime（本 project 的空间）：**
+
+1. **capacity-point + SLO-gated selection regret 无人用真机 ground-truth surface 验证。** Vidur 自认该 regime 误差爆炸并把评测停在 85% load；LLMServingSim 只验 trend。而 config tuning 的决策恰好发生在 capacity point。
+2. MoE、FP8、prefix reuse、speculative decoding、EP topology、长上下文均在已发表 fidelity envelope 之外。
+3. **alignment/profiling 成本从不与真机 tuning 成本同表比较。** Vidur 的 $218K 对比用穷举做分母；正确分母是 strong sequential tuner（我们实测 0.27–0.45 H20h/task，`runs/tuning-cost/metrics.json`）。
+4. envelope 失效的低成本检测（workload/runtime/topology 变化后何时还能信 simulator）无人提出。
+
+## 对本 project claim 的直接影响
+
+- **C1 定位句**：不是「Vidur 错了」，而是「Vidur-class claim 停在 sub-capacity load 的 prediction fidelity；把它外推到 SLO-gated capacity selection 是社区的隐含用法，我们证明该外推在 zero-shot 下失败（25–30% regret），并给出恢复 ranking 所需的最小真机证据层级」。
+- **C2**：Vidur 没有 minimum-real-evidence 的概念（要么全模拟、要么全真机）；per-TP calibration / 同栈 profile + KV capacity + patches 的证据层级是新贡献面。
+- **C3**：省钱叙事必须从「数量级」修正为「仅在 amortization 下成立」，分母换成 strong tuner 实测值；Vidur Fig 1b + 我们 P6 churn 共同支撑 retune 频率前提。
+
+## 待 triage 的相邻工作（未读原文，暂不写 claim）
+
+APEX（arXiv:2411.17651，并行执行计划模拟）、LLMServingSim 2.0（arXiv:2602.23036，异构+分离式）、Charon（arXiv:2605.17164，training+inference 统一）、inference-fleet-sim（arXiv:2603.16054，排队论容量规划）、AgentServeSim（arXiv:2606.09613，多轮 agent serving）。若审稿风险评估需要，按本表格式各补一行。
+
+## Sources
+
+- Vidur: <https://arxiv.org/abs/2405.05465>（全文核读版本：mlsys24 PDF）
+- LLMServingSim: <https://arxiv.org/pdf/2408.05499>
+- SimAI: <https://www.usenix.org/conference/nsdi25/presentation/wang-xizheng-simai>、<https://github.com/aliyun/SimAI>

docs/simulator-tuning/README.md

@@ -0,0 +1,14 @@
+# Simulator tuning evaluation
+
+This directory contains decision-level summaries for experiments that compare
+a serving simulator's selected configuration with the best configuration on
+real hardware.
+
+Current report:
+
+- [Frontier selection regret on Qwen3-30B and Qwen3-235B](frontier-selection-regret-qwen30-qwen235-20260719.md)
+
+The primary quantity is **real-hardware selection regret**, not simulator
+absolute-latency error. Raw commands, profiles, traces, and experiment-specific
+audit records remain under `runs/` or in the immutable remote artifact roots
+listed by each report.

docs/simulator-tuning/frontier-selection-regret-qwen30-qwen235-20260719.md

@@ -0,0 +1,75 @@
+# Frontier selection regret: Qwen3-30B and Qwen3-235B
+
+> Date: 2026-07-19
+> Scope: H20, community vLLM 0.20, Frontier piecewise simulation, no SLO gate
+
+## Question and metric
+
+For each workload and latency objective, Frontier selects the configuration
+with the lowest simulated latency. We then look up that configuration on the
+complete real-hardware surface and compare it with the real-hardware optimum.
+
+```text
+selection regret = real_latency(Frontier winner) / real_latency(real winner) - 1
+```
+
+Lower is better. `0%` means Frontier selected the real winner. Positive values
+mean that following Frontier produces slower real serving. Each objective is
+selected independently; this table does not combine TTFT, TPOT, and E2E into a
+single score.
+
+## Qwen3-30B-A3B
+
+Configuration surface: `TP in {1,2,4} x MNS in {8,16,32,64}`, with
+`MBT=8192`. Each real cell uses three fresh-server trials.
+
+| Workload | TTFT mean | TTFT p90 | TPOT mean | TPOT p90 | E2E mean | E2E p90 |
+|---|---:|---:|---:|---:|---:|---:|
+| Trace-PD | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |
+| Fixed-PD, 4096->256, 1.125 req/s/GPU | **58.0%** | **56.2%** | 0.0% | 0.0% | 1.7% | 5.5% |
+| Trace-PO, OSL=1 | 3.2% | 0.4% | N/A | N/A | 3.2% | 0.3% |
+| Fixed-PO, 4096->1, 1.125 req/s/GPU | 0.3% | 0.5% | N/A | N/A | 0.3% | 0.5% |
+
+Interpretation: Frontier is near-optimal for Trace-PD and both prefill-only
+cases, but the high-pressure Fixed-PD TTFT choice is materially wrong: its
+selected configuration is 56--58% slower than the real TTFT optimum.
+
+## Qwen3-235B-A22B-FP8
+
+Configuration surface: `{TP4/EP1, TP8/EP8} x MNS in {64,128}`, with
+`MBT=8192`. Each workload has 129 requests per cell and each real cell uses
+three fresh-server trials.
+
+| Workload | TTFT mean | TTFT p90 | TPOT mean | TPOT p90 | E2E mean | E2E p90 |
+|---|---:|---:|---:|---:|---:|---:|
+| Trace-PD | 0.0% | 0.0% | 0.0% | 0.0% | 0.6% | 6.2% |
+| Fixed-PD, 4096->256, 0.2 req/s/GPU | 4.2% | 0.2% | **33.0%** | **37.2%** | **30.7%** | **34.6%** |
+| Trace-PO, OSL=1 | 7.0% | **21.2%** | N/A | N/A | 7.0% | **21.2%** |
+| Fixed-PO, 4096->1, 0.2 req/s/GPU | 5.9% | 1.7% | N/A | N/A | 5.9% | 1.7% |
+
+Interpretation: Trace-PD is mostly near-optimal. Fixed-PD reverses the real
+decode/E2E preference between the tested parallel configurations and incurs
+31--37% regret. Trace-PO also has a material p90 failure of 21.2%.
+
+## Decision
+
+The tested Frontier stack has **not** solved serving configuration tuning.
+Its selected configuration can be near-optimal for one workload and materially
+wrong for another on the same model and hardware. The strongest current
+counterexamples are Qwen3-30B Fixed-PD TTFT and Qwen3-235B Fixed-PD TPOT/E2E.
+
+This statement is limited to the two tested MoE models and Frontier. It is not
+yet evidence about dense models, Vidur/APEX as separately reproduced systems,
+other hardware, or SLO-constrained tuning.
+
+## Provenance
+
+Primary immutable analysis artifacts on `dash0`:
+
+- Qwen3-30B Trace-PD: `/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis/comparison.json`
+- Qwen3-30B Fixed-PD/PO: `/home/admin/cpfs/wjh/aituner/qwen30-fixed-pressure-surface-20260719-r1/analysis/`
+- Qwen3-30B Trace-PO: `/home/admin/cpfs/wjh/aituner/qwen30-latency-expansion-20260718-r2/analysis-r6/trace-po-comparison.json`
+- Qwen3-235B four-case matrix: `/home/admin/cpfs/wjh/aituner/qwen235-v020-fourcase-20260719-r1/analysis/comparison.json`
+
+The Qwen3-235B artifact root includes `provenance/artifacts.sha256`; the final
+matrix contains 48/48 valid real trials and 16/16 complete simulator cells.

infra/gpu_fleet/gpu_fleet.py

@@ -182,6 +182,59 @@ def rsync_pull(config: FleetConfig, host: HostSpec, remote_path: str, local_path
     run_local(argv, cwd=config.project_root, capture_output=True, check=True)
 
 
+def scp_push(config: FleetConfig, host: HostSpec) -> None:
+    ensure_remote_dir(config, host, host.sync_remote_path)
+    local_src = str(config.sync.local_path.resolve()) + "/."
+    remote_dst = f"{host.ssh_alias}:{host.sync_remote_path.rstrip('/')}/"
+    argv = [
+        "scp",
+        "-o",
+        "BatchMode=yes",
+        "-o",
+        f"ConnectTimeout={config.ssh_timeout_sec}",
+        "-r",
+        "-p",
+        local_src,
+        remote_dst,
+    ]
+    run_local(argv, cwd=config.project_root, capture_output=True, check=True)
+
+
+def scp_pull(config: FleetConfig, host: HostSpec, remote_path: str, local_path: Path) -> None:
+    remote_src = remote_path
+    if remote_path.endswith("/"):
+        ensure_dir(local_path)
+        remote_src = remote_path.rstrip("/") + "/."
+    else:
+        ensure_dir(local_path.parent)
+    argv = [
+        "scp",
+        "-o",
+        "BatchMode=yes",
+        "-o",
+        f"ConnectTimeout={config.ssh_timeout_sec}",
+        "-r",
+        "-p",
+        f"{host.ssh_alias}:{remote_src}",
+        str(local_path),
+    ]
+    run_local(argv, cwd=config.project_root, capture_output=True, check=True)
+
+
+def sync_push(config: FleetConfig, host: HostSpec) -> None:
+    if config.sync.mode == "rsync":
+        rsync_push(config, host)
+    else:
+        scp_push(config, host)
+
+
+def sync_pull(config: FleetConfig, host: HostSpec, remote_path: str, local_path: Path) -> None:
+    if config.sync.mode == "rsync":
+        rsync_pull(config, host, remote_path, local_path)
+    else:
+        scp_pull(config, host, remote_path, local_path)
+
+
 def ensure_remote_dir(config: FleetConfig, host: HostSpec, remote_path: str) -> None:
     run_ssh(config, host, f"mkdir -p {shlex.quote(remote_path)}", capture_output=True, check=True)
 
@@ -206,8 +259,10 @@ def load_config(path: Path) -> FleetConfig:
         local_path=relative_to_root(project_root, sync_raw.get("local_path"), project_root),
         exclude=[str(item) for item in sync_raw.get("exclude", [])],
     )
-    if sync.mode != "rsync":
+    if sync.mode not in {"rsync", "scp"}:
         raise FleetError(f"unsupported sync.mode: {sync.mode}")
+    if sync.mode == "scp" and sync.exclude:
+        raise FleetError("sync.exclude is not supported for sync.mode=scp")
 
     scheduler_raw = raw.get("scheduler", {})
     scheduler = SchedulerSpec(
@@ -639,7 +694,7 @@ def harvest_run(config: FleetConfig, manifest: dict[str, Any]) -> dict[str, Any]
 
     local_base = ensure_dir(config.artifacts_dir / refreshed["run_id"])
     remote_run_dir = refreshed["remote_run_dir"].rstrip("/")
-    rsync_pull(config, host, f"{remote_run_dir}/", local_base / "remote_run")
+    sync_pull(config, host, f"{remote_run_dir}/", local_base / "remote_run")
 
     for artifact in refreshed.get("artifacts", []):
         artifact_remote = f"{refreshed['remote_sync_path'].rstrip('/')}/{artifact}"
@@ -652,7 +707,7 @@ def harvest_run(config: FleetConfig, manifest: dict[str, Any]) -> dict[str, Any]
             check=False,
         )
         if check.returncode == 0:
-            rsync_pull(config, host, artifact_remote, target)
+            sync_pull(config, host, artifact_remote, target)
 
     refreshed["harvested_at"] = utc_now()
     write_run_manifest(config, refreshed)
@@ -866,7 +921,7 @@ def dispatch_jobs(
             )
             continue
         if host.name not in synced_hosts:
-            rsync_push(config, host)
+            sync_push(config, host)
             synced_hosts.add(host.name)
         manifest = launch_job(config, host, job, gpu_ids)
         manifests.append(manifest)

runs/frontier-attn-structured-v0/.gitignore

@@ -0,0 +1,4 @@
+__pycache__/
+cache/
+replay/
+figure-prototype.svg

runs/frontier-attn-structured-v0/0001-Experiment-with-structured-attention-prefill-predict.patch

@@ -0,0 +1,249 @@
+From 1f8900a4ac64e45754b03d0aa7c1dddab65785cf Mon Sep 17 00:00:00 2001
+From: Gahow Wang <gahow.wang@gmail.com>
+Date: Thu, 23 Jul 2026 15:27:40 +0800
+Subject: [PATCH] Experiment with structured attention prefill predictor
+
+---
+ .../shared_prediction_model_manager.py        | 16 +++-
+ .../sklearn_execution_time_predictor.py       | 16 +++-
+ .../structured_attention_prefill.py           | 79 +++++++++++++++++++
+ .../unit/test_structured_attention_prefill.py | 59 ++++++++++++++
+ 4 files changed, 165 insertions(+), 5 deletions(-)
+ create mode 100644 frontier/execution_time_predictor/structured_attention_prefill.py
+ create mode 100644 tests/unit/test_structured_attention_prefill.py
+
+diff --git a/frontier/execution_time_predictor/shared_prediction_model_manager.py b/frontier/execution_time_predictor/shared_prediction_model_manager.py
+index 8a65a49..4a21165 100644
+--- a/frontier/execution_time_predictor/shared_prediction_model_manager.py
++++ b/frontier/execution_time_predictor/shared_prediction_model_manager.py
+@@ -19,6 +19,9 @@ from frontier.execution_time_predictor.attention_tp_policy import (
+ from frontier.execution_time_predictor.attention_dataset_contract import (
+     enforce_mixed_attention_input_contract,
+ )
++from frontier.execution_time_predictor.structured_attention_prefill import (
++    StructuredAttentionPrefillRegressor,
++)
+ from frontier.logger import init_logger
+ from frontier.moe_gating_runtime import (
+     DEFAULT_MOE_GATING_RUNTIME_CONTEXT,
+@@ -1254,7 +1257,10 @@ class ExecutionTimePredictionModelManager:
+                 raise ValueError(
+                     "Missing required column 'prefill_chunk_size' in attention profiling data."
+                 )
+-            standard_prefill_df = prefill_df[prefill_df["prefill_chunk_size"] > 0].copy()
++            standard_prefill_df = prefill_df[
++                (prefill_df["prefill_chunk_size"] > 0)
++                & (prefill_df["batch_size"] == 1)
++            ].copy()
+ 
+             prefill_model_signature = f"attn_prefill_{attention_signature}"
+             if prefill_model_signature not in trained_model_signatures:
+@@ -1742,7 +1748,13 @@ class ExecutionTimePredictionModelManager:
+         # initialization to generate missing cache files.
+         # ============================================================
+ 
+-        estimator, grid_search_params = self._create_estimator_and_params(execution_time_predictor_config)
++        if model_name == "attn_prefill":
++            estimator = StructuredAttentionPrefillRegressor()
++            grid_search_params = {}
++        else:
++            estimator, grid_search_params = self._create_estimator_and_params(
++                execution_time_predictor_config
++            )
+ 
+         cv = min(execution_time_predictor_config.k_fold_cv_splits, len(df)) if len(df) >= 2 else 2
+ 
+diff --git a/frontier/execution_time_predictor/sklearn_execution_time_predictor.py b/frontier/execution_time_predictor/sklearn_execution_time_predictor.py
+index 27b62bf..b7f5350 100644
+--- a/frontier/execution_time_predictor/sklearn_execution_time_predictor.py
++++ b/frontier/execution_time_predictor/sklearn_execution_time_predictor.py
+@@ -45,6 +45,9 @@ from frontier.execution_time_predictor.attention_tp_policy import (
+ from frontier.execution_time_predictor.attention_dataset_contract import (
+     enforce_mixed_attention_input_contract,
+ )
++from frontier.execution_time_predictor.structured_attention_prefill import (
++    StructuredAttentionPrefillRegressor,
++)
+ from frontier.logger import init_logger
+ from frontier.moe_gating_runtime import get_moe_gating_base_model_name
+ from frontier.profiling.cpu_overhead.schema import (
+@@ -2573,8 +2576,12 @@ class SklearnExecutionTimePredictor(BaseExecutionTimePredictor):
+         if cached_model:
+             return cached_model
+ 
+-        model = self._get_estimator()
+-        grid_search_params = self._get_grid_search_params()
++        if model_name == "attn_prefill":
++            model = StructuredAttentionPrefillRegressor()
++            grid_search_params = {}
++        else:
++            model = self._get_estimator()
++            grid_search_params = self._get_grid_search_params()
+ 
+         if len(df) < self._config.k_fold_cv_splits:
+             cv = 2
+@@ -2869,7 +2876,10 @@ class SklearnExecutionTimePredictor(BaseExecutionTimePredictor):
+                 raise ValueError(
+                     "Missing required column 'prefill_chunk_size' in attention profiling data."
+                 )
+-            standard_prefill_df = prefill_df[prefill_df["prefill_chunk_size"] > 0].copy()
++            standard_prefill_df = prefill_df[
++                (prefill_df["prefill_chunk_size"] > 0)
++                & (prefill_df["batch_size"] == 1)
++            ].copy()
+             if len(standard_prefill_df) == 0:
+                 raise ValueError(
+                     "No standard prefill rows (prefill_chunk_size > 0) found in eager attention profiling data."
+diff --git a/frontier/execution_time_predictor/structured_attention_prefill.py b/frontier/execution_time_predictor/structured_attention_prefill.py
+new file mode 100644
+index 0000000..1829047
+--- /dev/null
++++ b/frontier/execution_time_predictor/structured_attention_prefill.py
+@@ -0,0 +1,79 @@
++"""Structured latency model for single-request chunked prefill attention."""
++
++from typing import Any
++
++import numpy as np
++from sklearn.base import BaseEstimator, RegressorMixin
++from sklearn.isotonic import IsotonicRegression
++from sklearn.linear_model import LinearRegression
++
++
++class StructuredAttentionPrefillRegressor(RegressorMixin, BaseEstimator):
++    """Model attention as a monotone base curve plus continuous KV growth.
++
++    Input columns retain the existing Frontier contract:
++    ``[kv_cache_size, prefill_chunk_size_squared]``.
++    """
++
++    def fit(self, X: Any, y: Any) -> "StructuredAttentionPrefillRegressor":
++        values = self._as_feature_array(X)
++        target = np.asarray(y, dtype=float)
++        kv_cache_size = values[:, 0]
++        prefill_chunk_size = np.sqrt(np.maximum(values[:, 1], 0.0))
++
++        base_mask = np.isclose(kv_cache_size, 0.0)
++        growth_mask = kv_cache_size > 0.0
++        if not np.any(base_mask) or not np.any(growth_mask):
++            raise ValueError(
++                "structured attn_prefill training requires both KV=0 base rows "
++                "and KV>0 growth rows"
++            )
++
++        base_q = prefill_chunk_size[base_mask]
++        base_y = target[base_mask]
++        unique_q = np.unique(base_q)
++        grouped_y = np.asarray(
++            [np.mean(base_y[np.isclose(base_q, q)]) for q in unique_q],
++            dtype=float,
++        )
++        self._base_model = IsotonicRegression(
++            increasing=True,
++            out_of_bounds="clip",
++        ).fit(unique_q, grouped_y)
++
++        growth_q = prefill_chunk_size[growth_mask]
++        growth_kv = kv_cache_size[growth_mask]
++        growth_base = self._base_model.predict(growth_q)
++        growth_features = np.column_stack(
++            (growth_kv, growth_q * growth_kv)
++        )
++        self._growth_model = LinearRegression(
++            fit_intercept=False,
++            positive=True,
++        ).fit(growth_features, target[growth_mask] - growth_base)
++
++        self.n_features_in_ = 2
++        self._frontier_base_q_min = float(unique_q.min())
++        self._frontier_base_q_max = float(unique_q.max())
++        self._frontier_growth_kv_max = float(growth_kv.max())
++        return self
++
++    def predict(self, X: Any) -> np.ndarray:
++        values = self._as_feature_array(X)
++        kv_cache_size = values[:, 0]
++        prefill_chunk_size = np.sqrt(np.maximum(values[:, 1], 0.0))
++        base = self._base_model.predict(prefill_chunk_size)
++        growth_features = np.column_stack(
++            (kv_cache_size, prefill_chunk_size * kv_cache_size)
++        )
++        return np.maximum(base + self._growth_model.predict(growth_features), 0.0)
++
++    @staticmethod
++    def _as_feature_array(X: Any) -> np.ndarray:
++        values = np.asarray(X, dtype=float)
++        if values.ndim != 2 or values.shape[1] != 2:
++            raise ValueError(
++                "structured attn_prefill expects exactly two features: "
++                "kv_cache_size and prefill_chunk_size_squared"
++            )
++        return values
+diff --git a/tests/unit/test_structured_attention_prefill.py b/tests/unit/test_structured_attention_prefill.py
+new file mode 100644
+index 0000000..12c4247
+--- /dev/null
++++ b/tests/unit/test_structured_attention_prefill.py
+@@ -0,0 +1,59 @@
++import pickle
++import unittest
++
++import numpy as np
++
++from frontier.execution_time_predictor.structured_attention_prefill import (
++    StructuredAttentionPrefillRegressor,
++)
++
++
++class StructuredAttentionPrefillRegressorTest(unittest.TestCase):
++    def setUp(self) -> None:
++        q = np.asarray([64, 128, 256, 512, 1024, 2048, 4096, 8192], dtype=float)
++        base = 0.05 + 1e-4 * q + 4e-8 * q**2
++        context_q = np.asarray([2048, 4096, 8192] * 3, dtype=float)
++        context_kv = np.repeat([8192, 16384, 24576], 3).astype(float)
++        context_y = (
++            np.interp(context_q, q, base)
++            + 1.5e-5 * context_kv
++            + 3e-8 * context_q * context_kv
++        )
++        self.X = np.column_stack(
++            (
++                np.concatenate((np.zeros_like(q), context_kv)),
++                np.concatenate((q**2, context_q**2)),
++            )
++        )
++        self.y = np.concatenate((base, context_y))
++
++    def test_recovers_structured_curve(self) -> None:
++        model = StructuredAttentionPrefillRegressor().fit(self.X, self.y)
++        np.testing.assert_allclose(model.predict(self.X), self.y, rtol=1e-6)
++
++    def test_prediction_is_nonnegative_and_monotone(self) -> None:
++        model = StructuredAttentionPrefillRegressor().fit(self.X, self.y)
++        q = np.arange(1, 8193, dtype=float)
++        for kv in (0, 8192, 32768, 40912):
++            X = np.column_stack((np.full_like(q, kv), q**2))
++            prediction = model.predict(X)
++            self.assertTrue(np.all(prediction >= 0))
++            self.assertTrue(np.all(np.diff(prediction) >= -1e-12))
++
++        kv = np.arange(0, 40913, 64, dtype=float)
++        for q_value in (64, 2048, 8192):
++            X = np.column_stack((kv, np.full_like(kv, q_value**2)))
++            self.assertTrue(np.all(np.diff(model.predict(X)) >= -1e-12))
++
++    def test_pickle_round_trip(self) -> None:
++        model = StructuredAttentionPrefillRegressor().fit(self.X, self.y)
++        restored = pickle.loads(pickle.dumps(model))
++        np.testing.assert_allclose(restored.predict(self.X), self.y, rtol=1e-6)
++
++    def test_requires_base_and_growth_rows(self) -> None:
++        with self.assertRaisesRegex(ValueError, "KV=0 base rows"):
++            StructuredAttentionPrefillRegressor().fit(self.X[:8], self.y[:8])
++
++
++if __name__ == "__main__":
++    unittest.main()
+-- 
+2.43.0
+

runs/frontier-attn-structured-v0/analyze_predictor_ablation.py

@@ -0,0 +1,256 @@
+#!/usr/bin/env python3
+"""Offline predictor ablation for EXP-ATTN-STRUCTURED.
+
+This is deliberately profile-only: it decides whether the structured model is
+good enough to justify the expensive 7-cell trace replay.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import sys
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import pandas as pd
+from sklearn.ensemble import RandomForestRegressor
+
+ROOT = Path(__file__).resolve().parent
+REPO = ROOT.parents[1]
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--profile",
+        type=Path,
+        default=REPO
+        / "runs/frontier-prefill-kvgrowth-fix-v0/profiles/"
+        "profile-v5-kvgrowth/attention.csv",
+    )
+    parser.add_argument(
+        "--frontier-checkout",
+        type=Path,
+        default=Path("/tmp/frontier-attn-structured-v0"),
+    )
+    parser.add_argument("--output-root", type=Path, default=ROOT / "results")
+    return parser.parse_args()
+
+
+def normalize_bool(series: pd.Series) -> pd.Series:
+    return series.astype(str).str.strip().str.lower().isin(
+        {"1", "true", "t", "yes", "y"}
+    )
+
+
+def load_profile(path: Path) -> pd.DataFrame:
+    df = pd.read_csv(path).drop_duplicates()
+    for column in ("is_prefill", "is_true_mixed_batch"):
+        df[column] = normalize_bool(df[column])
+    df = df[
+        (df["n_embd"] == 2048)
+        & (df["n_q_head"] == 32)
+        & (df["n_kv_head"] == 4)
+        & (df["block_size"] == 16)
+        & df["is_prefill"]
+        & ~df["is_true_mixed_batch"]
+        & (df["prefill_chunk_size"] > 0)
+    ].copy()
+    df["prefill_chunk_size_squared"] = df["prefill_chunk_size"] ** 2
+    return df
+
+
+def mape(actual: np.ndarray, predicted: np.ndarray) -> float:
+    return float(np.mean(np.abs((predicted - actual) / actual)) * 100)
+
+
+def make_rf() -> RandomForestRegressor:
+    # Exact best parameters selected by the current profile-v5 GridSearchCV.
+    return RandomForestRegressor(
+        random_state=0,
+        n_estimators=250,
+        max_depth=8,
+        min_samples_split=2,
+    )
+
+
+def features(df: pd.DataFrame) -> pd.DataFrame:
+    return df[["kv_cache_size", "prefill_chunk_size_squared"]]
+
+
+def score_model(
+    name: str,
+    estimator: Any,
+    train: pd.DataFrame,
+    single: pd.DataFrame,
+    grid: pd.DataFrame,
+) -> dict[str, Any]:
+    target = "time_stats.attn_prefill.median"
+    estimator.fit(features(train), train[target])
+    grid_prediction = estimator.predict(features(grid))
+    single_prediction = estimator.predict(features(single))
+
+    heldout_actual: list[float] = []
+    heldout_prediction: list[float] = []
+    for context in sorted(grid["kv_cache_size"].unique()):
+        test = grid[grid["kv_cache_size"] == context]
+        fold_train = train.drop(index=test.index, errors="ignore")
+        fold_model = (
+            make_rf()
+            if name.startswith("rf")
+            else estimator.__class__()
+        )
+        fold_model.fit(features(fold_train), fold_train[target])
+        heldout_actual.extend(test[target].astype(float))
+        heldout_prediction.extend(fold_model.predict(features(test)))
+
+    q = np.arange(1, 8193, dtype=float)
+    q_deltas: list[float] = []
+    prediction_min: list[float] = []
+    for context in (0, 8192, 16384, 24576, 32768, 40912):
+        X = pd.DataFrame(
+            {
+                "kv_cache_size": np.full_like(q, context),
+                "prefill_chunk_size_squared": q**2,
+            }
+        )
+        prediction = estimator.predict(X)
+        prediction_min.append(float(prediction.min()))
+        q_deltas.append(float(np.diff(prediction).min()))
+
+    kv = np.arange(0, 40913, 64, dtype=float)
+    kv_deltas: list[float] = []
+    for query in (64, 512, 2048, 4096, 8192):
+        X = pd.DataFrame(
+            {
+                "kv_cache_size": kv,
+                "prefill_chunk_size_squared": np.full_like(kv, query**2),
+            }
+        )
+        kv_deltas.append(float(np.diff(estimator.predict(X)).min()))
+
+    heldout_actual_array = np.asarray(heldout_actual)
+    heldout_prediction_array = np.asarray(heldout_prediction)
+    return {
+        "candidate": name,
+        "training_rows": len(train),
+        "grid_fit_mape_pct": mape(
+            grid[target].to_numpy(), np.asarray(grid_prediction)
+        ),
+        "single_fit_mape_pct": mape(
+            single[target].to_numpy(), np.asarray(single_prediction)
+        ),
+        "heldout_context_mape_pct": mape(
+            heldout_actual_array, heldout_prediction_array
+        ),
+        "heldout_context_max_abs_error_pct": float(
+            np.max(
+                np.abs(
+                    (heldout_prediction_array - heldout_actual_array)
+                    / heldout_actual_array
+                )
+            )
+            * 100
+        ),
+        "prediction_min_ms": min(prediction_min),
+        "q_min_delta_ms": min(q_deltas),
+        "kv_min_delta_ms": min(kv_deltas),
+        "monotone_and_nonnegative": (
+            min(prediction_min) >= 0
+            and min(q_deltas) >= -1e-12
+            and min(kv_deltas) >= -1e-12
+        ),
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    sys.path.insert(0, str(args.frontier_checkout))
+    from frontier.execution_time_predictor.structured_attention_prefill import (
+        StructuredAttentionPrefillRegressor,
+    )
+
+    df = load_profile(args.profile)
+    records: list[dict[str, Any]] = []
+    data_audit: dict[str, Any] = {}
+    for tp in (1, 2, 4):
+        tp_df = df[df["num_tensor_parallel_workers"] == tp].copy()
+        single = tp_df[tp_df["batch_size"] == 1].copy()
+        grid = single[
+            single["prefill_chunk_size"].isin((2048, 4096, 8192))
+            & (single["kv_cache_size"] > 0)
+        ].copy()
+        duplicate_groups = (
+            tp_df.groupby(
+                ["kv_cache_size", "prefill_chunk_size_squared"]
+            )
+            .size()
+            .gt(1)
+            .sum()
+        )
+        data_audit[f"tp{tp}"] = {
+            "standard_rows": len(tp_df),
+            "single_request_rows": len(single),
+            "target_grid_rows": len(grid),
+            "duplicate_feature_groups": int(duplicate_groups),
+        }
+
+        candidates = (
+            ("rf_all", make_rf(), tp_df),
+            ("rf_single", make_rf(), single),
+            (
+                "structured_single",
+                StructuredAttentionPrefillRegressor(),
+                single,
+            ),
+        )
+        for name, model, train in candidates:
+            result = score_model(name, model, train, single, grid)
+            result["tp"] = tp
+            records.append(result)
+
+    structured = [r for r in records if r["candidate"] == "structured_single"]
+    checks = {
+        "heldout_context_mape_le_5pct": all(
+            r["heldout_context_mape_pct"] <= 5 for r in structured
+        ),
+        "monotone_and_nonnegative": all(
+            r["monotone_and_nonnegative"] for r in structured
+        ),
+    }
+    checks["profile_gate"] = all(checks.values())
+    payload = {
+        "schema": "frontier-attn-structured-ablation-v1",
+        "profile": str(args.profile.resolve()),
+        "frontier_checkout": str(args.frontier_checkout.resolve()),
+        "data_audit": data_audit,
+        "results": records,
+        "checks": checks,
+    }
+
+    args.output_root.mkdir(parents=True, exist_ok=True)
+    (args.output_root / "predictor-ablation.json").write_text(
+        json.dumps(payload, indent=2)
+    )
+    with (args.output_root / "predictor-ablation.csv").open(
+        "w", newline=""
+    ) as stream:
+        writer = csv.DictWriter(stream, fieldnames=list(records[0]))
+        writer.writeheader()
+        writer.writerows(records)
+    print(json.dumps(checks, indent=2))
+    for row in records:
+        print(
+            f"TP{row['tp']} {row['candidate']:18s} "
+            f"grid={row['grid_fit_mape_pct']:.2f}% "
+            f"heldout={row['heldout_context_mape_pct']:.2f}% "
+            f"max={row['heldout_context_max_abs_error_pct']:.2f}% "
+            f"monotone={row['monotone_and_nonnegative']}"
+        )
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-attn-structured-v0/analyze_trace_verdict.py

@@ -0,0 +1,304 @@
+#!/usr/bin/env python3
+"""Trial-aware verdict for the seven structured-attention trace replays."""
+
+from __future__ import annotations
+
+import csv
+import json
+import math
+from pathlib import Path
+from typing import Any
+
+ROOT = Path(__file__).resolve().parent
+REPO = ROOT.parents[1]
+S3_REAL = REPO / "runs/frontier-s3-real-v0"
+V5 = REPO / "runs/frontier-prefill-kvgrowth-fix-v0"
+
+CELLS = {
+    "tp1_rho0p00125": {
+        "real": "frontier-tp1-real-r0p00125-t*",
+        "old": V5 / "sim-replay-tp1/v5/tp1_rho0p00125",
+    },
+    "tp1_rho0p0025": {
+        "real": "frontier-tp1-real-r0p0025-t*",
+        "old": V5 / "sim-replay-tp1/v5/tp1_rho0p0025",
+    },
+    "tp2_rho0p0025": {
+        "real": "frontier-s3-real-full-r0p0025-tp2-t*",
+        "old": V5 / "sim-replay/tp2_rho0p0025",
+    },
+    "tp2_rho0p005": {
+        "real": "frontier-s3-real-full-r0p005-tp2-t*",
+        "old": V5 / "sim-replay/tp2_rho0p005",
+    },
+    "tp4_rho0p0025": {
+        "real": "frontier-s3-real-full-r0p0025-tp4-t*",
+        "old": V5 / "sim-replay/tp4_rho0p0025",
+    },
+    "tp4_rho0p005": {
+        "real": "frontier-s3-real-full-r0p005-tp4-t*",
+        "old": V5 / "sim-replay/tp4_rho0p005",
+    },
+    "tp4_rho0p01": {
+        "real": "frontier-s3-real-full-r0p01-tp4-t*",
+        "old": V5 / "sim-replay/tp4_rho0p01",
+    },
+}
+METRICS = {
+    "ttft": ("ttft_ms", "ttft"),
+    "tpot": ("tpot_ms", "tpot"),
+    "e2e": ("e2e_ms", "request_e2e_time"),
+}
+QUANTILES = {"mean": None, "p50": 0.5, "p90": 0.9, "p99": 0.99}
+
+
+def percentile(values: list[float], quantile: float) -> float:
+    ordered = sorted(values)
+    position = (len(ordered) - 1) * quantile
+    lower, upper = math.floor(position), math.ceil(position)
+    if lower == upper:
+        return ordered[lower]
+    return (
+        ordered[lower] * (upper - position)
+        + ordered[upper] * (position - lower)
+    )
+
+
+def summarize(values: list[float]) -> dict[str, float]:
+    return {
+        name: (
+            sum(values) / len(values)
+            if quantile is None
+            else percentile(values, quantile)
+        )
+        for name, quantile in QUANTILES.items()
+    }
+
+
+def load_real_trials(pattern: str) -> list[list[dict[str, Any]]]:
+    trials = []
+    for run_root in sorted((S3_REAL / "fleet-artifacts").glob(pattern)):
+        results = list(
+            run_root.glob(
+                "artifacts/outputs/full-real/*/*/trial-*/results/result.json"
+            )
+        )
+        if len(results) != 1:
+            raise ValueError(f"expected one result in {run_root}, got {results}")
+        trials.append(json.loads(results[0].read_text())["requests"])
+    if len(trials) != 2:
+        raise ValueError(f"expected two real trials for {pattern}, got {len(trials)}")
+    return trials
+
+
+def load_sim(root: Path) -> list[dict[str, str]]:
+    matches = list((root / "metrics").rglob("request_metrics.csv"))
+    if len(matches) != 1:
+        raise ValueError(f"expected one request_metrics.csv below {root}: {matches}")
+    rows = list(csv.DictReader(matches[0].open()))
+    rows.sort(key=lambda row: int(float(row["Request Id"])))
+    return rows
+
+
+def distribution_bias(
+    real_rows: list[dict[str, Any]],
+    sim_rows: list[dict[str, str]],
+) -> dict[str, dict[str, float]]:
+    output: dict[str, dict[str, float]] = {}
+    for metric, (real_key, sim_key) in METRICS.items():
+        pairs = [
+            (float(real[real_key]), float(sim[sim_key]))
+            for real, sim in zip(real_rows, sim_rows)
+            if real.get("success")
+        ]
+        real_summary = summarize([pair[0] for pair in pairs])
+        sim_summary = summarize([pair[1] for pair in pairs])
+        output[metric] = {
+            name: (sim_summary[name] - real_summary[name]) / real_summary[name]
+            for name in QUANTILES
+        }
+    return output
+
+
+def paired_relative_error(
+    real_rows: list[dict[str, Any]],
+    sim_rows: list[dict[str, str]],
+) -> dict[str, dict[str, float]]:
+    output: dict[str, dict[str, float]] = {}
+    for metric, (real_key, sim_key) in METRICS.items():
+        errors = [
+            (float(sim[sim_key]) - float(real[real_key])) / float(real[real_key])
+            for real, sim in zip(real_rows, sim_rows)
+            if real.get("success") and float(real[real_key]) != 0
+        ]
+        output[metric] = summarize(errors)
+    return output
+
+
+def aggregate_trial_bias(
+    trial_biases: list[dict[str, dict[str, float]]],
+) -> dict[str, dict[str, dict[str, float]]]:
+    return {
+        metric: {
+            quantile: {
+                "mean": sum(values) / len(values),
+                "min": min(values),
+                "max": max(values),
+            }
+            for quantile in QUANTILES
+            for values in [
+                [trial[metric][quantile] for trial in trial_biases]
+            ]
+        }
+        for metric in METRICS
+    }
+
+
+def legacy_pooled_bias(
+    real_trials: list[list[dict[str, Any]]],
+    sim_rows: list[dict[str, str]],
+) -> dict[str, dict[str, float]]:
+    output: dict[str, dict[str, float]] = {}
+    for metric, (real_key, sim_key) in METRICS.items():
+        real_values = [
+            float(row[real_key])
+            for trial in real_trials
+            for row in trial[: len(sim_rows)]
+            if row.get("success")
+        ]
+        sim_values = [float(row[sim_key]) for row in sim_rows]
+        real_summary = summarize(real_values)
+        sim_summary = summarize(sim_values)
+        output[metric] = {
+            name: (sim_summary[name] - real_summary[name]) / real_summary[name]
+            for name in QUANTILES
+        }
+    return output
+
+
+def waiting_p99(sim_rows: list[dict[str, str]]) -> float:
+    return percentile(
+        [float(row["request_waiting_time_total"]) for row in sim_rows], 0.99
+    )
+
+
+def main() -> None:
+    results: dict[str, Any] = {}
+    flat_rows: list[dict[str, Any]] = []
+    for label, paths in CELLS.items():
+        real_trials = load_real_trials(paths["real"])
+        old_sim = load_sim(paths["old"])
+        new_sim = load_sim(ROOT / "replay" / label)
+        old_trial_bias = [
+            distribution_bias(trial, old_sim) for trial in real_trials
+        ]
+        new_trial_bias = [
+            distribution_bias(trial, new_sim) for trial in real_trials
+        ]
+        old_legacy = legacy_pooled_bias(real_trials, old_sim)
+        new_legacy = legacy_pooled_bias(real_trials, new_sim)
+        wait_p99 = waiting_p99(new_sim)
+        results[label] = {
+            "old": {
+                "trialwise_distribution_bias": old_trial_bias,
+                "trialwise_distribution_bias_summary": aggregate_trial_bias(
+                    old_trial_bias
+                ),
+                "legacy_pooled_distribution_bias": old_legacy,
+            },
+            "new": {
+                "trialwise_distribution_bias": new_trial_bias,
+                "trialwise_distribution_bias_summary": aggregate_trial_bias(
+                    new_trial_bias
+                ),
+                "paired_relative_error": [
+                    paired_relative_error(trial, new_sim)
+                    for trial in real_trials
+                ],
+                "legacy_pooled_distribution_bias": new_legacy,
+                "waiting_p99_ms": wait_p99,
+                "validity": (
+                    "PASS_SUBCRITICAL"
+                    if wait_p99 < 1000
+                    else "GATE_FAIL_DIAGNOSTIC"
+                ),
+            },
+        }
+        for metric in METRICS:
+            for quantile in QUANTILES:
+                flat_rows.append(
+                    {
+                        "cell": label,
+                        "metric": metric,
+                        "quantile": quantile,
+                        "old_bias": old_legacy[metric][quantile],
+                        "new_bias": new_legacy[metric][quantile],
+                        "abs_bias_delta_pp": 100
+                        * (
+                            abs(new_legacy[metric][quantile])
+                            - abs(old_legacy[metric][quantile])
+                        ),
+                        "validity": results[label]["new"]["validity"],
+                    }
+                )
+
+    tp1_checks = []
+    for cell in ("tp1_rho0p00125", "tp1_rho0p0025"):
+        for quantile in ("mean", "p99"):
+            old = results[cell]["old"]["legacy_pooled_distribution_bias"]["ttft"][
+                quantile
+            ]
+            new = results[cell]["new"]["legacy_pooled_distribution_bias"]["ttft"][
+                quantile
+            ]
+            tp1_checks.append(abs(old) - abs(new) >= 0.05)
+
+    regressions = [
+        row
+        for row in flat_rows
+        if row["cell"].startswith(("tp2", "tp4"))
+        and row["metric"] in ("ttft", "e2e")
+        and row["abs_bias_delta_pp"] > 5
+    ]
+    checks = {
+        "tp1_ttft_mean_p99_improve_ge_5pp": all(tp1_checks),
+        "tp2_tp4_ttft_e2e_no_abs_regression_gt_5pp": not regressions,
+        "regressions": regressions,
+    }
+    checks["trace_gate"] = (
+        checks["tp1_ttft_mean_p99_improve_ge_5pp"]
+        and checks["tp2_tp4_ttft_e2e_no_abs_regression_gt_5pp"]
+    )
+
+    payload = {
+        "schema": "frontier-attn-structured-trial-aware-verdict-v1",
+        "metric_note": (
+            "Primary values are per-real-trial distribution biases with request "
+            "alignment by index. legacy_pooled reproduces the old milestone "
+            "quantile convention only for direct comparison."
+        ),
+        "cells": results,
+        "checks": checks,
+    }
+    output = ROOT / "results"
+    output.mkdir(parents=True, exist_ok=True)
+    (output / "trace-verdict.json").write_text(json.dumps(payload, indent=2))
+    with (output / "trace-verdict.csv").open("w", newline="") as stream:
+        writer = csv.DictWriter(stream, fieldnames=list(flat_rows[0]))
+        writer.writeheader()
+        writer.writerows(flat_rows)
+
+    print(json.dumps(checks, indent=2))
+    for label, result in results.items():
+        old = result["old"]["legacy_pooled_distribution_bias"]["ttft"]
+        new = result["new"]["legacy_pooled_distribution_bias"]["ttft"]
+        print(
+            f"{label}: TTFT mean {old['mean']:+.1%}->{new['mean']:+.1%}, "
+            f"p99 {old['p99']:+.1%}->{new['p99']:+.1%}, "
+            f"waiting_p99={result['new']['waiting_p99_ms']:.0f}ms "
+            f"{result['new']['validity']}"
+        )
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-attn-structured-v0/experiment-card.md

@@ -0,0 +1,82 @@
+# 实验 EXP-ATTN-STRUCTURED：结构化 predictor 能否关闭大 KV 端的 RF 欠拟合
+
+> **状态：** 已完成（profile gate PASS；global merge gate FAIL）
+>
+> Parent campaign：[`../frontier-simulator-gap-campaign-v0/README.md`](../frontier-simulator-gap-campaign-v0/README.md)
+
+## Claim 与决策
+
+- **Parent claim：** profile-v5 已补齐 chunked-prefill KV-context 测量，但当前 RF 仍在 TP1/2/4 的新网格上产生约 12%--14% self-fit MAPE，并在 TP1 真实 trace 中留下 −13% 到 −22% TTFT 偏差。
+- **目的：** 检查该 residual 是否来自可工程修复的 predictor representation，而不是 profile 数据或 serving path。
+- **Competing hypotheses：**
+  - H1：standard prefill 模型错误混入 pure multi-request rows，且 RF 对连续 attention scaling 作阶梯平滑；使用单请求数据和结构化 `base(q)+KV×(a+bq)` 可关闭残余。
+  - H2：残余主要来自未建模的 serving-path 组件；替换 predictor 不会改善 7-cell trace fidelity。
+- **事前预测：**
+  - H1：held-out context MAPE ≤5%，TP1 TTFT mean/p99 绝对偏差至少改善 5 pp。
+  - H2：profile gate 失败，或 profile gate 通过但 trace TTFT 几乎不动。
+- **判定规则：**
+  - profile gate：TP1/2/4 held-out context MAPE 均 ≤5%；q/KV 单调且预测非负。
+  - trace gate：两个 TP1 cell 的 TTFT mean/p99 |bias| 各改善 ≥5 pp；TP2/TP4 任一 TTFT/E2E quantile 不恶化 >5 pp。
+  - profile gate 失败即停止；trace gate 失败则回退 patch，不进入 EXP-2。
+
+## Setup
+
+- **自变量：**
+  - A：现有 RF，standard prefill 全部非 true-mixed rows。
+  - B：现有 RF，但仅 `batch_size=1`。
+  - C：仅 `batch_size=1` 的 structured predictor：
+    - `base(q)`：KV=0 profile 的单调分段线性插值；
+    - growth：非负 least-squares `KV×(a+bq)`。
+- **控制变量：** attention/linear/MoE/collective profile、trace、prefix cache、scheduler、graph mode、KV blocks、MNS、全部 argv。
+- **System context：** Qwen3-30B-A3B BF16；H20；Frontier `deadc4a3`；TP1/2/4；MNS16；chunk 8192；prefix caching。
+- **Workload 或 trace：** 现有 7-cell 60-min production chat trace matrix；real 侧每 cell 两个 trial。
+- **Baselines：** `docs/assets/frontier-fidelity/full-matrix.csv` 的 sim-v5。
+- **Metrics：**
+  - profile：grid fit MAPE、leave-one-context MAPE/max error、q/KV monotonicity；
+  - trace：request-ID paired bias；每个 real trial 单独计算后报告 mean 与 trial interval；
+  - queue validity：waiting p99，TP1 超过 1 s 的 cell 标为 diagnostic。
+
+## 预期产物与 review
+
+- **预期数据：** `results/predictor-ablation.{json,csv}`、`replay/<cell>/`、`results/paired-verdict.json`。
+- **Figure prototype：** `figure-prototype.png`；左图为 q8k 随 KV 增长的 actual/RF/structured，右图为 7-cell TTFT bias 的事前期望。
+- **人工 review：** 已按 campaign 顺序批准执行。
+- **Review 意见：** 只改 standard single-request predictor；不得改 mixed predictor 或任何 profile row。
+
+## 复现信息
+
+- **Code：** Frontier base `deadc4a321f0baaa534c6ebd17f974123733cdc2`；实验 patch 将保存为 `frontier-structured-attn.patch` 并记录 SHA256。
+- **Environment：** 本地 CPU replay；Python dependency roots 复用 `runs/frontier-collective-joint-v0/counterfactual/joint-r2/manifest.json`。
+- **产物路径：** 本目录。
+- **已知 deviation：** milestone 文档将 7-cell 口径称为“逐 request paired”，但旧脚本实际 pool 两个 real trial 后比较 quantile；本实验会修正分析口径，不改旧结果文件。
+
+## 预分析事实
+
+- 现有训练代码使用 `["kv_cache_size", "prefill_chunk_size_squared"]` 与 RF grid search。
+- runtime cache 注释明确 standard model 是 per-request；多请求 prefill 在模型存在时走 `attn_prefill_mixed`。
+- profile-v5 的 standard 训练集每 TP 有 29 行，其中单请求 23 行；有 4 组相同 `(KV,q²)` feature 对应多个 pure-batch 标签。
+- 初步 structured candidate 的 leave-one-context MAPE：TP1 0.84%、TP2 1.61%、TP4 3.01%；max error 分别 2.04%、3.47%、5.49%。这些是实现前的临时计算，须由版本化脚本复现后才进入结果。
+
+## 结果
+
+- **观察事实：**
+  - structured held-out-context MAPE 为 TP1/2/4=`0.84%/1.60%/3.01%`；
+    当前 RF 为 `44.40%/44.20%/43.54%`。单调/非负 gate 通过。
+  - TP1 两点 TTFT mean bias `−13.5/−17.7% → −6.3/−9.5%`，p99
+    `−16.8/−22.1% → −8.6/−14.4%`。
+  - TP2 两点 TTFT mean bias `−11.2/−14.1% → −4.5/−7.1%`，p99
+    `−17.3/−19.4% → −7.7/−9.0%`。
+  - TP4 三点 TTFT mean bias `+2.5/+2.9/−0.1% → +7.8/+8.4/+6.0%`；
+    三点均使绝对误差恶化 `5.3--5.8 pp`，触发预设回归 gate。
+  - validity 重新审计：TP1 两点 waiting p99=`1.34/1.89 s`；TP2
+    ρ=.005=`1.17 s`，均标为 `GATE_FAIL_DIAGNOSTIC`。其余四点通过。
+- **异常：** TP4 ρ=.005 的 TPOT p99 从 `+31.2%` 变为 `+36.6%`，
+  表明该 tail 对 prefill/mixed-decode 相位敏感，不是本 patch 能关闭的稳定
+  decode predictor 偏差。
+- **含义：** H1 的 representation 机制得到支持，但“全局替换 RF 可直接提升
+  7-cell fidelity”被反驳。TP4 原先接近零的 mean TTFT 含有 predictor
+  欠拟合与其它正向 residual 的误差抵消；单独修正 attention 会揭开后者。
+- **Claim update：** structured predictor 是明确的工程候选，但必须与 TP4
+  residual 联合收敛后才可 merge；当前 patch 只保留为 ablation。
+- **下一步：** EXP-2 先重算 structured 分支的 TP2 chunk-level residual；
+  仅 residual ≥10% 才运行 GPU serving-path 三臂 profile。

runs/frontier-attn-structured-v0/frontier-reference.json

@@ -0,0 +1,525 @@
+{
+  "cc_cache": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/cc-cache",
+  "cells": {
+    "tp1_mns16": {
+      "argv": [
+        "/usr/bin/python3",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/run_frontier_with_curves.py",
+        "--simulation_mode",
+        "online",
+        "--sys_arch",
+        "co-location",
+        "--cc_backend_config_type",
+        "vidur",
+        "--cluster_config_num_replicas",
+        "1",
+        "--cluster_scheduler_config_type",
+        "sticky_round_robin",
+        "--replica_config_model_name",
+        "qwen3-a3b-30b-moe",
+        "--replica_config_device",
+        "h20",
+        "--replica_config_network_device",
+        "h20_dgx",
+        "--replica_config_attn_tensor_parallel_size",
+        "1",
+        "--replica_config_attn_data_parallel_size",
+        "1",
+        "--replica_config_moe_tensor_parallel_size",
+        "1",
+        "--replica_config_moe_expert_parallel_size",
+        "1",
+        "--replica_config_num_pipeline_stages",
+        "1",
+        "--replica_scheduler_config_type",
+        "vllm_v1",
+        "--decode_cuda_graph_mode",
+        "piecewise",
+        "--vllm_v1_scheduler_config_batch_size_cap",
+        "16",
+        "--vllm_v1_scheduler_config_max_tokens_in_batch",
+        "8192",
+        "--vllm_v1_scheduler_config_long_prefill_token_threshold",
+        "0",
+        "--vllm_v1_scheduler_config_block_size",
+        "16",
+        "--vllm_v1_scheduler_config_num_blocks_mode",
+        "explicit",
+        "--vllm_v1_scheduler_config_gpu_memory_utilization",
+        "0.92",
+        "--vllm_v1_scheduler_config_non_kv_cache_overhead_bytes",
+        "0",
+        "--request_generator_config_type",
+        "trace_replay",
+        "--trace_request_generator_config_trace_file",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp1-frontier.csv",
+        "--trace_request_generator_config_max_tokens",
+        "40960",
+        "--metrics_config_output_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/sim/tp1_mns16/metrics",
+        "--metrics_config_run_id",
+        "joint_tp1_mns16",
+        "--metrics_config_write_metrics",
+        "--metrics_config_store_request_metrics",
+        "--metrics_config_store_batch_metrics",
+        "--metrics_config_store_token_completion_metrics",
+        "--metrics_config_store_utilization_metrics",
+        "--no-metrics_config_store_plots",
+        "--no-metrics_config_enable_chrome_trace",
+        "--no-metrics_config_write_json_trace",
+        "--metrics_config_store_frontier_stage_batch_ledger",
+        "--no-random_forrest_execution_time_predictor_config_enable_dummy_mode",
+        "--random_forrest_execution_time_predictor_config_linear_op_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/moe.csv",
+        "--random_forrest_execution_time_predictor_config_linear_op_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/moe.csv",
+        "--random_forrest_execution_time_predictor_config_prediction_max_prefill_chunk_size",
+        "8192",
+        "--random_forrest_execution_time_predictor_config_prediction_max_batch_size",
+        "32",
+        "--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request",
+        "40960",
+        "--random_forrest_execution_time_predictor_config_no_cache",
+        "--random_forrest_execution_time_predictor_config_skip_cpu_overhead_modeling",
+        "--vllm_v1_scheduler_config_num_blocks",
+        "20128",
+        "--vllm_v1_scheduler_config_enable_chunked_prefill",
+        "--random_forrest_execution_time_predictor_config_num_training_job_threads",
+        "4",
+        "--cudagraph_capture_sizes",
+        "1",
+        "2",
+        "4",
+        "8",
+        "16",
+        "24",
+        "32",
+        "--vidur_cc_backend_config_all_reduce_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/measured-allreduce.csv",
+        "--vidur_cc_backend_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/cc-cache",
+        "--vidur_cc_backend_config_k_fold_cv_splits",
+        "6",
+        "--vidur_cc_backend_config_num_training_job_threads",
+        "1",
+        "--metrics_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/model-cache"
+      ],
+      "log": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/logs/tp1_mns16.log",
+      "source_command": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp1_mns16/tp1/command.json",
+      "source_command_sha256": "a9815797b1601bf6f6cdf0269e84acb376a84945609e338868dc8347aab650e6",
+      "usage": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/usage/tp1_mns16.json"
+    },
+    "tp2_mns16": {
+      "argv": [
+        "/usr/bin/python3",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/run_frontier_with_curves.py",
+        "--simulation_mode",
+        "online",
+        "--sys_arch",
+        "co-location",
+        "--cc_backend_config_type",
+        "vidur",
+        "--cluster_config_num_replicas",
+        "1",
+        "--cluster_scheduler_config_type",
+        "sticky_round_robin",
+        "--replica_config_model_name",
+        "qwen3-a3b-30b-moe",
+        "--replica_config_device",
+        "h20",
+        "--replica_config_network_device",
+        "h20_dgx",
+        "--replica_config_attn_tensor_parallel_size",
+        "2",
+        "--replica_config_attn_data_parallel_size",
+        "1",
+        "--replica_config_moe_tensor_parallel_size",
+        "2",
+        "--replica_config_moe_expert_parallel_size",
+        "1",
+        "--replica_config_num_pipeline_stages",
+        "1",
+        "--replica_scheduler_config_type",
+        "vllm_v1",
+        "--decode_cuda_graph_mode",
+        "piecewise",
+        "--vllm_v1_scheduler_config_batch_size_cap",
+        "16",
+        "--vllm_v1_scheduler_config_max_tokens_in_batch",
+        "8192",
+        "--vllm_v1_scheduler_config_long_prefill_token_threshold",
+        "0",
+        "--vllm_v1_scheduler_config_block_size",
+        "16",
+        "--vllm_v1_scheduler_config_num_blocks_mode",
+        "explicit",
+        "--vllm_v1_scheduler_config_gpu_memory_utilization",
+        "0.92",
+        "--vllm_v1_scheduler_config_non_kv_cache_overhead_bytes",
+        "0",
+        "--request_generator_config_type",
+        "trace_replay",
+        "--trace_request_generator_config_trace_file",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp2-frontier.csv",
+        "--trace_request_generator_config_max_tokens",
+        "40960",
+        "--metrics_config_output_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/sim/tp2_mns16/metrics",
+        "--metrics_config_run_id",
+        "joint_tp2_mns16",
+        "--metrics_config_write_metrics",
+        "--metrics_config_store_request_metrics",
+        "--metrics_config_store_batch_metrics",
+        "--metrics_config_store_token_completion_metrics",
+        "--metrics_config_store_utilization_metrics",
+        "--no-metrics_config_store_plots",
+        "--no-metrics_config_enable_chrome_trace",
+        "--no-metrics_config_write_json_trace",
+        "--metrics_config_store_frontier_stage_batch_ledger",
+        "--no-random_forrest_execution_time_predictor_config_enable_dummy_mode",
+        "--random_forrest_execution_time_predictor_config_linear_op_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/moe.csv",
+        "--random_forrest_execution_time_predictor_config_linear_op_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/moe.csv",
+        "--random_forrest_execution_time_predictor_config_prediction_max_prefill_chunk_size",
+        "8192",
+        "--random_forrest_execution_time_predictor_config_prediction_max_batch_size",
+        "32",
+        "--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request",
+        "40960",
+        "--random_forrest_execution_time_predictor_config_no_cache",
+        "--random_forrest_execution_time_predictor_config_skip_cpu_overhead_modeling",
+        "--vllm_v1_scheduler_config_num_blocks",
+        "76620",
+        "--vllm_v1_scheduler_config_enable_chunked_prefill",
+        "--random_forrest_execution_time_predictor_config_num_training_job_threads",
+        "4",
+        "--cudagraph_capture_sizes",
+        "1",
+        "2",
+        "4",
+        "8",
+        "16",
+        "24",
+        "32",
+        "--vidur_cc_backend_config_all_reduce_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/measured-allreduce.csv",
+        "--vidur_cc_backend_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/cc-cache",
+        "--vidur_cc_backend_config_k_fold_cv_splits",
+        "6",
+        "--vidur_cc_backend_config_num_training_job_threads",
+        "1",
+        "--metrics_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/model-cache"
+      ],
+      "log": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/logs/tp2_mns16.log",
+      "source_command": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp2_mns16/tp2/command.json",
+      "source_command_sha256": "61788a8810be301c9dbc006624aa19b6a932bc44d341b836861087833cffc3df",
+      "usage": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/usage/tp2_mns16.json"
+    },
+    "tp4_mns16": {
+      "argv": [
+        "/usr/bin/python3",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/run_frontier_with_curves.py",
+        "--simulation_mode",
+        "online",
+        "--sys_arch",
+        "co-location",
+        "--cc_backend_config_type",
+        "vidur",
+        "--cluster_config_num_replicas",
+        "1",
+        "--cluster_scheduler_config_type",
+        "sticky_round_robin",
+        "--replica_config_model_name",
+        "qwen3-a3b-30b-moe",
+        "--replica_config_device",
+        "h20",
+        "--replica_config_network_device",
+        "h20_dgx",
+        "--replica_config_attn_tensor_parallel_size",
+        "4",
+        "--replica_config_attn_data_parallel_size",
+        "1",
+        "--replica_config_moe_tensor_parallel_size",
+        "4",
+        "--replica_config_moe_expert_parallel_size",
+        "1",
+        "--replica_config_num_pipeline_stages",
+        "1",
+        "--replica_scheduler_config_type",
+        "vllm_v1",
+        "--decode_cuda_graph_mode",
+        "piecewise",
+        "--vllm_v1_scheduler_config_batch_size_cap",
+        "16",
+        "--vllm_v1_scheduler_config_max_tokens_in_batch",
+        "8192",
+        "--vllm_v1_scheduler_config_long_prefill_token_threshold",
+        "0",
+        "--vllm_v1_scheduler_config_block_size",
+        "16",
+        "--vllm_v1_scheduler_config_num_blocks_mode",
+        "explicit",
+        "--vllm_v1_scheduler_config_gpu_memory_utilization",
+        "0.92",
+        "--vllm_v1_scheduler_config_non_kv_cache_overhead_bytes",
+        "0",
+        "--request_generator_config_type",
+        "trace_replay",
+        "--trace_request_generator_config_trace_file",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp4-frontier.csv",
+        "--trace_request_generator_config_max_tokens",
+        "40960",
+        "--metrics_config_output_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/sim/tp4_mns16/metrics",
+        "--metrics_config_run_id",
+        "joint_tp4_mns16",
+        "--metrics_config_write_metrics",
+        "--metrics_config_store_request_metrics",
+        "--metrics_config_store_batch_metrics",
+        "--metrics_config_store_token_completion_metrics",
+        "--metrics_config_store_utilization_metrics",
+        "--no-metrics_config_store_plots",
+        "--no-metrics_config_enable_chrome_trace",
+        "--no-metrics_config_write_json_trace",
+        "--metrics_config_store_frontier_stage_batch_ledger",
+        "--no-random_forrest_execution_time_predictor_config_enable_dummy_mode",
+        "--random_forrest_execution_time_predictor_config_linear_op_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/moe.csv",
+        "--random_forrest_execution_time_predictor_config_linear_op_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/moe.csv",
+        "--random_forrest_execution_time_predictor_config_prediction_max_prefill_chunk_size",
+        "8192",
+        "--random_forrest_execution_time_predictor_config_prediction_max_batch_size",
+        "32",
+        "--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request",
+        "40960",
+        "--random_forrest_execution_time_predictor_config_no_cache",
+        "--random_forrest_execution_time_predictor_config_skip_cpu_overhead_modeling",
+        "--vllm_v1_scheduler_config_num_blocks",
+        "191882",
+        "--vllm_v1_scheduler_config_enable_chunked_prefill",
+        "--random_forrest_execution_time_predictor_config_num_training_job_threads",
+        "4",
+        "--cudagraph_capture_sizes",
+        "1",
+        "2",
+        "4",
+        "8",
+        "16",
+        "24",
+        "32",
+        "--vidur_cc_backend_config_all_reduce_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/measured-allreduce.csv",
+        "--vidur_cc_backend_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/cc-cache",
+        "--vidur_cc_backend_config_k_fold_cv_splits",
+        "6",
+        "--vidur_cc_backend_config_num_training_job_threads",
+        "1",
+        "--metrics_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/model-cache"
+      ],
+      "log": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/logs/tp4_mns16.log",
+      "source_command": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp4_mns16/tp4/command.json",
+      "source_command_sha256": "9bbcf10446336ba5885193f391dd628cd18ff64d91a51ebcd463ffd24be95532",
+      "usage": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/usage/tp4_mns16.json"
+    },
+    "tp4_mns32": {
+      "argv": [
+        "/usr/bin/python3",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/run_frontier_with_curves.py",
+        "--simulation_mode",
+        "online",
+        "--sys_arch",
+        "co-location",
+        "--cc_backend_config_type",
+        "vidur",
+        "--cluster_config_num_replicas",
+        "1",
+        "--cluster_scheduler_config_type",
+        "sticky_round_robin",
+        "--replica_config_model_name",
+        "qwen3-a3b-30b-moe",
+        "--replica_config_device",
+        "h20",
+        "--replica_config_network_device",
+        "h20_dgx",
+        "--replica_config_attn_tensor_parallel_size",
+        "4",
+        "--replica_config_attn_data_parallel_size",
+        "1",
+        "--replica_config_moe_tensor_parallel_size",
+        "4",
+        "--replica_config_moe_expert_parallel_size",
+        "1",
+        "--replica_config_num_pipeline_stages",
+        "1",
+        "--replica_scheduler_config_type",
+        "vllm_v1",
+        "--decode_cuda_graph_mode",
+        "piecewise",
+        "--vllm_v1_scheduler_config_batch_size_cap",
+        "32",
+        "--vllm_v1_scheduler_config_max_tokens_in_batch",
+        "8192",
+        "--vllm_v1_scheduler_config_long_prefill_token_threshold",
+        "0",
+        "--vllm_v1_scheduler_config_block_size",
+        "16",
+        "--vllm_v1_scheduler_config_num_blocks_mode",
+        "explicit",
+        "--vllm_v1_scheduler_config_gpu_memory_utilization",
+        "0.92",
+        "--vllm_v1_scheduler_config_non_kv_cache_overhead_bytes",
+        "0",
+        "--request_generator_config_type",
+        "trace_replay",
+        "--trace_request_generator_config_trace_file",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp4-frontier.csv",
+        "--trace_request_generator_config_max_tokens",
+        "40960",
+        "--metrics_config_output_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/sim/tp4_mns32/metrics",
+        "--metrics_config_run_id",
+        "joint_tp4_mns32",
+        "--metrics_config_write_metrics",
+        "--metrics_config_store_request_metrics",
+        "--metrics_config_store_batch_metrics",
+        "--metrics_config_store_token_completion_metrics",
+        "--metrics_config_store_utilization_metrics",
+        "--no-metrics_config_store_plots",
+        "--no-metrics_config_enable_chrome_trace",
+        "--no-metrics_config_write_json_trace",
+        "--metrics_config_store_frontier_stage_batch_ledger",
+        "--no-random_forrest_execution_time_predictor_config_enable_dummy_mode",
+        "--random_forrest_execution_time_predictor_config_linear_op_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/moe.csv",
+        "--random_forrest_execution_time_predictor_config_linear_op_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/linear_op.csv",
+        "--random_forrest_execution_time_predictor_config_atten_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/attention.csv",
+        "--random_forrest_execution_time_predictor_config_moe_kernel_only_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/frozen-kernel-only/moe.csv",
+        "--random_forrest_execution_time_predictor_config_prediction_max_prefill_chunk_size",
+        "8192",
+        "--random_forrest_execution_time_predictor_config_prediction_max_batch_size",
+        "64",
+        "--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request",
+        "40960",
+        "--random_forrest_execution_time_predictor_config_no_cache",
+        "--random_forrest_execution_time_predictor_config_skip_cpu_overhead_modeling",
+        "--vllm_v1_scheduler_config_num_blocks",
+        "191786",
+        "--vllm_v1_scheduler_config_enable_chunked_prefill",
+        "--random_forrest_execution_time_predictor_config_num_training_job_threads",
+        "4",
+        "--cudagraph_capture_sizes",
+        "1",
+        "2",
+        "4",
+        "8",
+        "16",
+        "24",
+        "32",
+        "40",
+        "48",
+        "56",
+        "64",
+        "--vidur_cc_backend_config_all_reduce_input_file",
+        "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/measured-allreduce.csv",
+        "--vidur_cc_backend_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/cc-cache",
+        "--vidur_cc_backend_config_k_fold_cv_splits",
+        "6",
+        "--vidur_cc_backend_config_num_training_job_threads",
+        "1",
+        "--metrics_config_cache_dir",
+        "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/model-cache"
+      ],
+      "log": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/logs/tp4_mns32.log",
+      "source_command": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp4_mns32/tp4/command.json",
+      "source_command_sha256": "fbc7dee55590b415ed1cde8072de835ed155a0c20ba0eb305c3cb22aa8065a51",
+      "usage": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/usage/tp4_mns32.json"
+    }
+  },
+  "collective_curve": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/results/collective-curve.json",
+  "collective_curve_sha256": "f9543649d4ea78f08240bf1284ab74083aa5cf5671ed47e386047f1453300b36",
+  "collective_curve_variant": "drop_mean",
+  "frontier_checkout": "/tmp/frontier-attn-structured-v0",
+  "frontier_commit": "1f8900a4ac64e45754b03d0aa7c1dddab65785cf",
+  "mode": "joint",
+  "model_cache": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/model-cache",
+  "moe_curve": "/home/gahow/phd/aituner/runs/frontier-fused-moe-profile-v0/results/fused-moe-curve.json",
+  "moe_curve_sha256": "b94d65d9d581adefcc6c14ed4920cce6a1136f74f1014737dc3e4249bc8250d2",
+  "python": "/usr/bin/python3",
+  "python_dependency_roots": [
+    "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/python-deps",
+    "/home/gahow/.cache/uv/archive-v0/-_kzErLcPO5nASZFX8b9k",
+    "/home/gahow/.cache/uv/archive-v0/FbaBs_QJ9QKEbQ9V_4aIR",
+    "/home/gahow/.cache/uv/archive-v0/fuHsGXD0Lv_UjFC8yI4-7",
+    "/home/gahow/.cache/uv/archive-v0/jFGdqQLpB1eopfm9VxT3j",
+    "/home/gahow/.cache/uv/archive-v0/YWW6ExSJuPVvv4-qYQTin",
+    "/home/gahow/.cache/uv/archive-v0/3_qxZ5Ll-EpVAGZfbksfe"
+  ],
+  "traces": {
+    "1": {
+      "first_arrival_s": 0.0,
+      "last_arrival_s": 595.348837209302,
+      "requests": 129,
+      "source_request_metrics": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp1_mns16/tp1/metrics/qwen3_a3b_30b_moe/online_serving/qwen30_trace_tp1_mns16_tp1/request_metrics.csv",
+      "source_sha256": "0b82e09644a5884fcd10d894b68495daefdabb32b770146c2f9ece37b8469f4f",
+      "trace": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp1-frontier.csv",
+      "trace_sha256": "59dd8996ff879ef94330004104dfdf515b791bce4036576eccc93290e9206dad"
+    },
+    "2": {
+      "first_arrival_s": 0.0,
+      "last_arrival_s": 297.674418604651,
+      "requests": 129,
+      "source_request_metrics": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp2_mns16/tp2/metrics/qwen3_a3b_30b_moe/online_serving/qwen30_trace_tp2_mns16_tp2/request_metrics.csv",
+      "source_sha256": "33983081bb20dd5e2053e9e3d13def8732e958150c9b47a8609ba345123f2316",
+      "trace": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp2-frontier.csv",
+      "trace_sha256": "64fc077b38274a76a8279884ac4115836cd1157c95119c64fabac50d81124f69"
+    },
+    "4": {
+      "first_arrival_s": 0.0,
+      "last_arrival_s": 148.837209302326,
+      "requests": 129,
+      "source_request_metrics": "/home/gahow/phd/aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q30-lo-fixed-pd-cells/sim/fixed-pd/runs/tp4_mns16/tp4/metrics/qwen3_a3b_30b_moe/online_serving/qwen30_trace_tp4_mns16_tp4/request_metrics.csv",
+      "source_sha256": "b36cd383c07b546d2c1f2fac754d5dbb92efd6880316b4233a7aef9fa1115a36",
+      "trace": "/home/gahow/phd/aituner/runs/frontier-collective-joint-v0/counterfactual/joint-r2/inputs/tp4-frontier.csv",
+      "trace_sha256": "adb3d6f3932a44c86c3d9e7cf1e57739594e8c48d19a26b5aa54b37dce0e0c19"
+    }
+  }
+}

runs/frontier-attn-structured-v0/plot_figure_prototype.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+"""Schematic figure frozen before EXP-ATTN-STRUCTURED execution."""
+
+from pathlib import Path
+
+import matplotlib
+
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import numpy as np
+
+ROOT = Path(__file__).resolve().parent
+SURFACE = "#fcfcfb"
+INK = "#111111"
+MUTED = "#77736c"
+GRID = "#dedbd2"
+RF = "#d95f02"
+STRUCTURED = "#1b75bc"
+
+fig, axes = plt.subplots(1, 2, figsize=(10.8, 4.2), dpi=160)
+fig.patch.set_facecolor(SURFACE)
+
+for ax in axes:
+    ax.set_facecolor(SURFACE)
+    ax.grid(axis="y", color=GRID, linewidth=0.8)
+    ax.set_axisbelow(True)
+    ax.spines[["top", "right"]].set_visible(False)
+    ax.tick_params(colors=MUTED, labelsize=8)
+
+kv = np.array([8, 16, 24], dtype=float)
+actual = np.array([11.97, 19.92, 27.84])
+rf = np.array([9.46, 16.40, 24.52])
+structured_expected = np.array([12.0, 19.9, 27.9])
+axes[0].plot(kv, actual, "o-", color=INK, label="profile actual")
+axes[0].plot(kv, rf, "s--", color=RF, label="current RF")
+axes[0].plot(
+    kv,
+    structured_expected,
+    "^:",
+    color=STRUCTURED,
+    label="structured (expected)",
+)
+axes[0].set_xlabel("KV context (ktok)")
+axes[0].set_ylabel("TP1 q8k attention time (ms)")
+axes[0].set_title("(a) Continuous KV growth", loc="left", fontsize=10)
+axes[0].legend(frameon=False, fontsize=8)
+
+labels = ["TP1\n.00125", "TP1\n.0025", "TP2\n.0025", "TP2\n.005",
+          "TP4\n.0025", "TP4\n.005", "TP4\n.01"]
+x = np.arange(len(labels))
+v5_mean = np.array([-13.5, -17.7, -11.2, -14.1, 2.5, 2.9, -0.2])
+expected = np.array([-5, -8, -9, -11, 3, 3, 0])
+axes[1].axhspan(-15, 15, color=GRID, alpha=0.5)
+axes[1].axhline(0, color=MUTED, linewidth=0.8)
+axes[1].plot(x, v5_mean, "o-", color=RF, label="sim-v5 measured")
+axes[1].plot(x, expected, "s--", color=STRUCTURED, label="H1 expected")
+axes[1].set_xticks(x, labels)
+axes[1].set_ylabel("TTFT mean bias (%)")
+axes[1].set_title("(b) 7-cell trace gate", loc="left", fontsize=10)
+axes[1].legend(frameon=False, fontsize=8)
+
+fig.suptitle(
+    "MOCK / schematic — EXP-ATTN-STRUCTURED (not measured results)",
+    x=0.01,
+    ha="left",
+    color=RF,
+    fontsize=9,
+)
+fig.tight_layout(rect=(0, 0, 1, 0.95))
+fig.savefig(ROOT / "figure-prototype.png", facecolor=SURFACE)
+fig.savefig(ROOT / "figure-prototype.svg", facecolor=SURFACE)
+print(ROOT / "figure-prototype.png")

runs/frontier-attn-structured-v0/results/predictor-ablation.csv

@@ -0,0 +1,10 @@
+candidate,training_rows,grid_fit_mape_pct,single_fit_mape_pct,heldout_context_mape_pct,heldout_context_max_abs_error_pct,prediction_min_ms,q_min_delta_ms,kv_min_delta_ms,monotone_and_nonnegative,tp
+rf_all,29,15.074275515235467,19.842994757611535,44.40139318281943,82.78279487156401,0.06029164119272453,0.0,-4.2841601371801374e-05,False,1
+rf_single,23,11.834992526698676,22.893412972496023,34.45356428541224,62.49334437588834,0.059327708247725125,-0.00022153525203457564,-1.4336001873005433e-05,False,1
+structured_single,23,0.8432511364168856,2.227110646811972,0.841926169535806,2.0408978739639134,0.05679146709541477,0.0,0.0016745062683911627,True,1
+rf_all,29,13.812689689573157,17.879870012606048,44.20143320278334,82.0656368501208,0.06000113548192927,-0.004361070463210395,-0.009949388915300408,False,2
+rf_single,23,10.738962684891058,20.818820413545826,34.84360402100271,66.67253880294443,0.059971319361210015,-0.003811210796127021,-0.009949388915300408,False,2
+structured_single,23,1.518412258922364,6.226292654776418,1.6047958673086566,3.464671475193195,0.05767893331746252,0.0,0.0013929374121726124,True,2
+rf_all,29,14.117015331381916,15.49366794487052,43.53650868837558,81.33508178007524,0.05866772018640992,-0.0009967416035880083,-5.5955198407176e-05,False,4
+rf_single,23,12.07517666989496,17.672692652721008,34.14562332866605,62.31478818862995,0.058213693721655094,-0.0012244979345549661,-8.259841203689389e-05,False,4
+structured_single,23,3.088740104976349,5.969109476280061,3.0103379904473164,5.490598706238697,0.05747733327249683,0.0,0.0012048051417407057,True,4

runs/frontier-attn-structured-v0/results/predictor-ablation.json

@@ -0,0 +1,149 @@
+{
+  "schema": "frontier-attn-structured-ablation-v1",
+  "profile": "/home/gahow/phd/aituner/runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/attention.csv",
+  "frontier_checkout": "/tmp/frontier-attn-structured-v0",
+  "data_audit": {
+    "tp1": {
+      "standard_rows": 29,
+      "single_request_rows": 23,
+      "target_grid_rows": 10,
+      "duplicate_feature_groups": 4
+    },
+    "tp2": {
+      "standard_rows": 29,
+      "single_request_rows": 23,
+      "target_grid_rows": 10,
+      "duplicate_feature_groups": 4
+    },
+    "tp4": {
+      "standard_rows": 29,
+      "single_request_rows": 23,
+      "target_grid_rows": 10,
+      "duplicate_feature_groups": 4
+    }
+  },
+  "results": [
+    {
+      "candidate": "rf_all",
+      "training_rows": 29,
+      "grid_fit_mape_pct": 15.074275515235467,
+      "single_fit_mape_pct": 19.842994757611535,
+      "heldout_context_mape_pct": 44.40139318281943,
+      "heldout_context_max_abs_error_pct": 82.78279487156401,
+      "prediction_min_ms": 0.06029164119272453,
+      "q_min_delta_ms": 0.0,
+      "kv_min_delta_ms": -4.2841601371801374e-05,
+      "monotone_and_nonnegative": false,
+      "tp": 1
+    },
+    {
+      "candidate": "rf_single",
+      "training_rows": 23,
+      "grid_fit_mape_pct": 11.834992526698676,
+      "single_fit_mape_pct": 22.893412972496023,
+      "heldout_context_mape_pct": 34.45356428541224,
+      "heldout_context_max_abs_error_pct": 62.49334437588834,
+      "prediction_min_ms": 0.059327708247725125,
+      "q_min_delta_ms": -0.00022153525203457564,
+      "kv_min_delta_ms": -1.4336001873005433e-05,
+      "monotone_and_nonnegative": false,
+      "tp": 1
+    },
+    {
+      "candidate": "structured_single",
+      "training_rows": 23,
+      "grid_fit_mape_pct": 0.8432511364168856,
+      "single_fit_mape_pct": 2.227110646811972,
+      "heldout_context_mape_pct": 0.841926169535806,
+      "heldout_context_max_abs_error_pct": 2.0408978739639134,
+      "prediction_min_ms": 0.05679146709541477,
+      "q_min_delta_ms": 0.0,
+      "kv_min_delta_ms": 0.0016745062683911627,
+      "monotone_and_nonnegative": true,
+      "tp": 1
+    },
+    {
+      "candidate": "rf_all",
+      "training_rows": 29,
+      "grid_fit_mape_pct": 13.812689689573157,
+      "single_fit_mape_pct": 17.879870012606048,
+      "heldout_context_mape_pct": 44.20143320278334,
+      "heldout_context_max_abs_error_pct": 82.0656368501208,
+      "prediction_min_ms": 0.06000113548192927,
+      "q_min_delta_ms": -0.004361070463210395,
+      "kv_min_delta_ms": -0.009949388915300408,
+      "monotone_and_nonnegative": false,
+      "tp": 2
+    },
+    {
+      "candidate": "rf_single",
+      "training_rows": 23,
+      "grid_fit_mape_pct": 10.738962684891058,
+      "single_fit_mape_pct": 20.818820413545826,
+      "heldout_context_mape_pct": 34.84360402100271,
+      "heldout_context_max_abs_error_pct": 66.67253880294443,
+      "prediction_min_ms": 0.059971319361210015,
+      "q_min_delta_ms": -0.003811210796127021,
+      "kv_min_delta_ms": -0.009949388915300408,
+      "monotone_and_nonnegative": false,
+      "tp": 2
+    },
+    {
+      "candidate": "structured_single",
+      "training_rows": 23,
+      "grid_fit_mape_pct": 1.518412258922364,
+      "single_fit_mape_pct": 6.226292654776418,
+      "heldout_context_mape_pct": 1.6047958673086566,
+      "heldout_context_max_abs_error_pct": 3.464671475193195,
+      "prediction_min_ms": 0.05767893331746252,
+      "q_min_delta_ms": 0.0,
+      "kv_min_delta_ms": 0.0013929374121726124,
+      "monotone_and_nonnegative": true,
+      "tp": 2
+    },
+    {
+      "candidate": "rf_all",
+      "training_rows": 29,
+      "grid_fit_mape_pct": 14.117015331381916,
+      "single_fit_mape_pct": 15.49366794487052,
+      "heldout_context_mape_pct": 43.53650868837558,
+      "heldout_context_max_abs_error_pct": 81.33508178007524,
+      "prediction_min_ms": 0.05866772018640992,
+      "q_min_delta_ms": -0.0009967416035880083,
+      "kv_min_delta_ms": -5.5955198407176e-05,
+      "monotone_and_nonnegative": false,
+      "tp": 4
+    },
+    {
+      "candidate": "rf_single",
+      "training_rows": 23,
+      "grid_fit_mape_pct": 12.07517666989496,
+      "single_fit_mape_pct": 17.672692652721008,
+      "heldout_context_mape_pct": 34.14562332866605,
+      "heldout_context_max_abs_error_pct": 62.31478818862995,
+      "prediction_min_ms": 0.058213693721655094,
+      "q_min_delta_ms": -0.0012244979345549661,
+      "kv_min_delta_ms": -8.259841203689389e-05,
+      "monotone_and_nonnegative": false,
+      "tp": 4
+    },
+    {
+      "candidate": "structured_single",
+      "training_rows": 23,
+      "grid_fit_mape_pct": 3.088740104976349,
+      "single_fit_mape_pct": 5.969109476280061,
+      "heldout_context_mape_pct": 3.0103379904473164,
+      "heldout_context_max_abs_error_pct": 5.490598706238697,
+      "prediction_min_ms": 0.05747733327249683,
+      "q_min_delta_ms": 0.0,
+      "kv_min_delta_ms": 0.0012048051417407057,
+      "monotone_and_nonnegative": true,
+      "tp": 4
+    }
+  ],
+  "checks": {
+    "heldout_context_mape_le_5pct": true,
+    "monotone_and_nonnegative": true,
+    "profile_gate": true
+  }
+}

runs/frontier-attn-structured-v0/results/trace-verdict.csv

@@ -0,0 +1,85 @@
+cell,metric,quantile,old_bias,new_bias,abs_bias_delta_pp,validity
+tp1_rho0p00125,ttft,mean,-0.13461915993830945,-0.06328385143753473,-7.133530850077471,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,ttft,p50,-0.1888032883165517,-0.1117698463137391,-7.7033442002812595,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,ttft,p90,-0.23223319000679374,-0.0884727580604037,-14.376043194639005,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,ttft,p99,-0.16824856840168442,-0.08585570369326061,-8.239286470842382,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,tpot,mean,0.1305653876178269,0.14303898259628217,1.2473594978455265,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,tpot,p50,0.16837991778063455,0.1688780144991309,0.04980967184963492,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,tpot,p90,0.012855564422932954,0.02293841255516459,1.0082848132231637,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,tpot,p99,-0.08151080694091946,-0.06480973978913974,-1.6701067151779714,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,e2e,mean,0.06539157436639341,0.0838094906426525,1.8417916276259092,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,e2e,p50,0.10867934171755954,0.11142188099086506,0.2742539273305519,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,e2e,p90,0.10946179160934073,0.11872607178176105,0.9264280172420314,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p00125,e2e,p99,-0.0657146472433028,-0.04585643943465822,-1.9858207808644577,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,ttft,mean,-0.17677087458166194,-0.09487906467830536,-8.189180990335657,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,ttft,p50,0.008311436147272566,0.015508635458377175,0.7197199311104608,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,ttft,p90,-0.2608553178054708,-0.129865645852306,-13.098967195316478,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,ttft,p99,-0.22116442296103195,-0.14410151018893788,-7.706291277209407,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,tpot,mean,0.014380733682430142,0.052302860177306544,3.7922126494876403,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,tpot,p50,0.13351145963877706,0.14244154194999165,0.8930082311214588,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,tpot,p90,-0.05456950130438105,0.018069551387063856,-3.649994991731719,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,tpot,p99,-0.23635406197836167,-0.1790206784227079,-5.733338355565376,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,e2e,mean,-0.020164189983441452,0.01563919326555167,-0.4524996717889782,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,e2e,p50,0.07632815851795742,0.10505951594320918,2.8731357425251765,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,e2e,p90,0.049809009042946335,0.08165607475145953,3.1847065708513194,GATE_FAIL_DIAGNOSTIC
+tp1_rho0p0025,e2e,p99,-0.18303183751478602,-0.15333409130015813,-2.96977462146279,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p0025,ttft,mean,-0.11161154024124531,-0.045433491316312524,-6.617804892493279,PASS_SUBCRITICAL
+tp2_rho0p0025,ttft,p50,-0.18420934047220774,-0.0875990583320057,-9.661028214020204,PASS_SUBCRITICAL
+tp2_rho0p0025,ttft,p90,-0.21836264490202395,-0.1142200855839675,-10.414255931805645,PASS_SUBCRITICAL
+tp2_rho0p0025,ttft,p99,-0.17283197594971011,-0.07674087497700505,-9.609110097270507,PASS_SUBCRITICAL
+tp2_rho0p0025,tpot,mean,0.13711499081487563,0.15073709978689778,1.362210897202215,PASS_SUBCRITICAL
+tp2_rho0p0025,tpot,p50,0.17555321305308488,0.18527285925405948,0.9719646200974597,PASS_SUBCRITICAL
+tp2_rho0p0025,tpot,p90,0.0591579975137338,0.07863996768588354,1.9481970172149734,PASS_SUBCRITICAL
+tp2_rho0p0025,tpot,p99,0.1300483675091633,0.1661995397125918,3.6151172203428503,PASS_SUBCRITICAL
+tp2_rho0p0025,e2e,mean,0.10925353865257875,0.12696146154030977,1.7707922887731016,PASS_SUBCRITICAL
+tp2_rho0p0025,e2e,p50,0.1339165600755408,0.14796251184179712,1.404595176625631,PASS_SUBCRITICAL
+tp2_rho0p0025,e2e,p90,0.10666828724664539,0.12767235986604622,2.1004072619400835,PASS_SUBCRITICAL
+tp2_rho0p0025,e2e,p99,-0.07437942975605877,-0.03965040123988098,-3.472902851617779,PASS_SUBCRITICAL
+tp2_rho0p005,ttft,mean,-0.14129871969878843,-0.07082500585057615,-7.047371384821228,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,ttft,p50,-0.2042045530944649,-0.1284088888361358,-7.579566425832909,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,ttft,p90,-0.19003454588767263,-0.111136573344055,-7.889797254361763,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,ttft,p99,-0.19373351009539902,-0.09012556161973535,-10.360794847566366,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,tpot,mean,0.03117337438562004,0.05457336599027876,2.3399991604658723,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,tpot,p50,0.07668249597302182,0.0872512146277895,1.0568718654767675,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,tpot,p90,-0.03724827658429621,0.01132352325010614,-2.5924753334190074,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,tpot,p99,-0.10050436157089844,-0.05948329733667248,-4.102106423422596,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,e2e,mean,0.023867807624916495,0.050164236759908075,2.629642913499158,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,e2e,p50,0.07536668131278851,0.0912493994395978,1.588271812680929,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,e2e,p90,-0.03100470462321266,-0.0004979191794830456,-3.0506785443729614,GATE_FAIL_DIAGNOSTIC
+tp2_rho0p005,e2e,p99,-0.04909581632382212,-0.0014369075488634014,-4.765890877495872,GATE_FAIL_DIAGNOSTIC
+tp4_rho0p0025,ttft,mean,0.025022574738277282,0.07766646565061346,5.2643890912336175,PASS_SUBCRITICAL
+tp4_rho0p0025,ttft,p50,-0.040042171846277425,0.029153379043297147,-1.0888792802980278,PASS_SUBCRITICAL
+tp4_rho0p0025,ttft,p90,-0.056230097634382616,0.025222989748299444,-3.100710788608317,PASS_SUBCRITICAL
+tp4_rho0p0025,ttft,p99,-0.07535478089127973,0.0205549324689376,-5.479984842234213,PASS_SUBCRITICAL
+tp4_rho0p0025,tpot,mean,0.2170232618103144,0.2295170016795841,1.2493739869269715,PASS_SUBCRITICAL
+tp4_rho0p0025,tpot,p50,0.22406751004936917,0.22406940610958842,0.00018960602192474862,PASS_SUBCRITICAL
+tp4_rho0p0025,tpot,p90,0.1725668492041552,0.1795965483385546,0.7029699134399409,PASS_SUBCRITICAL
+tp4_rho0p0025,tpot,p99,0.1603764334794579,0.254199450927048,9.382301744759008,PASS_SUBCRITICAL
+tp4_rho0p0025,e2e,mean,0.18328079455378776,0.19190660476104554,0.8625810207257778,PASS_SUBCRITICAL
+tp4_rho0p0025,e2e,p50,0.2057109615696404,0.21253242300694286,0.6821461437302473,PASS_SUBCRITICAL
+tp4_rho0p0025,e2e,p90,0.1869703879211648,0.19279855916666536,0.5828171245500557,PASS_SUBCRITICAL
+tp4_rho0p0025,e2e,p99,0.14838304065885655,0.15160588346499027,0.3222842806133719,PASS_SUBCRITICAL
+tp4_rho0p005,ttft,mean,0.028648879997638963,0.08356973519379125,5.492085519615229,PASS_SUBCRITICAL
+tp4_rho0p005,ttft,p50,0.028237979190582876,0.09021324737193111,6.197526818134823,PASS_SUBCRITICAL
+tp4_rho0p005,ttft,p90,-0.055961238578361966,-0.0006012940489499138,-5.535994452941205,PASS_SUBCRITICAL
+tp4_rho0p005,ttft,p99,-0.045491187615110146,0.05253708684194205,0.7045899226831902,PASS_SUBCRITICAL
+tp4_rho0p005,tpot,mean,0.1707193936623511,0.1813506819061229,1.0631288243771824,PASS_SUBCRITICAL
+tp4_rho0p005,tpot,p50,0.16851374859025317,0.1743324539248605,0.5818705334607321,PASS_SUBCRITICAL
+tp4_rho0p005,tpot,p90,0.10492621353626864,0.11816822907155744,1.3242015535288796,PASS_SUBCRITICAL
+tp4_rho0p005,tpot,p99,0.31231888769471766,0.3662267201704473,5.390783247572961,PASS_SUBCRITICAL
+tp4_rho0p005,e2e,mean,0.151102823467785,0.1630793421767109,1.197651870892591,PASS_SUBCRITICAL
+tp4_rho0p005,e2e,p50,0.1594213531394918,0.17360527090575292,1.418391776626113,PASS_SUBCRITICAL
+tp4_rho0p005,e2e,p90,0.1266352410718406,0.13652104764058856,0.9885806568747962,PASS_SUBCRITICAL
+tp4_rho0p005,e2e,p99,0.15576537699445703,0.17677450343779522,2.100912644333819,PASS_SUBCRITICAL
+tp4_rho0p01,ttft,mean,-0.0014652143973501086,0.059650620031540064,5.8185405634189955,PASS_SUBCRITICAL
+tp4_rho0p01,ttft,p50,0.22936601881498542,0.24159106387124998,1.2225045056264565,PASS_SUBCRITICAL
+tp4_rho0p01,ttft,p90,-0.0646093465409875,-0.011548419893895705,-5.30609266470918,PASS_SUBCRITICAL
+tp4_rho0p01,ttft,p99,-0.09621678853313553,-0.015475807392170575,-8.074098114096495,PASS_SUBCRITICAL
+tp4_rho0p01,tpot,mean,0.06675609059150077,0.10596101263201793,3.9204922040517163,PASS_SUBCRITICAL
+tp4_rho0p01,tpot,p50,0.08477302587551214,0.09703754188844527,1.2264516012933129,PASS_SUBCRITICAL
+tp4_rho0p01,tpot,p90,0.0379183273767328,0.08369800201750718,4.577967464077439,PASS_SUBCRITICAL
+tp4_rho0p01,tpot,p99,-0.010663954751357074,0.06667813160571406,5.601417685435699,PASS_SUBCRITICAL
+tp4_rho0p01,e2e,mean,0.07212904317306096,0.09777288053702092,2.564383736395996,PASS_SUBCRITICAL
+tp4_rho0p01,e2e,p50,0.12069215463307655,0.1388377217196832,1.8145567086606653,PASS_SUBCRITICAL
+tp4_rho0p01,e2e,p90,0.03317293927357903,0.058060760567474216,2.4887821293895183,PASS_SUBCRITICAL
+tp4_rho0p01,e2e,p99,0.0338378271163308,0.06380410696893425,2.996627985260345,PASS_SUBCRITICAL

runs/frontier-attn-structured-v0/run_replay.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3
+"""Replay one real-trace cell with the structured-attention experiment commit."""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent
+REPO = ROOT.parents[1]
+S3_REAL = REPO / "runs/frontier-s3-real-v0"
+BASE_REFERENCE = (
+    REPO
+    / "runs/frontier-collective-joint-v0/counterfactual/joint-r2/manifest.json"
+)
+BASE_COMMIT = "deadc4a321f0baaa534c6ebd17f974123733cdc2"
+EXPERIMENT_COMMIT = "1f8900a4ac64e45754b03d0aa7c1dddab65785cf"
+PATCH = ROOT / "0001-Experiment-with-structured-attention-prefill-predict.patch"
+
+
+def load_s3_module():
+    spec = importlib.util.spec_from_file_location(
+        "s3_prefix_replay", S3_REAL / "run_frontier_prefix_replay.py"
+    )
+    module = importlib.util.module_from_spec(spec)
+    sys.path.insert(0, str(S3_REAL))
+    spec.loader.exec_module(module)
+    return module
+
+
+def git(checkout: Path, *args: str) -> str:
+    return subprocess.check_output(
+        ["git", "-C", str(checkout), *args], text=True
+    ).strip()
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--trace", type=Path, required=True)
+    parser.add_argument("--output-root", type=Path, required=True)
+    parser.add_argument(
+        "--config",
+        choices=("tp4_mns16", "tp2_mns16", "tp1_mns16"),
+        required=True,
+    )
+    parser.add_argument("--label", required=True)
+    parser.add_argument("--max-tokens", type=int, required=True)
+    parser.add_argument("--duration-s", type=float)
+    parser.add_argument("--cache-root", type=Path, required=True)
+    parser.add_argument(
+        "--frontier-checkout",
+        type=Path,
+        default=Path("/tmp/frontier-attn-structured-v0"),
+    )
+    parser.add_argument(
+        "--attention-profile",
+        type=Path,
+        default=REPO
+        / "runs/frontier-prefill-kvgrowth-fix-v0/profiles/"
+        "profile-v5-kvgrowth/attention.csv",
+    )
+    args = parser.parse_args()
+
+    frontier = args.frontier_checkout.resolve()
+    profile = args.attention_profile.resolve()
+    if git(frontier, "rev-parse", "HEAD") != EXPERIMENT_COMMIT:
+        raise SystemExit(f"unexpected experiment checkout HEAD: {frontier}")
+    if git(frontier, "rev-parse", "HEAD^") != BASE_COMMIT:
+        raise SystemExit("experiment commit is not directly based on frozen Frontier")
+    if git(frontier, "status", "--porcelain"):
+        raise SystemExit("experiment Frontier checkout must be clean")
+    if not profile.is_file():
+        raise SystemExit(f"attention profile missing: {profile}")
+
+    reference = json.loads(BASE_REFERENCE.read_text())
+    reference["frontier_checkout"] = str(frontier)
+    reference["frontier_commit"] = EXPERIMENT_COMMIT
+    generated_reference = ROOT / "frontier-reference.json"
+    generated_reference.write_text(json.dumps(reference, indent=2))
+
+    module = load_s3_module()
+    module.REFERENCE = generated_reference
+    module.EXPECTED_FRONTIER_COMMIT = EXPERIMENT_COMMIT
+    original_replace = module.replace_flag
+
+    def replace_and_override(argv: list[str], flag: str, value: str) -> None:
+        original_replace(argv, flag, value)
+        if flag.endswith("trace_file"):
+            atten_flag = (
+                "--random_forrest_execution_time_predictor_config_atten_input_file"
+            )
+            original_replace(argv, atten_flag, str(profile))
+            no_cache = (
+                "--random_forrest_execution_time_predictor_config_no_cache"
+            )
+            if no_cache in argv:
+                argv.remove(no_cache)
+
+    module.replace_flag = replace_and_override
+    module.parse_args = lambda: args
+    module.main()
+
+    manifest_path = args.output_root / "manifest.json"
+    manifest = json.loads(manifest_path.read_text())
+    manifest.update(
+        {
+            "schema": "frontier-attn-structured-replay-v1",
+            "frontier_base_commit": BASE_COMMIT,
+            "frontier_experiment_commit": EXPERIMENT_COMMIT,
+            "frontier_patch": str(PATCH.resolve()),
+            "frontier_patch_sha256": module.sha256(PATCH),
+            "attention_profile_override": str(profile),
+            "attention_profile_sha256": module.sha256(profile),
+            "model_cache_enabled": True,
+        }
+    )
+    manifest_path.write_text(json.dumps(manifest, indent=2))
+    print(f"structured replay done: {args.output_root}")
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/README.md

@@ -0,0 +1,153 @@
+# Frontier code-trace campaign handoff
+
+Phase A code prefill+decode 已进入 61min real matrix。data/profile、
+max-length、Frontier rho calibration 和 TP2/TP4 paired canary 均已完成；
+第一批 TP4 三个 load 与 TP2 low-rho diagnostic 正在 dash1–dash4 并行
+运行。code prefill-only 的独立 sim calibration 也已完成。
+
+完整设计与 gate 见 [`experiment-card.md`](experiment-card.md)。
+
+## 当前资产与下一步
+
+- development window：0513 `[3480,7140)`，61min；
+- held-out window：0529 `[2640,6240)`，只在 development 判据冻结后使用；
+- profile：`profiles/profile-v6-code-longctx/`，覆盖 TP1/2/4 和 131072
+  KV context；
+- full paired inputs（CPFS）：
+  `runs/frontier-code-trace-v0/inputs/full-r0p{0002,0004,0008,0016}-v1/`；
+- compact provenance：`results/calibration-summary.json`、
+  `results/prefill-only-calibration-summary.json`、
+  `results/canary-analysis-tp{2,4}-v*.json` 和
+  `results/paired-input-manifests/`；
+- 当前 A4 wave 1：TP4 `rho={0.0002,0.0008,0.0016}` trial 1，以及
+  TP2 `rho=0.0002` trial 1 diagnostic；
+- TP4 canary 的 TTFT/E2E、prefix hit 与 decode batch 通过；TP2
+  TTFT p90 低估 32.1%，因此 TP2 其余 cell 暂不扩展；
+- prefill-only：TP2 已冻结 `rho={0.0004,0.0008,0.0016}`；
+  TP4 到 `0.0032` 仍亚临界，需追加更高 rho 后冻结 near-knee。
+
+source trace 的远端位置是：
+
+```text
+/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/
+```
+
+以下命令保留为从 source 重新构建时的复现入口。
+
+## 1. 审计所有 1h+ code source
+
+在持有 trace 的机器、repo 根目录执行：
+
+```bash
+python3 runs/frontier-code-trace-v0/audit_code_trace.py \
+  --trace-root ~/ali-trace/trace-glm5.1-formatted \
+  --output runs/frontier-code-trace-v0/inputs/code-audit.json
+```
+
+如果目录里混有非 request JSONL，先只读列举文件，再用多个 `--source` 显式指定。审计输出必须满足：
+
+```text
+data_gate = PASS
+selected.hash_contract.exact_source_block_size != null
+max_model_len_recommendation != null
+selected.selected_window_stats.max_model_len_coverage[推荐值].coverage = 1.0
+```
+
+全量审计已确认 source block size=512；development source window 若 100%
+覆盖需要 262144，但正式 server cap 以 session-sampled paired cell 的实际
+`ISL+OSL max` 向上对齐，不能把 full-window 262144 无条件套到低 rho cell。
+审计会单独记录并排除 `input_length<=0` 或 `output_length<=0` 的 source
+行；这些行只有在 raw trace 同样显示 zero usage/empty response 时才按
+“未发生模型执行”处理，不能无记录过滤。
+
+## 2. 物化稳定窗口
+
+```bash
+python3 runs/frontier-code-trace-v0/prepare_code_window.py \
+  --audit runs/frontier-code-trace-v0/inputs/code-audit.json \
+  --output-root runs/frontier-code-trace-v0/inputs/code-window
+```
+
+输出是 60–75min `code-raw-window.jsonl` 和 manifest。source 文件不修改。
+
+## 3. 生成 P+D paired trace
+
+若没有 prompt sidecar，先生成 shape/prefix-faithful synthetic prompts：
+
+```bash
+python3 runs/frontier-s3-real-v0/remap_hash_blocks.py \
+  --input runs/frontier-code-trace-v0/inputs/code-window/code-raw-window.jsonl \
+  --output-root runs/frontier-code-trace-v0/inputs/code-pd-rho-max \
+  --source-block-size 512 \
+  --workload-mode prefill_decode \
+  --rho 1.0 \
+  --max-total-tokens 131072 \
+  --validate-parents
+```
+
+命令中的 `512` 和 `131072` 必须替换为 audit manifest 值。若存在对齐 prompt sidecar，加 `--prompt` 与 `--tokenizer`，并要求 synthetic fallback 为 0。
+
+正式 rho 不能直接用 1.0；先从最大 remap cache 按 session-coherent `sampling_u` 过滤，分别标定 low/mid/near-knee。
+
+## 4. 生成 prefill-only paired trace
+
+对 chat/code 使用同一个转换接口：
+
+```bash
+python3 runs/frontier-s3-real-v0/remap_hash_blocks.py \
+  --input INPUT_WINDOW.jsonl \
+  --output-root OUTPUT_ROOT \
+  --source-block-size SOURCE_BLOCK_SIZE \
+  --workload-mode prefill_only \
+  --rho RHO \
+  --max-total-tokens MAX_MODEL_LEN \
+  --validate-parents
+```
+
+该模式会同时把 Frontier `num_decode_tokens`、real request `min/max_tokens` 和 remapped row 的 `output_length` 固定为 1。
+
+## 5. max-model-len 真机 gate
+
+现有 real runner 新增了三个显式环境变量，chat 默认行为不变：
+
+```bash
+MAX_MODEL_LEN=ACTUAL_CELL_MAX_ROUNDED_UP \
+TRACE_INPUT_ROOT=/absolute/path/to/materialized/code-cell \
+ALLOW_SYNTHETIC_PROMPTS=true \
+OUTPUT_ROOT=/absolute/path/to/new/output \
+bash runs/frontier-s3-real-v0/run_full_real.sh RHO_LABEL tp4_mns16 1 PORT
+```
+
+- `MAX_MODEL_LEN` 必须覆盖 manifest 中该 paired cell 的实际最大请求；
+- `TRACE_INPUT_ROOT` 内必须有 `real_requests.jsonl` 和 `manifest.json`；
+- synthetic prompt 默认拒绝，只有在 experiment card 明确降级 claim 后才设为 `true`；
+- runner 会在启动前扫描 paired requests，若任何 `ISL+OSL` 超 cap 立即失败。
+
+长上下文 server 必须同时设置
+`VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` 和
+`--hf-overrides '{"max_position_embeddings":MAX_MODEL_LEN}'`；runner 已在
+`ALLOW_LONG_CONTEXT_SERVER=true` 时自动处理。长上下文默认使用 host-local
+vLLM compile cache，并按 topology 复用 FlashInfer workspace；启动 compile
+不进入 workload latency。
+
+## 6. decode-only
+
+当前 materializer 故意不提供 `decode_only` 选项。已安装 vLLM 0.20.0
+包含 `DecodeBenchConnector`，但它在首次 admission 后同步填 dummy KV；
+fill time 必须与 KV-ready arrival 分离。Frontier `Request` 支持
+`num_processed_tokens`，当前 trace generator 尚未从 CSV 注入该值。
+只有 real 首步无 prefill、sim ledger 首步为 decode 的 C0 gate 通过后，
+才创建 strict decode-only jobs。
+
+## 本地验证
+
+```bash
+python3 -m unittest -v \
+  runs/frontier-code-trace-v0/test_code_trace_preflight.py \
+  runs/frontier-s3-real-v0/test_remap_hash_blocks.py \
+  runs/frontier-s3-real-v0/test_select_chat_window.py
+python3 -m py_compile \
+  runs/frontier-code-trace-v0/*.py \
+  runs/frontier-s3-real-v0/*.py
+bash -n runs/frontier-s3-real-v0/run_full_real.sh
+```

runs/frontier-code-trace-v0/analyze_canary.py

@@ -0,0 +1,424 @@
+#!/usr/bin/env python3
+"""Analyze one paired 10-minute code-trace real/sim canary topology."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import hashlib
+import json
+import math
+import re
+import statistics
+from collections import Counter
+from pathlib import Path
+from typing import Any
+
+
+METRICS = ("ttft", "tpot", "e2e")
+TPOT_MIN_OUTPUT_TOKENS = (2, 8, 32)
+SLO_TARGET_PASS_RATE = 0.95
+PROM_COUNTERS = ("vllm:prefix_cache_queries_total", "vllm:prefix_cache_hits_total")
+csv.field_size_limit(16 * 1024 * 1024)
+ITERATION_RE = re.compile(
+    r"Iteration.*?:\s+"
+    r"(?P<context_requests>\d+) context requests, "
+    r"(?P<context_tokens>\d+) context tokens, "
+    r"(?P<generation_requests>\d+) generation requests, "
+    r"(?P<generation_tokens>\d+) generation tokens"
+)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input-root", type=Path, required=True)
+    parser.add_argument("--sim-root", type=Path, required=True)
+    parser.add_argument("--real-root", type=Path, action="append", required=True)
+    parser.add_argument("--topology", required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as stream:
+        for chunk in iter(lambda: stream.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def percentile(values: list[float], q: float) -> float:
+    ordered = sorted(values)
+    position = (len(ordered) - 1) * q
+    lower = math.floor(position)
+    upper = math.ceil(position)
+    if lower == upper:
+        return ordered[lower]
+    return ordered[lower] * (upper - position) + ordered[upper] * (position - lower)
+
+
+def distribution(values: list[float]) -> dict[str, float | int]:
+    if not values:
+        raise ValueError("empty distribution")
+    return {
+        "count": len(values),
+        "mean": statistics.fmean(values),
+        "p50": percentile(values, 0.5),
+        "p90": percentile(values, 0.9),
+        "p95": percentile(values, 0.95),
+        "p99": percentile(values, 0.99),
+        "max": max(values),
+    }
+
+
+def read_csv(path: Path) -> list[dict[str, str]]:
+    with path.open(newline="") as stream:
+        return list(csv.DictReader(stream))
+
+
+def find_one(root: Path, name: str) -> Path:
+    matches = list(root.glob(f"**/{name}"))
+    if len(matches) != 1:
+        raise ValueError(f"expected one {name} below {root}, found {matches}")
+    return matches[0]
+
+
+def prom_counter(path: Path, name: str) -> float:
+    values = []
+    with path.open() as stream:
+        for line in stream:
+            if line.startswith(name + "{") or line.startswith(name + " "):
+                values.append(float(line.rsplit(maxsplit=1)[1]))
+    if not values:
+        raise ValueError(f"{path}: missing Prometheus counter {name}")
+    return sum(values)
+
+
+def prefix_cache_delta(root: Path) -> dict[str, float]:
+    before = root / "metrics/before.prom"
+    after = root / "metrics/after.prom"
+    deltas = {
+        name: prom_counter(after, name) - prom_counter(before, name)
+        for name in PROM_COUNTERS
+    }
+    queries = deltas[PROM_COUNTERS[0]]
+    hits = deltas[PROM_COUNTERS[1]]
+    if queries <= 0 or hits < 0 or hits > queries:
+        raise ValueError(f"{root}: invalid prefix counter deltas {deltas}")
+    return {
+        "query_tokens": queries,
+        "hit_tokens": hits,
+        "hit_ratio": hits / queries,
+    }
+
+
+def real_decode_batch(root: Path) -> dict[str, Any]:
+    counts: Counter[int] = Counter()
+    mixed_steps = 0
+    for path in sorted(root.rglob("server.log")):
+        with path.open(errors="replace") as stream:
+            for line in stream:
+                match = ITERATION_RE.search(line)
+                if match is None:
+                    continue
+                context_requests = int(match.group("context_requests"))
+                generation_requests = int(match.group("generation_requests"))
+                generation_tokens = int(match.group("generation_tokens"))
+                if context_requests:
+                    mixed_steps += 1
+                    continue
+                if generation_requests and generation_tokens == generation_requests:
+                    counts[generation_requests] += 1
+
+    if not counts:
+        return {
+            "steps": 0,
+            "mixed_steps_excluded": mixed_steps,
+            "max": None,
+            "share_gt_1": None,
+            "histogram": {},
+        }
+    steps = sum(counts.values())
+    return {
+        "steps": steps,
+        "mixed_steps_excluded": mixed_steps,
+        "max": max(counts),
+        "share_gt_1": sum(value for key, value in counts.items() if key > 1) / steps,
+        "histogram": {str(key): value for key, value in sorted(counts.items())},
+    }
+
+
+def load_sim(root: Path, trace: list[dict[str, str]], trace_sha: str) -> dict[str, Any]:
+    manifest = json.loads((root / "manifest.json").read_text())
+    if manifest["trace_sha256"] != trace_sha:
+        raise ValueError(f"{root}: sim/input trace SHA mismatch")
+    rows = read_csv(find_one(root / "metrics", "request_metrics.csv"))
+    if len(rows) != len(trace):
+        raise ValueError(f"{root}: sim/input request count mismatch")
+    values = {
+        "ttft": [float(row["ttft"]) for row in rows],
+        "tpot": [float(row["tpot"]) for row in rows if row["tpot"].strip()],
+        "e2e": [float(row["request_e2e_time"]) for row in rows],
+        "waiting": [float(row["request_waiting_time_total"]) for row in rows],
+    }
+    completions = [
+        float(trace_row["arrived_at"]) + float(metric_row["request_e2e_time"]) / 1000
+        for trace_row, metric_row in zip(trace, rows)
+    ]
+    tail_index = max(range(len(completions)), key=completions.__getitem__)
+    last_arrival = max(float(row["arrived_at"]) for row in trace)
+    summary = json.loads((root / "summary.json").read_text())
+    slo_pass = []
+    for trace_row, metric_row in zip(trace, rows):
+        input_tokens = int(trace_row["num_prefill_tokens"])
+        ttft_threshold_ms = 1000 + 1000 * input_tokens / 8000
+        tpot = (
+            float(metric_row["tpot"])
+            if metric_row["tpot"].strip()
+            else None
+        )
+        slo_pass.append(
+            float(metric_row["ttft"]) <= ttft_threshold_ms
+            and (tpot is None or tpot <= 150)
+        )
+    return {
+        "values": values,
+        "tpot_by_min_output_tokens": {
+            str(threshold): [
+                float(metric_row["tpot"])
+                for trace_row, metric_row in zip(trace, rows)
+                if int(trace_row["num_decode_tokens"]) >= threshold
+                and metric_row["tpot"].strip()
+            ]
+            for threshold in TPOT_MIN_OUTPUT_TOKENS
+        },
+        "slo": {
+            "passed": sum(slo_pass),
+            "pass_rate": sum(slo_pass) / len(slo_pass),
+            "feasible": sum(slo_pass) / len(slo_pass) >= SLO_TARGET_PASS_RATE,
+        },
+        "summary": summary,
+        "drain": {
+            "last_arrival_s": last_arrival,
+            "last_completion_s": completions[tail_index],
+            "tail_after_last_arrival_s": completions[tail_index] - last_arrival,
+            "tail_driver": {
+                "request_index": tail_index,
+                "arrival_s": float(trace[tail_index]["arrived_at"]),
+                "arrival_before_cutoff_s": last_arrival
+                - float(trace[tail_index]["arrived_at"]),
+                "input_tokens": int(trace[tail_index]["num_prefill_tokens"]),
+                "output_tokens": int(trace[tail_index]["num_decode_tokens"]),
+                "waiting_ms": values["waiting"][tail_index],
+                "e2e_ms": values["e2e"][tail_index],
+            },
+        },
+    }
+
+
+def load_real(
+    root: Path,
+    input_manifest: dict[str, Any],
+    trace: list[dict[str, str]],
+) -> dict[str, Any]:
+    result_path = root / "results/result.json"
+    result = json.loads(result_path.read_text())
+    if result["contract"]["row_vector_sha256"] != input_manifest["paired_row_vector_sha256"]:
+        raise ValueError(f"{root}: real/input row digest mismatch")
+    requests = result["requests"]
+    if len(requests) != len(trace) or not all(row["success"] for row in requests):
+        raise ValueError(f"{root}: incomplete or failed real request vector")
+    for index, (request, trace_row) in enumerate(zip(requests, trace)):
+        observed = (int(request["input_tokens"]), int(request["requested_output_tokens"]))
+        expected = (
+            int(trace_row["num_prefill_tokens"]),
+            int(trace_row["num_decode_tokens"]),
+        )
+        if observed != expected:
+            raise ValueError(f"{root}: request {index} shape {observed} != {expected}")
+    values = {
+        metric: [
+            float(request[f"{metric}_ms"])
+            for request in requests
+            if request.get(f"{metric}_ms") is not None
+        ]
+        for metric in METRICS
+    }
+    completions = [
+        float(request["admitted_s"]) + float(request["e2e_ms"]) / 1000
+        for request in requests
+    ]
+    tail_index = max(range(len(completions)), key=completions.__getitem__)
+    last_arrival = max(float(request["scheduled_s"]) for request in requests)
+    return {
+        "root": str(root),
+        "result_sha256": sha256(result_path),
+        "values": values,
+        "tpot_by_min_output_tokens": {
+            str(threshold): [
+                float(request["tpot_ms"])
+                for request in requests
+                if int(request["requested_output_tokens"]) >= threshold
+                and request.get("tpot_ms") is not None
+            ]
+            for threshold in TPOT_MIN_OUTPUT_TOKENS
+        },
+        "slo": {
+            "passed": sum(bool(request["slo_pass"]) for request in requests),
+            "pass_rate": sum(bool(request["slo_pass"]) for request in requests)
+            / len(requests),
+            "feasible": sum(bool(request["slo_pass"]) for request in requests)
+            / len(requests)
+            >= SLO_TARGET_PASS_RATE,
+        },
+        "summary": result["summary"],
+        "prefix_cache": prefix_cache_delta(root),
+        "decode_batch": real_decode_batch(root),
+        "drain": {
+            "last_arrival_s": last_arrival,
+            "last_completion_s": completions[tail_index],
+            "tail_after_last_arrival_s": completions[tail_index] - last_arrival,
+            "tail_driver": {
+                "request_index": tail_index,
+                "arrival_s": float(requests[tail_index]["scheduled_s"]),
+                "arrival_before_cutoff_s": last_arrival
+                - float(requests[tail_index]["scheduled_s"]),
+                "input_tokens": int(requests[tail_index]["input_tokens"]),
+                "output_tokens": int(requests[tail_index]["requested_output_tokens"]),
+                "admission_lag_ms": float(requests[tail_index]["admission_lag_ms"]),
+                "e2e_ms": float(requests[tail_index]["e2e_ms"]),
+            },
+        },
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    input_manifest = json.loads((args.input_root / "manifest.json").read_text())
+    trace_path = args.input_root / "frontier.csv"
+    trace = read_csv(trace_path)
+    if len(trace) != input_manifest["requests"]:
+        raise ValueError("input manifest/trace request count mismatch")
+    trace_sha = sha256(trace_path)
+    sim = load_sim(args.sim_root, trace, trace_sha)
+    reals = [load_real(root, input_manifest, trace) for root in args.real_root]
+    pooled = {
+        metric: [value for real in reals for value in real["values"][metric]]
+        for metric in METRICS
+    }
+    latency = {}
+    for metric in METRICS:
+        real_dist = distribution(pooled[metric])
+        real_per_trial = [
+            distribution(real["values"][metric]) for real in reals
+        ]
+        real_reference = {
+            statistic: statistics.fmean(
+                float(trial[statistic]) for trial in real_per_trial
+            )
+            for statistic in ("mean", "p50", "p90", "p95", "p99")
+        }
+        sim_dist = distribution(sim["values"][metric])
+        latency[metric] = {
+            "real": real_dist,
+            "real_trial_statistic_mean": real_reference,
+            "sim": sim_dist,
+            "relative_bias_percent": {
+                statistic: 100
+                * (float(sim_dist[statistic]) - real_reference[statistic])
+                / real_reference[statistic]
+                for statistic in ("mean", "p50", "p90", "p95", "p99")
+            },
+            "real_per_trial": real_per_trial,
+        }
+    tpot_sensitivity = {}
+    for threshold in TPOT_MIN_OUTPUT_TOKENS:
+        key = str(threshold)
+        real_per_trial = [
+            distribution(real["tpot_by_min_output_tokens"][key])
+            for real in reals
+        ]
+        real_values = [
+            value
+            for real in reals
+            for value in real["tpot_by_min_output_tokens"][key]
+        ]
+        sim_values = sim["tpot_by_min_output_tokens"][key]
+        real_dist = distribution(real_values)
+        real_reference = {
+            statistic: statistics.fmean(
+                float(trial[statistic]) for trial in real_per_trial
+            )
+            for statistic in ("mean", "p50", "p90", "p95", "p99")
+        }
+        sim_dist = distribution(sim_values)
+        tpot_sensitivity[key] = {
+            "real": real_dist,
+            "real_trial_statistic_mean": real_reference,
+            "real_per_trial": real_per_trial,
+            "sim": sim_dist,
+            "relative_bias_percent": {
+                statistic: 100
+                * (float(sim_dist[statistic]) - real_reference[statistic])
+                / real_reference[statistic]
+                for statistic in ("mean", "p50", "p90", "p95", "p99")
+            },
+        }
+    payload = {
+        "schema": "frontier-code-trace-canary-analysis-v1",
+        "topology": args.topology,
+        "requests_per_trial": len(trace),
+        "trials": len(reals),
+        "input": {
+            "manifest": str(args.input_root / "manifest.json"),
+            "paired_row_vector_sha256": input_manifest["paired_row_vector_sha256"],
+            "frontier_csv_sha256": trace_sha,
+        },
+        "latency_ms": latency,
+        "tpot_by_min_output_tokens": tpot_sensitivity,
+        "slo": {
+            "definition": {
+                "ttft_ms": "1000 + 1000 * input_tokens / 8000",
+                "tpot_ms": 150,
+                "target_pass_rate": SLO_TARGET_PASS_RATE,
+            },
+            "real_per_trial": [real["slo"] for real in reals],
+            "sim": sim["slo"],
+            "feasibility_flip": any(
+                real["slo"]["feasible"] != sim["slo"]["feasible"]
+                for real in reals
+            ),
+        },
+        "prefix_cache": {
+            "real_per_trial": [real["prefix_cache"] for real in reals],
+            "real_hit_ratio_mean": statistics.fmean(
+                real["prefix_cache"]["hit_ratio"] for real in reals
+            ),
+            "sim": sim["summary"]["prefix_cache"],
+        },
+        "drain": {
+            "interpretation": (
+                "Report the max-completion request explicitly; a response that "
+                "arrived well before the cutoff can create a long drain tail "
+                "without implying queue accumulation."
+            ),
+            "real_per_trial": [real["drain"] for real in reals],
+            "sim": sim["drain"],
+        },
+        "sim_decode_batch": sim["summary"]["decode_batch"],
+        "real_decode_batch_per_trial": [real["decode_batch"] for real in reals],
+        "real_artifacts": [
+            {
+                "root": real["root"],
+                "result_sha256": real["result_sha256"],
+            }
+            for real in reals
+        ],
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps({"output": str(args.output), "topology": args.topology}))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/audit_code_trace.py

@@ -0,0 +1,382 @@
+#!/usr/bin/env python3
+"""Audit long code traces before choosing a replay window and max model length."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import statistics
+from collections import Counter
+from pathlib import Path
+from typing import Any, Iterable, Sequence
+
+
+BLOCK_SIZE_CANDIDATES = (16, 32, 64, 128, 256, 512, 1024)
+MAX_MODEL_LEN_CANDIDATES = (40960, 65536, 98304, 131072, 262144)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--trace-root", type=Path)
+    parser.add_argument("--source", type=Path, action="append")
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--min-minutes", type=int, default=60)
+    parser.add_argument("--max-minutes", type=int, default=75)
+    parser.add_argument("--bin-seconds", type=int, default=60)
+    parser.add_argument("--max-acceptable-gap-s", type=float, default=5.0)
+    parser.add_argument("--model-position-limit", type=int, default=262144)
+    return parser.parse_args()
+
+
+def percentile(values: Sequence[int | float], fraction: float) -> float | None:
+    if not values:
+        return None
+    ordered = sorted(float(value) for value in values)
+    position = (len(ordered) - 1) * fraction
+    lower = math.floor(position)
+    upper = math.ceil(position)
+    if lower == upper:
+        return ordered[lower]
+    return ordered[lower] * (upper - position) + ordered[upper] * (position - lower)
+
+
+def distribution(values: Sequence[int | float]) -> dict[str, int | float | None]:
+    return {
+        "count": len(values),
+        "min": min(values) if values else None,
+        "p50": percentile(values, 0.50),
+        "p90": percentile(values, 0.90),
+        "p95": percentile(values, 0.95),
+        "p99": percentile(values, 0.99),
+        "max": max(values) if values else None,
+        "mean": statistics.fmean(values) if values else None,
+    }
+
+
+def parse_hash_ids(value: Any) -> list[Any]:
+    if isinstance(value, list):
+        return value
+    if isinstance(value, str):
+        stripped = value.strip()
+        if not stripped:
+            return []
+        if stripped.startswith("["):
+            decoded = json.loads(stripped)
+            if not isinstance(decoded, list):
+                raise ValueError("hash_ids JSON must decode to a list")
+            return decoded
+        delimiter = "|" if "|" in stripped else ","
+        return [part for part in stripped.split(delimiter) if part.strip()]
+    if value is None:
+        return []
+    return [value]
+
+
+def iter_jsonl(path: Path) -> Iterable[tuple[int, dict[str, Any]]]:
+    with path.open() as stream:
+        for line_number, line in enumerate(stream, 1):
+            if not line.strip():
+                continue
+            row = json.loads(line)
+            if not isinstance(row, dict):
+                raise ValueError(f"{path}:{line_number}: row must be an object")
+            yield line_number, row
+
+
+def choose_window(
+    *,
+    counts: Sequence[int],
+    max_gaps: Sequence[float],
+    first_timestamp: float,
+    min_minutes: int,
+    max_minutes: int,
+    bin_seconds: int,
+    max_acceptable_gap_s: float,
+) -> dict[str, Any] | None:
+    candidates = []
+    for minutes in range(max_minutes, min_minutes - 1, -1):
+        bins = math.ceil(minutes * 60 / bin_seconds)
+        for start_bin in range(0, len(counts) - bins + 1):
+            selected = counts[start_bin : start_bin + bins]
+            mean = statistics.fmean(selected)
+            cv = statistics.pstdev(selected) / mean if mean else math.inf
+            max_gap = max(max_gaps[start_bin : start_bin + bins], default=0.0)
+            candidates.append(
+                {
+                    "_score": (
+                        max_gap > max_acceptable_gap_s,
+                        cv,
+                        max_gap,
+                        -minutes,
+                        start_bin,
+                    ),
+                    "start_bin": start_bin,
+                    "minutes": minutes,
+                    "count_mean_per_bin": mean,
+                    "count_cv": cv,
+                    "count_min_per_bin": min(selected),
+                    "count_max_per_bin": max(selected),
+                    "max_gap_s": max_gap,
+                }
+            )
+    if not candidates:
+        return None
+    chosen = min(candidates, key=lambda item: item["_score"])
+    chosen.pop("_score")
+    chosen["start_timestamp"] = first_timestamp + chosen["start_bin"] * bin_seconds
+    chosen["end_timestamp"] = chosen["start_timestamp"] + chosen["minutes"] * 60
+    return chosen
+
+
+def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
+    rows = 0
+    source_rows = 0
+    invalid_zero_token_rows = 0
+    invalid_zero_token_examples: list[dict[str, Any]] = []
+    first_timestamp = None
+    last_timestamp = None
+    previous_timestamp = None
+    counts: Counter[int] = Counter()
+    max_gaps: dict[int, float] = {}
+    input_lengths: list[int] = []
+    output_lengths: list[int] = []
+    total_lengths: list[int] = []
+    hash_rows = 0
+    hash_matches = Counter()
+    prompt_rows = 0
+    sampling_rows = 0
+    schema_keys: Counter[str] = Counter()
+    for line_number, row in iter_jsonl(path):
+        source_rows += 1
+        missing = [
+            key
+            for key in ("timestamp", "input_length", "output_length")
+            if key not in row
+        ]
+        if missing:
+            raise ValueError(f"{path}:{line_number}: missing required fields {missing}")
+        timestamp = float(row["timestamp"])
+        input_tokens = int(row["input_length"])
+        output_tokens = int(row["output_length"])
+        schema_keys.update(row.keys())
+        if input_tokens <= 0 or output_tokens <= 0:
+            invalid_zero_token_rows += 1
+            if len(invalid_zero_token_examples) < 20:
+                invalid_zero_token_examples.append(
+                    {
+                        "line_number": line_number,
+                        "chat_id": row.get("chat_id"),
+                        "timestamp": timestamp,
+                        "input_length": input_tokens,
+                        "output_length": output_tokens,
+                    }
+                )
+            continue
+        if first_timestamp is None:
+            first_timestamp = timestamp
+        if previous_timestamp is not None and timestamp < previous_timestamp:
+            raise ValueError(
+                f"{path}:{line_number}: timestamp {timestamp} < {previous_timestamp}"
+            )
+        bin_index = math.floor((timestamp - first_timestamp) / args.bin_seconds)
+        counts[bin_index] += 1
+        if previous_timestamp is not None:
+            previous_bin = math.floor(
+                (previous_timestamp - first_timestamp) / args.bin_seconds
+            )
+            max_gaps[previous_bin] = max(
+                max_gaps.get(previous_bin, 0.0),
+                timestamp - previous_timestamp,
+            )
+        input_lengths.append(input_tokens)
+        output_lengths.append(output_tokens)
+        total_lengths.append(input_tokens + output_tokens)
+        hashes = parse_hash_ids(row.get("hash_ids"))
+        if hashes:
+            hash_rows += 1
+            for block_size in BLOCK_SIZE_CANDIDATES:
+                if len(hashes) == math.ceil(input_tokens / block_size):
+                    hash_matches[block_size] += 1
+        prompt_rows += int(
+            isinstance(row.get("prompt"), (str, list)) and bool(row.get("prompt"))
+        )
+        sampling_rows += int("sampling_u" in row)
+        rows += 1
+        previous_timestamp = timestamp
+        last_timestamp = timestamp
+    if not rows or first_timestamp is None or last_timestamp is None:
+        raise ValueError(f"{path}: empty trace")
+    total_bins = math.floor((last_timestamp - first_timestamp) / args.bin_seconds) + 1
+    chosen = choose_window(
+        counts=[counts[index] for index in range(total_bins)],
+        max_gaps=[max_gaps.get(index, 0.0) for index in range(total_bins)],
+        first_timestamp=first_timestamp,
+        min_minutes=args.min_minutes,
+        max_minutes=args.max_minutes,
+        bin_seconds=args.bin_seconds,
+        max_acceptable_gap_s=args.max_acceptable_gap_s,
+    )
+    return {
+        "source": str(path.resolve()),
+        "rows": rows,
+        "source_rows": source_rows,
+        "invalid_zero_token_rows": invalid_zero_token_rows,
+        "invalid_zero_token_fraction": invalid_zero_token_rows / source_rows,
+        "invalid_zero_token_examples": invalid_zero_token_examples,
+        "first_timestamp": first_timestamp,
+        "last_timestamp": last_timestamp,
+        "span_s": last_timestamp - first_timestamp,
+        "request_rate_per_s": rows / max(last_timestamp - first_timestamp, 1.0),
+        "input_length": distribution(input_lengths),
+        "output_length": distribution(output_lengths),
+        "total_length": distribution(total_lengths),
+        "over_max_model_len": {
+            str(limit): {
+                "requests": sum(value > limit for value in total_lengths),
+                "fraction": sum(value > limit for value in total_lengths) / rows,
+            }
+            for limit in MAX_MODEL_LEN_CANDIDATES
+        },
+        "hash_contract": {
+            "rows_with_hash_ids": hash_rows,
+            "candidate_exact_match_rows": {
+                str(size): hash_matches[size] for size in BLOCK_SIZE_CANDIDATES
+            },
+            "exact_source_block_size": next(
+                (
+                    size
+                    for size in BLOCK_SIZE_CANDIDATES
+                    if hash_rows and hash_matches[size] == hash_rows
+                ),
+                None,
+            ),
+        },
+        "prompt_rows": prompt_rows,
+        "sampling_u_rows": sampling_rows,
+        "schema_field_counts": dict(sorted(schema_keys.items())),
+        "stable_window": chosen,
+    }
+
+
+def scan_window(source: Path, window: dict[str, Any]) -> dict[str, Any]:
+    start = float(window["start_timestamp"])
+    end = float(window["end_timestamp"])
+    inputs: list[int] = []
+    outputs: list[int] = []
+    totals: list[int] = []
+    for _, row in iter_jsonl(source):
+        timestamp = float(row["timestamp"])
+        if timestamp < start:
+            continue
+        if timestamp >= end:
+            break
+        input_tokens = int(row["input_length"])
+        output_tokens = int(row["output_length"])
+        if input_tokens <= 0 or output_tokens <= 0:
+            continue
+        inputs.append(input_tokens)
+        outputs.append(output_tokens)
+        totals.append(input_tokens + output_tokens)
+    return {
+        "requests": len(totals),
+        "input_length": distribution(inputs),
+        "output_length": distribution(outputs),
+        "total_length": distribution(totals),
+        "max_model_len_coverage": {
+            str(limit): {
+                "covered_requests": sum(value <= limit for value in totals),
+                "excluded_requests": sum(value > limit for value in totals),
+                "coverage": sum(value <= limit for value in totals) / len(totals),
+            }
+            for limit in MAX_MODEL_LEN_CANDIDATES
+        },
+    }
+
+
+def resolve_sources(args: argparse.Namespace) -> list[Path]:
+    if args.source:
+        return [path.resolve() for path in args.source]
+    if args.trace_root is None:
+        raise ValueError("provide --trace-root or one or more --source")
+    sources = sorted(
+        path.resolve()
+        for path in args.trace_root.glob("*.jsonl")
+        if "prompt" not in path.stem.lower()
+    )
+    if not sources:
+        raise FileNotFoundError(f"no non-prompt JSONL files under {args.trace_root}")
+    return sources
+
+
+def main() -> None:
+    args = parse_args()
+    if not 0 < args.min_minutes <= args.max_minutes:
+        raise ValueError("require 0 < min_minutes <= max_minutes")
+    sources = resolve_sources(args)
+    files = [scan_source(path, args) for path in sources]
+    eligible = [item for item in files if item["stable_window"] is not None]
+    if not eligible:
+        chosen = None
+        data_gate = "BLOCKED_NO_1H_WINDOW"
+    else:
+        chosen = min(
+            eligible,
+            key=lambda item: (
+                item["stable_window"]["max_gap_s"] > args.max_acceptable_gap_s,
+                item["stable_window"]["count_cv"],
+                -item["stable_window"]["minutes"],
+                item["source"],
+            ),
+        )
+        chosen["selected_window_stats"] = scan_window(
+            Path(chosen["source"]), chosen["stable_window"]
+        )
+        exact_block_size = chosen["hash_contract"]["exact_source_block_size"]
+        max_total = chosen["selected_window_stats"]["total_length"]["max"]
+        data_gate = (
+            "PASS"
+            if exact_block_size is not None
+            and max_total is not None
+            and max_total <= args.model_position_limit
+            else "BLOCKED_HASH_OR_POSITION_CONTRACT"
+        )
+    recommendation = None
+    if chosen is not None:
+        maximum = chosen["selected_window_stats"]["total_length"]["max"]
+        recommendation = next(
+            (
+                limit
+                for limit in MAX_MODEL_LEN_CANDIDATES
+                if maximum <= limit <= args.model_position_limit
+            ),
+            None,
+        )
+    payload = {
+        "schema": "frontier-code-trace-audit-v1",
+        "trace_root": str(args.trace_root.resolve()) if args.trace_root else None,
+        "sources": [str(path) for path in sources],
+        "window_policy": {
+            "min_minutes": args.min_minutes,
+            "max_minutes": args.max_minutes,
+            "bin_seconds": args.bin_seconds,
+            "max_acceptable_gap_s": args.max_acceptable_gap_s,
+            "selection": "lowest density CV after rejecting anomalous-gap windows",
+        },
+        "model_position_limit": args.model_position_limit,
+        "files": files,
+        "selected": chosen,
+        "max_model_len_recommendation": recommendation,
+        "data_gate": data_gate,
+        "runtime_gate": (
+            "PENDING: vLLM startup must prove enough KV blocks and nonzero "
+            "max concurrency at the recommended max_model_len for each TP"
+        ),
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps({"data_gate": data_gate, "output": str(args.output)}))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/check_longctx_profile_repeats.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+"""Check fresh-process repeat stability for the code long-context grid."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--first", type=Path, nargs="+", required=True)
+    parser.add_argument("--second", type=Path, nargs="+", required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--max-relative-difference", type=float, default=0.05)
+    return parser.parse_args()
+
+
+def load(paths: list[Path]) -> dict[tuple[int, str], dict]:
+    rows: dict[tuple[int, str], dict] = {}
+    for path in paths:
+        payload = json.loads(path.read_text())
+        for row in payload["rows"]:
+            if row.get("error"):
+                raise ValueError(
+                    f"{path}: failed profile row {row['config']['batch_spec']}"
+                )
+            key = (
+                int(row["tensor_parallel_size"]),
+                str(row["config"]["batch_spec"]),
+            )
+            if key in rows:
+                raise ValueError(f"duplicate row {key}")
+            rows[key] = row
+    return rows
+
+
+def main() -> None:
+    args = parse_args()
+    first = load(args.first)
+    second = load(args.second)
+    if first.keys() != second.keys():
+        raise ValueError(
+            f"repeat key mismatch: first_only={sorted(first.keys()-second.keys())}, "
+            f"second_only={sorted(second.keys()-first.keys())}"
+        )
+    comparisons = []
+    for key in sorted(first):
+        left = float(first[key]["mean_time"])
+        right = float(second[key]["mean_time"])
+        relative = abs(left - right) / ((left + right) / 2)
+        comparisons.append(
+            {
+                "tp": key[0],
+                "batch_spec": key[1],
+                "first_mean_s": left,
+                "second_mean_s": right,
+                "relative_difference": relative,
+                "pass": relative <= args.max_relative_difference,
+            }
+        )
+    maximum = max(item["relative_difference"] for item in comparisons)
+    payload = {
+        "schema": "frontier-code-longctx-repeat-check-v1",
+        "threshold": args.max_relative_difference,
+        "maximum_relative_difference": maximum,
+        "status": "PASS" if maximum <= args.max_relative_difference else "FAIL",
+        "comparisons": comparisons,
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps({"status": payload["status"], "max": maximum}))
+    if payload["status"] != "PASS":
+        raise SystemExit(1)
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/experiment-card.md

@@ -0,0 +1,346 @@
+# EXP-CODE-TRACE：从 chat 1h trace 扩展到 code 与 phase-separated replay
+
+> **状态：RUNNING（Phase A code P+D）。** A0 数据/profile、A1
+> max-length smoke、A2 paired canary 与 A3 sim calibration 已完成；
+> A4 第一批 61min real jobs 正在 `dash1`--`dash4` 运行。禁止使用 `dash0`。
+
+## 目标与成功定义
+
+当前 1h+ 证据只覆盖 Qwen3-30B-A3B 的生产 chat trace、prefill+decode（P+D）和亚临界负载。本 campaign 分两步扩展：
+
+1. **主任务：** 使用 `~/ali-trace/trace-glm5.1-formatted/` 中的 1h+ code trace，先完成 P+D real-vs-Frontier 回放；
+2. **后续 phase matrix：** 对 chat/code 都补 prefill-only 和严格 decode-only。
+
+本轮不是只看“能否跑完”。每个正式 cell 必须满足：同一 request vector、同一 arrival、同一 token shape、同一 prefix/initial-KV 合约、real 零失败、无持续 backlog，并同时报告 TTFT/TPOT/E2E、queue/batch、KV/prefix state 与 5min 分窗漂移。
+
+## 当前决策快照
+
+| 项目 | 当前结论 | 下一 gate |
+|---|---|---|
+| code P+D 数据 | 61min development window、long-context profile-v6、四个 paired full 输入已冻结 | 第一批 1h real 运行中 |
+| TP4 负载 | low/mid/near-knee=`rho 0.0002/0.0008/0.0016`；paired canary 的 TTFT/E2E、KV、batch 通过 | 三个 load 的 trial 1 运行中 |
+| TP2 负载 | `rho 0.0004` canary 暴露 TTFT p90 `-32.1%` bad case | 只跑 `rho 0.0002` 1h diagnostic，暂不铺满 |
+| code prefill-only | TP2 已冻结 `rho 0.0004/0.0008/0.0016`，`0.0032` 过载；TP4 到 `0.0032` 仍亚临界 | TP4 追加更高 rho 边界 |
+| strict decode-only | vLLM 0.20.0 有 `DecodeBenchConnector`；Frontier trace generator 尚不能注入 initial computed tokens | C0 contract canary，未进入正式结果 |
+
+## 三种 workload mode 的冻结定义
+
+| Mode | 保留 | 改写 | 主指标 | 明确不声称 |
+|---|---|---|---|---|
+| P+D | 原 ISL/OSL、arrival、session/prefix | 仅做 source block→16-token runtime block 映射 | TTFT、TPOT、E2E、hit ratio、batch/queue | 不代表 PD 分离 |
+| prefill-only | 原 ISL、arrival、session/prefix | OSL 固定为 1，real `min_tokens=max_tokens=1`，sim decode tokens=1 | TTFT、prefill service/tokens/s、prefix hit、queue | TPOT 不定义；1-token decode 只用于完成请求 |
+| strict decode-only | 原 OSL、context length、arrival burst | arrival 定义为 **KV-ready time**；请求进入 decode 时已有 ISL 长度的 initial KV | TPOT、decode tokens/s、batch/queue、preemption | 不包含 prefill 与 KV transfer latency，不把短 prompt proxy 称为 decode-only |
+
+strict decode-only 必须同时具备：
+
+- real：vLLM `DecodeBenchConnector`（或等价、经验证的 initial-KV 注入）；
+- sim：Frontier request 在 admission 时已拥有相同长度/块布局的 computed KV；
+- 两侧都不在 decode critical path 重做 prefill；
+- arrival 以 KV-ready time 对齐。若只保留原 trace 的相对到达形状，结论限定为 decode engine compute/scheduling fidelity。
+
+在该合约完成前，只允许跑并标注为 **decode-dominant proxy**，不能进入 strict decode-only 结果表。
+
+## 为什么 code P+D 不能直接复用 chat 配置
+
+已知历史探查显示 code trace ISL p90 约 81.9k，约 32.6% 请求超过旧 `40960` 上限；真实数值必须由本 campaign 重新审计。至少有四个独立适配面：
+
+1. **Serving cap：** `max_model_len` 必须覆盖 `ISL+OSL`，不能只看 ISL，也不能静默丢掉超长请求；
+2. **KV capacity：** Qwen3-30B 模型 position limit 为 262144，但 TP1/2/4 在 H20 上是否有足够 KV blocks 是 runtime gate，不由 config.json 自动保证；
+3. **Prefix block：** code source hash 预计为 512-token block，chat harness 原先固定 64→16；
+4. **Profile support：** 当前修复后的 attention profile 只覆盖到约 32k KV context。即使 vLLM 能跑 128k，Frontier 对 32k–128k 仍会出 profile 支撑域；在补 long-context 网格前只能做诊断 replay，不能做 fidelity claim。
+
+## Hypotheses
+
+- **H-code-generalizes：** 在补齐 long-context profile 支撑域后，code P+D 的 TTFT/TPOT/E2E 分布统计偏差仍处于当前 chat 量级，且 1h 残差不发散。
+- **H-longctx-gap：** code 的主要新增 gap 来自 32k 以上 KV-context 外推；补到 trace p99/max 对应的网格后，TTFT bias 随 ISL 的二次项显著收敛。
+- **H-phase-specific：** prefill-only 主要暴露 long-context/profile gap；strict decode-only 主要暴露 batch-conditioned whole-layer service 与 scheduler fixed-point gap。二者不能用 P+D 的误差抵消来互相证明准确。
+
+## Preflight gates（按顺序，任一失败即停止后续真机矩阵）
+
+### G0：数据位置与 provenance
+
+- 只读列举 `trace-glm5.1-formatted/*.jsonl`，记录文件大小与 SHA256；
+- 确认至少两个独立日期段：一个作为 development，一个 held-out；
+- 本机当前没有该目录；仓库历史记录的远端位置为
+  `/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/`。恢复机器后先确认 `~/ali-trace/...` 是否为同一路径/软链，不能假设。
+
+### G1：1h window、schema 与 block contract
+
+运行 `audit_code_trace.py`，要求：
+
+- timestamp 单调，存在 60–75min 连续稳定窗口；
+- `timestamp/input_length/output_length` 全行存在；
+- `input_length<=0` 或 `output_length<=0` 的行不进入 replay，但必须计数并保留样例。已抽查的两条 0→0 行在 raw trace 中同时满足 `usage.total_tokens=0`、`response_message={}`，属于未发生模型执行的 source request，不是 full-cache decode；
+- `hash_ids` 数量与某个 source block size 在全行严格满足
+  `ceil(ISL/source_block_size)`；预计值 512，但以审计结果为准；
+- 记录 ISL/OSL/ISL+OSL 的 p50/p90/p95/p99/max、gap、request rate、prompt/sampling 字段覆盖。
+
+选择窗口后用 `prepare_code_window.py` 物化只读派生文件，并按 session root 生成确定性的 `sampling_u`。另一日期段不参与 rho 与 profile 选择。
+
+### G2：`max_model_len` data gate
+
+source-window audit 先用 `40960/65536/98304/131072/262144` 给出完整
+窗口上界；真实 server 则使用能 **100% 覆盖该 rho 实际 paired requests
+的 `ISL+OSL`** 的最小 16-token 对齐值。规则：
+
+- sampling 只按 session-coherent `sampling_u`，不得按 token length 过滤；
+- full source window 的 cap 用于记录 workload envelope，不强迫低 rho cell
+  为未被抽中的 outlier 预留 KV capacity；
+- 若某 paired cell max≤131072，使用 131072 或更小的对齐值；超过
+  131072 时按该 cell 实际 max 向上对齐，而不是直接跳到 262144；
+- Frontier 的 trace max tokens、predictor max tokens/request、vLLM `--max-model-len` 三处使用同一个 manifest 值。
+
+### G3：prompt 与 prefix fidelity
+
+优先级：
+
+1. 有对齐 prompt text sidecar：用 Qwen tokenizer 重分词，要求 token length 与 trace ISL 全行一致；
+2. trace 内已有 prompt text/token IDs：同样做长度与 hash relation 检查；
+3. 两者都没有：允许用 source hash 确定性展开为 synthetic Qwen token IDs，但结果降级为 **length/arrival/prefix-shape faithful**，不声称 prompt-content 或 MoE routing faithful。
+
+不论走哪条路径，source→16 映射冲突、runtime identity collision、parent prefix violation 都必须为 0。P+D/prefill-only 两侧 prefix caching 同开；先用 5–10min TP4/MNS16 做 hit-ratio audit。
+
+### G4：long-context profile support
+
+现有 profile-v5 的 KV context 上界约 32k，对 code 不足。根据 development window 的 uncached-ISL 分布生成 profile-v6-code-longctx：
+
+- full chunk：`q8k`，context 至少覆盖 40k/56k/72k/88k/104k/120k/128k；
+- tail chunk：从真实 `ISL mod 8192` 的 p50/p90 选择 2–4k/4–6k 代表点；
+- TP1/2/4 分开采集，复测 `q1ks8k/q8ks32k` anchor；
+- 每点至少两次 fresh-process repeat；CV≤5%，anchor drift≤10%；
+- profile max context 必须 ≥ development window p99；正式 max claim 要求 ≥ max。若只覆盖 p99，max 以上请求单独列为 out-of-support，不进入总体准确度数字。
+
+这是 code P+D 正式 fidelity 的硬 gate。可以先用旧 profile 跑 diagnostic sim 来估 load，但不得与真机组成最终 gap。
+
+### G5：vLLM max-length/KV runtime gate
+
+对每个候选 topology（先 TP4，再 TP2，TP1 后置）：
+
+1. fresh server，以 manifest cap 启动；
+2. 记录 vLLM 版本、model config、GPU KV blocks、maximum concurrency、启动日志；
+3. 发 3 个单请求：ISL p50、p99、max（OSL=1），usage 必须逐 token 对齐；
+4. 发 5min sampled P+D canary，零 OOM/timeout/preemption storm；
+5. 只有 maximum concurrency>1 且 canary drain tail≤窗口时长 10% 才进入 rho calibration。
+
+`max_model_len` 变大不等于每个请求都预占最大 KV，但会改变启动合法性与可表达的单请求上界；实际 KV 压力仍由并发 token state 决定。
+
+对本模型，`VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` 只放宽 scheduler/config
+校验，不会扩展模型内部 RoPE cache；server 还必须显式传
+`--hf-overrides '{"max_position_embeddings":147456}'`。runner 对
+`MAX_MODEL_LEN>40960` 自动同时设置这两层。长上下文 job 默认使用
+host-local vLLM compile cache；FlashInfer workspace 按 topology 复用，
+避免每个 rho/trial 重编译同一组 fused-MoE kernels。两者只影响启动，
+不进入 replay latency。
+
+### G6：每种 mode 独立标定 rho
+
+不能复用 P+D rho：
+
+- P+D 同时按 raw/prefix-adjusted prefill tokens/s 与 decode tokens/s 看 knee；
+- prefill-only 因 OSL=1，重新按 prefill work 标定；
+- strict decode-only 因无 prefill，按 decode tokens/s 和 batch fixed point 标定。
+
+每种 workload×mode 选择 `low/mid/near-knee` 三点；正式点必须亚临界：全请求完成、无持续 backlog、drain tail≤10%、waiting p99 不单调随时间增长。跨 knee 点若运行，只作为 overload boundary，不支持“不发散”结论。
+
+`drain tail` 必须同时列出最后完成请求的 arrival、ISL、OSL、waiting
+和 E2E。早于 cutoff 到达但 OSL 很长的请求可以在最后 arrival 后继续
+decode；这属于 intrinsic response tail，不等价于 arrival cutoff 时仍有
+持续增长的 queue backlog。亚临界判断以 queue/waiting trajectory 和
+tail driver 分解共同决定，不能只用一个 drain 秒数。
+
+### G7：strict decode-only capability gate
+
+先在 10min synthetic trace 上验证：
+
+- real connector 确认没有执行 prefill kernel；
+- Frontier ledger 第一个阶段就是 decode，computed tokens=ISL；
+- 相同 context length 下两侧 KV block count 一致；
+- connector preload/transfer 时间独立记账，不混入 TPOT；
+- decode batch telemetry 能覆盖 b1 到目标 batch。
+
+若 vLLM 0.20 community stack 没有等价 connector，严格 case 保持 BLOCKED；可另跑 decode-dominant proxy，但单独命名和汇报。
+
+## 正式实验矩阵与推进顺序
+
+### Phase A：code P+D（第一优先级）
+
+1. **A0 CPU/data：** G0–G4；
+2. **A1 max-len smoke：** TP4→TP2；TP1 只在 KV gate 通过后加入；
+3. **A2 paired 10min canary：** TP4/MNS16，low rho，real+sim；
+4. **A3 calibration：** 各 rho 只先跑 sim，冻结 low/mid/near-knee；
+5. **A4 full：** TP4/MNS16、TP2/MNS16 × 3 rho × 2 trial × 60–75min；
+6. **A5 held-out：** 只在 development window 判据冻结后，对第二日期段跑 TP4 的 mid/near-knee。
+
+若某 topology 的 near-knee 过载，像现有 chat TP2/ρ0.01 一样排除，不为凑齐矩阵强跑。
+
+当前状态：A0–A3 完成。A4 第一批为 TP4
+`rho={0.0002,0.0008,0.0016}` trial 1，以及 TP2 `rho=0.0002`
+trial 1 diagnostic；其余 TP2 cell 等该 diagnostic 验证 canary bad case
+后再决定是否扩展。
+
+### Phase B：chat/code prefill-only
+
+- 复用各自已物化 window，只把 OSL 改为 1；
+- primary：TP4/MNS16、TP2/MNS16 × 3 独立 rho × 2 trial；
+- 报 TTFT/CDF/quantiles、prefill tokens/s、prefix hit、waiting、chunk/context 分带 residual；
+- TPOT 记为 N/A，E2E 仅作为“一 token completion”辅助值；
+- code 必须继续使用 profile-v6 long-context；chat 使用已验证 profile-v5。
+
+### Phase C：chat/code strict decode-only
+
+先做 batch-sensitive screening，再决定是否铺满：
+
+- **C0 capability canary：** 两 workload × TP4 × MNS{16,128}，10min；
+- **C1 core full：** TP{2,4} × MNS{16,128} × rho{low,near-knee} × 2 trial；
+- **C2 conditional expansion：** 只有当 C1 的 batch 分布从 b≤8 跨到 b>8，或 accuracy gap 随 MNS 改变>5pp，才补 MNS{32,64} 与 mid rho。
+
+decode profile/serving anchors 至少覆盖实际 batch p99。当前 whole-layer grid 只对少数 b≤8 有证据，且 b6 有长尾；在 MNS128 case 前必须补 b{1,2,4,8,16,32,64,128} 或实际访问 bucket，不能把 b8 常数外推到 b128。
+
+已安装 vLLM 0.20.0 的 `DecodeBenchConnector` 会在首次 schedule 时把
+`request.num_tokens-num_computed_tokens-1` 个 token 标为 external，
+同步向已分配的每层 KV blocks 写 dummy non-zero values，再从最后一个
+prompt token 开始 forward。因此它适合测大 context 下的 decode
+compute/scheduling，但 connector fill 发生在 client admission 之后：
+fill time 必须单独记账并从 KV-ready arrival/TPOT 口径中排除。
+dummy KV 也不提供真实 prompt-content 或 MoE-routing fidelity。
+
+Frontier commit `deadc4a3` 的 `Request` 已支持构造
+`num_processed_tokens`，但 `TraceReplayRequestGenerator` 不读取该列；
+因此 sim 侧仍需一个显式、可测试的 `initial_computed_tokens` trace
+contract。C0 必须同时证明 real 首个 model step 是 decode、Frontier
+首个 ledger stage 是 decode，之后才能解除 strict decode-only 的 BLOCKED。
+
+## 指标与判据
+
+共同口径：
+
+- 分布统计偏差：`(sim statistic-real statistic)/real statistic`，不是 per-request MAPE；
+- mean/p50/p90/p99 与 empirical CDF；
+- 5min 分窗，前 15min warmup 不进漂移 slope；
+- batch histogram、time-weighted running/waiting、drain tail、preemption；
+- 两 trial pooled 结果和 trial-to-trial noise floor 分开报告。
+
+判据分两层：
+
+1. **准确度：** primary latency mean/p90/p99 的 |bias|≤15% 为强通过，15–30% 为有界但需标注 correction，>30% 立 bad case；任何 topology 排序或 SLO feasibility 翻转都单独判 failure，不能被平均值掩盖。
+2. **长时稳定：** `|residual Theil–Sen slope|×12 / real noise floor < 1` 为 H-BOUNDED；只适用于亚临界 cell。
+
+mode-specific：
+
+- P+D：TTFT/TPOT/E2E 全部 primary；
+- prefill-only：TTFT primary，TPOT N/A；
+- strict decode-only：TPOT primary，TTFT 仅表示 admission/connector overhead，不进入 compute-fidelity gate。
+
+## 成本与调度
+
+- Phase A core：12 个 60–75min jobs（2 topology×3 load×2 trial），约 15 host-hours；按 TP 加权约 45 H20-GPU-hours，加 2–4 个 smoke/canary；
+- Phase B 两 workload：24 个 full jobs，按相同 75min 上界约 90 H20-GPU-hours；
+- Phase C 不一次铺满。C0 4 个 10min canary；C1 32 个 full jobs；C2 按触发条件追加。
+
+每个 job fresh server；只在 `dash1`–`dash4` 全 8 卡 idle/healthy 时启动。即使 TP2/TP4 job 只用部分 GPU，也不在同一 host 并跑，避免 fresh-server 空窗竞态。每一批使用新的 jobs TOML，现有 dispatcher 非幂等。
+
+## 预期产物
+
+- `inputs/code-audit.json`、`inputs/code-window/window-manifest.json`；
+- P+D/prefill-only 的 paired `frontier.csv`、`real_requests.jsonl` 与 manifest；
+- profile-v6-code-longctx raw/merged profile 与 variance report；
+- 每 cell real/sim request metrics、server telemetry、stage ledger；
+- `results/code-pd-fidelity.md`；
+- 最终 `chat/code × P+D/prefill-only/decode-only` compatibility table。
+
+## 已知边界
+
+- code trace 来自 GLM5.1 业务，serving model 是 Qwen3-30B；若无原 prompt text，测试只能保持 shape/prefix 结构，不能证明内容相关 routing fidelity；
+- `max_model_len=128k/256k` 解决的是接入上界，不自动解决 32k 以上 profile 外推；
+- strict decode-only 只测 decode engine；完整 PD 分离还需要单独建模 prefill、KV transfer、backpressure 与 KV-ready arrival。
+
+## 执行记录（2026-07-23）
+
+- fleet probe：dash1–dash4 均为 8×H20；32 张卡 memory.used=0、
+  utilization=0、无 compute process、uncorrected ECC=0；
+- 两个 formatted trace 都严格满足 512-token source hash contract；
+- 0513：2,108,130 个有效请求、6090 个 zero-usage source 行；稳定
+  development window=`[3480,7140)`，61min、1,078,928 请求；
+- 0529：1,977,423 个有效请求、6031 个 zero-usage source 行；冻结为
+  held-out，稳定候选 window=`[2640,6240)`；
+- development window：ISL p50/p90/p99/max =
+  20,051/88,224/125,803/202,371；OSL p50/p90/p99/max =
+  78/758/6449/131,072；`ISL+OSL max=202,745`；
+- full-window 131072 coverage=99.399%，262144 coverage=100%。但
+  session sampling 的候选 `rho<=0.0032` 实际 max total=137,016，因此
+  primary server cap 将按最终 cell max 对齐，不为未抽中的 202k outlier
+  直接预留 262k；
+- source 无 Qwen-aligned prompt/token IDs。raw canonical prompt 使用 GLM
+  token contract，不能同时保持 Qwen token content 与 trace ISL；本 campaign
+  采用 synthetic Qwen tokens 保持 length/hash/prefix shape，并降级内容 claim。
+- selected rho=0.0032 中有 1355 个可检查 parent links：tail rewrite
+  p50/p90/p95/p99/max=1/1/4/57/169 个 source blocks，说明 coder
+  `parent_chat_id` 不等价于 append-only prompt。source hash 序列作为 prefix
+  truth；synthetic content block 生成后再计算 parent-sensitive runtime
+  identities，避免“相同内容块出现在不同前缀后”造成 Frontier false hit。
+- 远端 Qwen3-30B `config.json` 的原生 position limit 是 40960
+  (`rope_theta=1e6`,无 rope_scaling)。147456 profile smoke 在显式
+  `VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` 下成功；该 override 只支持
+  performance/shape fidelity，不形成生成质量或模型长上下文正确性 claim，
+  并作为 provenance 中的显式实验变量。
+- profile-v6-code-longctx 覆盖 TP1/2/4、KV context 到 131072，
+  33 个 long-context rows；两次 fresh-process repeat 的最大相对差
+  4.648%，旧 anchor drift 最大 1.7%。attention profile SHA256 =
+  `fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550`。
+- TP4 `max_model_len=147456` smoke 已在 dash4 通过。server 日志同时确认
+  `max_model_len=147456` 与 `hf_overrides.max_position_embeddings=147456`；
+  20,051+78、119,702+68、136,774+242 三个 shape 均成功。对应
+  TTFT=853.29/12,699.04/3,820.55ms，TPOT=16.65/7.21/8.13ms。
+  最长请求的非单调 TTFT 来自 cold compile/cache state，因此这里只作为
+  runtime support gate，不作为 profile accuracy 数据。
+- calibration 全部使用同一 profile-v6 SHA。TP4 的 `rho=0.0016`
+  decode batch max=16、drain=21.07s，仍通过 10% 亚临界 gate；TP2 的
+  `rho=0.0016` waiting p50=332.97s、drain=976.86s，明确过载并排除。
+  完整 compact table 在 `results/calibration-summary.json`。
+- TP2/TP4 的 `rho={0.0002,0.0004,0.0008,0.0016}` 61min full paired
+  inputs 已在 CPFS 物化；每个 paired row digest 和 Frontier CSV SHA
+  均与 calibration input 逐项一致。manifest 副本在
+  `results/paired-input-manifests/`。最大一个目录约 901MiB，不把大型
+  token arrays 提交进 Git。
+- code prefill-only 的最大 calibration cache 已物化：
+  3477 requests、总 prefill 115,828,371 tokens、OSL 全为 1，
+  paired digest=`40865068e02414612ba1cd4595894e20e85e01fd34d73f8660185552d531ecea`。
+- 多 host 并发 server startup 暴露出 shared CPFS AOT cache 和每-job
+  FlashInfer JIT 的 apparatus cost。它发生在 readiness 前，不进入 TTFT；
+  runner commit `d5bb974` 改为长上下文默认使用 host-local vLLM cache，
+  并按 topology 复用 FlashInfer workspace。
+- 第一轮 paired real canary 的 3 次旧 client 运行都只在同一个
+  `106709+197` 请求失败，根因是 `return_token_ids` 把 100k+ prompt
+  vector 放进单条 SSE event，超过 aiohttp 默认 512KiB line limit。
+  commit `e1f2557` 把 exact client read buffer 提到 8MiB；700KiB
+  单-event runtime 对照和随后 TP4×2、TP2×1 的 53/53 replay 均通过。
+- TP4 canary 的 real-vs-sim prefix hit ratio =
+  `0.239908/0.239973`；pure-decode batch max 都为 4，
+  `share(b>1)=15.87%/15.69%`（real 两 trial）vs `16.35%`（sim）。
+  TTFT mean/p50/p90/p95/p99 bias =
+  `-8.3/-4.1/-8.8/-13.1/-6.3%`，E2E =
+  `+2.2/+6.1/+11.0/+1.4/-1.3%`。长 drain 的同一
+  `61976+21361` 请求 real=92.61/92.34s、sim=91.37s，不是 backlog。
+- TP4 若把 OSL=4 请求纳入 TPOT，mean/p99 bias 会被单个
+  `~213ms/token` 样本放大到 `-29.9%/-71.5%`；OSL≥8 后
+  mean/p50/p90/p95/p99 bias =
+  `+8.7/+8.2/+0.6/+13.4/+3.7%`。因此 raw TPOT 仍保留，但正式报告必须
+  同时给 OSL threshold sensitivity，不能把短输出的三段 inter-token
+  interval 当作稳定 decode service。
+- TP4 canary 两 trial 的 real SLO pass rate 都是 `50/53=94.34%`，
+  sim 为 `52/53=98.11%`，在 95% feasibility threshold 上发生翻转；
+  这由两个临界 TTFT 请求和上述 OSL=4 请求共同造成，作为明确 bad case
+  进入 1h 检验，不能被总体 latency gap 掩盖。
+- TP2 canary 的 cache/batch/drain 仍对齐，但 TTFT p90 bias=`-32.1%`，
+  E2E p90/p95=`-18.4%/-23.8%`。因此先只启动 low-rho 1h diagnostic，
+  不直接铺满 TP2 六个正式 jobs。
+- code prefill-only 已完成 10-cell Frontier calibration。TP2
+  `rho=0.0032` drain=1384.59s、waiting p50=751.01s，明确过载；
+  `0.0004/0.0008/0.0016` 冻结为 low/mid/near-knee。TP4 到
+  `rho=0.0032` 仍只有 9.09s drain，暂称 highest-tested，追加更高 rho
+  后才冻结 near-knee。compact table 在
+  `results/prefill-only-calibration-summary.json`。
+- 2026-07-23 18:37 UTC 启动 A4 wave 1：dash1=`TP4/rho0.0002/t1`、
+  dash3=`TP4/rho0.0008/t1`、dash4=`TP4/rho0.0016/t1`、
+  dash2=`TP2/rho0.0002/t1 diagnostic`；四台启动前再次确认 8×H20
+  memory/utilization=0、无 compute process、uncorrected ECC=0。

runs/frontier-code-trace-v0/prepare_code_window.py

@@ -0,0 +1,116 @@
+#!/usr/bin/env python3
+"""Materialize the stable code window selected by audit_code_trace.py."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+from pathlib import Path
+from typing import Any
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--audit", type=Path, required=True)
+    parser.add_argument("--output-root", type=Path, required=True)
+    parser.add_argument("--sample-seed", type=int, default=20260723)
+    return parser.parse_args()
+
+
+def session_uniform(seed: int, window_id: str, session_root: Any) -> float:
+    payload = json.dumps(
+        {"seed": seed, "window_id": window_id, "session_root": session_root},
+        sort_keys=True,
+        separators=(",", ":"),
+    ).encode()
+    return int.from_bytes(hashlib.blake2b(payload, digest_size=8).digest(), "big") / (
+        1 << 64
+    )
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as stream:
+        for chunk in iter(lambda: stream.read(1 << 20), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def main() -> None:
+    args = parse_args()
+    audit = json.loads(args.audit.read_text())
+    if audit["data_gate"] != "PASS":
+        raise ValueError(f"trace data gate is not PASS: {audit['data_gate']}")
+    selected = audit["selected"]
+    source = Path(selected["source"])
+    window = selected["stable_window"]
+    start = float(window["start_timestamp"])
+    end = float(window["end_timestamp"])
+    if args.output_root.exists():
+        raise ValueError(f"refusing to overwrite {args.output_root}")
+    args.output_root.mkdir(parents=True)
+    destination = args.output_root / "code-raw-window.jsonl"
+    root_of: dict[Any, Any] = {}
+    request_count = 0
+    with source.open() as input_stream, destination.open("w") as output_stream:
+        for source_index, line in enumerate(input_stream):
+            if not line.strip():
+                continue
+            row = json.loads(line)
+            timestamp = float(row["timestamp"])
+            if timestamp < start:
+                continue
+            if timestamp >= end:
+                break
+            if int(row["input_length"]) <= 0 or int(row["output_length"]) <= 0:
+                continue
+            chat = row.get("chat_id", source_index)
+            parent = row.get("parent_chat_id")
+            has_parent = parent not in (None, "", -1, "-1")
+            session_root = root_of.get(parent, parent) if has_parent else chat
+            root_of[chat] = session_root
+            materialized = {
+                **row,
+                "source_index": source_index,
+                "session_root": session_root,
+                "sampling_u": session_uniform(
+                    args.sample_seed,
+                    f"code-{start:.6f}-{end:.6f}",
+                    session_root,
+                ),
+            }
+            output_stream.write(
+                json.dumps(materialized, ensure_ascii=False, separators=(",", ":"))
+                + "\n"
+            )
+            request_count += 1
+    expected = int(selected["selected_window_stats"]["requests"])
+    if request_count != expected:
+        raise ValueError(f"window request mismatch: materialized={request_count}, audit={expected}")
+    manifest = {
+        "schema": "frontier-code-window-v1",
+        "audit": str(args.audit.resolve()),
+        "audit_sha256": sha256(args.audit),
+        "source": str(source.resolve()),
+        "source_block_size": selected["hash_contract"]["exact_source_block_size"],
+        "target_block_size": 16,
+        "start_timestamp": start,
+        "end_timestamp": end,
+        "duration_s": end - start,
+        "requests": request_count,
+        "sample_seed": args.sample_seed,
+        "sampling_rule": "session-coherent deterministic sampling_u",
+        "max_model_len": audit["max_model_len_recommendation"],
+        "window_stats": selected["selected_window_stats"],
+        "raw_window": str(destination.resolve()),
+        "raw_window_sha256": sha256(destination),
+    }
+    (args.output_root / "window-manifest.json").write_text(
+        json.dumps(manifest, indent=2, sort_keys=True) + "\n"
+    )
+    print(json.dumps({"requests": request_count, "output_root": str(args.output_root)}))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/attention.csv

@@ -0,0 +1,277 @@
+time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type,is_true_mixed_batch,prefill_seq_lens,prefill_kv_cache_sizes,decode_kv_cache_sizes,num_prefill_seqs,num_decode_seqs,decode_batch_size,total_batch_size,total_decode_tokens,decode_avg_kv_cache_size,batch_composition_ratio,batch_spec,projection_policy
+0.0,0.0,0.0,0.0,0.0,0.01414399966597557,0.028863999992609024,0.019705599918961526,0.01771199982613325,0.005157200849836681,0.047968000173568726,0.07046400010585785,0.05810240097343922,0.05810240097343922,0.007477463486041561,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.04947200044989586,0.020412799902260303,0.01635199971497059,0.010107497379722417,0.046560000628232956,0.08323200047016144,0.05587520003318787,0.05587520003318787,0.011126758739503428,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.022207999601960182,0.017033600155264138,0.015312000177800655,0.002819235991970241,0.05104000121355057,0.07692799717187881,0.056396800279617305,0.056396800279617305,0.007481982178637539,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.022272000089287758,0.01706880023702979,0.01616000011563301,0.002460889579319197,0.06931199878454208,0.0838719978928566,0.07432000041007995,0.07432000041007995,0.004777766433175866,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.018592000007629395,0.028543999418616295,0.02095999978482723,0.019183999858796597,0.003198175496053494,0.12179200351238251,0.15408000349998474,0.1307712011039257,0.1307712011039257,0.00858807797538298,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.027775999158620834,0.03385600075125694,0.030131200328469276,0.029680000618100166,0.0021152558103575215,0.32678401470184326,0.3450239896774292,0.33396480381488797,0.33396480381488797,0.0045872424917606375,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04438399896025658,0.05084799975156784,0.046540799736976626,0.04531199857592583,0.002277905811237223,1.0959680080413818,1.1151360273361206,1.0999775886535645,1.0999775886535645,0.005694403246120485,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.078015998005867,0.08691199868917465,0.08114239946007729,0.08019199967384338,0.00292795706334475,4.070400238037109,4.113152027130127,4.087088012695312,4.087088012695312,0.013660567012509554,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016063999384641647,0.05167999863624573,0.022115200012922286,0.017583999782800674,0.010340094822340818,0.05196800082921982,0.09011200070381165,0.06328320093452933,0.06328320093452933,0.012557341255467452,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01583999954164028,0.026623999699950218,0.018927999772131443,0.017376000061631203,0.003514650316260619,0.06681600213050842,0.07993599772453308,0.0725280001759529,0.0725280001759529,0.004343502558613716,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01648000068962574,0.030880000442266464,0.01945280022919178,0.017967999912798405,0.004096211183007485,0.1311360001564026,0.1546880006790161,0.13908160030841826,0.13908160030841826,0.007511906874366178,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.03558399900794029,0.020851199887692927,0.018559999763965607,0.005235911594130716,0.32950401306152344,0.350271999835968,0.33912960588932034,0.33912960588932034,0.006027400986663648,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.019328000023961067,0.040608000010252,0.022790400311350822,0.020704000256955624,0.006113051965778337,1.1415679454803467,1.1518720388412476,1.144483208656311,1.144483208656311,0.0032332311374389127,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015807999297976494,0.030688000842928886,0.019596799835562707,0.01774400006979704,0.004343384771033462,0.0,0.0,0.0,0.0,0.0,0.049056001007556915,0.07580800354480743,0.05948160067200661,0.05948160067200661,0.009031541471446955,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01603199914097786,0.02486399933695793,0.01923839971423149,0.018079999834299088,0.0032282528537266424,0.0,0.0,0.0,0.0,0.0,0.05142400041222572,0.07353600114583969,0.059328000620007516,0.059328000620007516,0.0073307807735143084,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016543999314308167,0.03977600112557411,0.021379199624061585,0.018511999398469925,0.006593576176246171,0.0,0.0,0.0,0.0,0.0,0.0488319993019104,0.06435199826955795,0.05479039996862411,0.05479039996862411,0.005672522998491864,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01635199971497059,0.02844800055027008,0.019267200119793416,0.017952000722289085,0.0035068687666949577,0.0,0.0,0.0,0.0,0.0,0.049855999648571014,0.07798399776220322,0.05986879989504815,0.05986879989504815,0.01043914754878828,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016383999958634377,0.026079999282956123,0.01923519968986511,0.017791999503970146,0.0032161974331284568,0.0,0.0,0.0,0.0,0.0,0.058111999183893204,0.1045759990811348,0.06708480007946492,0.06708480007946492,0.013479022462646494,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.04057599976658821,0.019100800156593323,0.015455999877303839,0.007512281243011577,0.0,0.0,0.0,0.0,0.0,0.05363199859857559,0.07782399654388428,0.06090559959411622,0.06090559959411622,0.007544620176348091,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.02191999927163124,0.016684799920767546,0.01563199982047081,0.0024293118621811216,0.0,0.0,0.0,0.0,0.0,0.0629120022058487,0.07891199737787247,0.06891520097851753,0.06891520097851753,0.005472695665695425,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.038943998515605927,0.018313600029796363,0.01561600062996149,0.007127270260115769,0.0,0.0,0.0,0.0,0.0,0.08675199747085571,0.10662399977445602,0.09391999915242194,0.09391999915242194,0.006988099589086635,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.054687999188899994,0.021439999900758268,0.01539199985563755,0.012052764849597775,0.0,0.0,0.0,0.0,0.0,0.13488000631332397,0.1528639942407608,0.1431359991431236,0.1431359991431236,0.005436271464033599,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.041919998824596405,0.01899839974939823,0.015343999955803156,0.007989843526623287,0.0,0.0,0.0,0.0,0.0,0.06176000088453293,0.08374399691820145,0.06747519969940186,0.06747519969940186,0.0066067747128778,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016256000846624374,0.11353600025177002,0.042761600017547606,0.028960000723600388,0.029104301538020762,0.0,0.0,0.0,0.0,0.0,0.09734400361776352,0.14422400295734406,0.11392960175871848,0.11392960175871848,0.013198594600417867,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.034143999218940735,0.018918400071561335,0.01643200032413006,0.005500943993080684,0.0,0.0,0.0,0.0,0.0,0.12918399274349213,0.15087999403476715,0.13807999789714814,0.13807999789714814,0.007658330538677587,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016063999384641647,0.03641600161790848,0.0198208000510931,0.01780799962580204,0.0057264128169845765,0.0,0.0,0.0,0.0,0.0,0.22099199891090393,0.23904000222682953,0.2293503984808922,0.2293503984808922,0.004861342907006028,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.035840000957250595,0.018908800091594458,0.015792000107467175,0.0064374817924757475,0.0,0.0,0.0,0.0,0.0,0.10134399682283401,0.12201599776744843,0.10896319895982742,0.10896319895982742,0.006336330809165179,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.03417599946260452,0.017846399918198586,0.014800000004470348,0.006495007539635255,0.0,0.0,0.0,0.0,0.0,0.13126400113105774,0.15561600029468536,0.1389280006289482,0.1389280006289482,0.008381472811075022,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.03519999980926514,0.019168000388890504,0.01600000075995922,0.005995522477654695,0.0,0.0,0.0,0.0,0.0,0.21728000044822693,0.2343679964542389,0.2231455981731415,0.2231455981731415,0.004720730646739123,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.03670400008559227,0.018441599886864425,0.015023999847471714,0.006596535162793127,0.0,0.0,0.0,0.0,0.0,0.39321601390838623,0.4524799883365631,0.4058080047369003,0.4058080047369003,0.01578349755088941,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.022143999114632607,0.016912000067532063,0.015168000012636185,0.0028156089295136347,0.0,0.0,0.0,0.0,0.0,0.15587200224399567,0.3079040050506592,0.17838079929351805,0.17838079929351805,0.04355865575265927,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.02112000063061714,0.015516800060868263,0.01488000014796853,0.001940870731593904,0.0,0.0,0.0,0.0,0.0,0.21587200462818146,0.23561599850654602,0.22250880002975468,0.22250880002975468,0.006181951170646666,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.039264000952243805,0.023609600123018028,0.02131200022995472,0.007236979625711548,0.0,0.0,0.0,0.0,0.0,0.408735990524292,0.470335990190506,0.4336863994598388,0.4336863994598388,0.01844662383160074,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.025407999753952026,0.016227199975401164,0.014960000291466713,0.0031617375441736185,0.0,0.0,0.0,0.0,0.0,0.7412800192832947,0.7627840042114258,0.7464000046253203,0.7464000046253203,0.006112167448837547,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.02812799997627735,0.020652799773961304,0.021359999664127827,0.004308706957613102,0.028383498565450627,0.039859687970646644,0.032492258074592426,0.032492258074592426,0.00453597266208597,0.029312501176103633,0.04116431058787463,0.03355574193327539,0.03355574193327539,0.004684436757701648,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.034143999218940735,0.023171199765056372,0.024255999363958836,0.005908614918096971,0.03333159243114438,0.038935341782478095,0.03580428402241854,0.03580428402241854,0.002082270297044095,0.03633240903369937,0.04244065945634484,0.03902771507087562,0.03902771507087562,0.0022697354261490147,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02611199952661991,0.019635199941694735,0.018112000077962875,0.0038298050749257795,0.04189529417991216,0.057484239920526384,0.04744885718421094,0.04744885718421094,0.004779830748455743,0.051672703037266184,0.07089975418195164,0.05852234134479412,0.05852234134479412,0.005895334539786378,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.029311999678611755,0.018662399891763926,0.01673599984496832,0.0044017112162725355,0.04322973959325901,0.05049827064705393,0.04535414343408448,0.04535414343408448,0.0022944156000240697,0.08733025617719538,0.10201372836398578,0.09162185574241775,0.09162185574241775,0.004635047631846023,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.026367999613285065,0.016336000058799982,0.01515199989080429,0.003415353455946402,0.06031842775160765,0.06618323188375198,0.06274415549817247,0.06274415549817247,0.001985647289644255,0.14768157653992678,0.16204076249052324,0.15362064543185072,0.15362064543185072,0.004861590945216247,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.02454400062561035,0.017167999967932702,0.016191999427974224,0.0028685016454498436,0.09128700688359712,0.09689150775996329,0.09360396051475776,0.09360396051475776,0.0013477542744916764,0.2740889887523892,0.2909164873209846,0.2810456356995366,0.2810456356995366,0.004046628526808561,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01881599985063076,0.03097599931061268,0.024598400108516216,0.024848000146448612,0.0037539437391565975,0.1035249255866932,0.10603132147437412,0.10399797220840973,0.10399797220840973,0.0007025109429056814,0.5652750707893444,0.5789606938874117,0.5678580377517648,0.5678580377517648,0.003835906384194897,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018624000251293182,0.043487999588251114,0.024460799992084503,0.019600000232458115,0.008922081629781394,0.196169204945307,0.2076140047945799,0.2008444429250931,0.2008444429250931,0.00394350953292805,1.1196708009293268,1.1849940417370974,1.1463555573610091,1.1463555573610091,0.022508285530529783,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.027135999873280525,0.03667199984192848,0.02945920005440712,0.028256000019609928,0.0029446678408169553,0.37757279619664613,0.3898113624476372,0.38075520430942184,0.38075520430942184,0.0036600203831042254,0.2522831942132049,0.2604606493092597,0.25440958703617433,0.25440958703617433,0.0024455194930253126,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.0435199998319149,0.049695998430252075,0.04502719938755036,0.04395199939608574,0.002035785660169308,1.1048984388245497,1.1189053886476108,1.1112371236754128,1.1112371236754128,0.0050220696724624985,0.1395495077239122,0.14131859607071193,0.14035008841031085,0.14035008841031085,0.0006342911944856293,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018783999606966972,0.024288000538945198,0.020627199858427047,0.019567999988794327,0.0019457052717059358,0.09455999732017517,0.12185599654912949,0.10207359939813614,0.10207359939813614,0.00753346544014261,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.027295999228954315,0.034752000123262405,0.02943360023200512,0.028528000228106976,0.0023159160681123767,0.14416000247001648,0.15904000401496887,0.14979200065135959,0.14979200065135959,0.00480624675866005,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043455999344587326,0.05084799975156784,0.045500800386071204,0.04411200061440468,0.002490556062831049,0.24454399943351746,0.25865599513053894,0.2504959970712662,0.2504959970712662,0.003778494140301353,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07673600316047668,0.08374399691820145,0.07970559895038605,0.07873599976301193,0.0022907976135004057,0.445248007774353,0.4758400022983551,0.45409599840641024,0.45409599840641024,0.009133230340383306,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043296001851558685,0.051072001457214355,0.044972800090909,0.04399999976158142,0.002451216824441962,0.6147199869155884,0.6290879845619202,0.6204927921295166,0.6204927921295166,0.004344846739788608,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07737600058317184,0.09123200178146362,0.08094720020890236,0.07980800047516823,0.004065264798368611,1.1744320392608643,1.1887680292129517,1.178323209285736,1.178323209285736,0.004395876469319665,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.03254399821162224,0.01810879958793521,0.01611199975013733,0.005058478889747962,0.0,0.0,0.0,0.0,0.0,0.06339199841022491,0.0841279998421669,0.07019519805908202,0.07019519805908202,0.006070126182471763,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.030271999537944794,0.018124799989163876,0.015711999498307705,0.004696400814632773,0.0,0.0,0.0,0.0,0.0,0.2739199995994568,0.2922239899635315,0.28281279802322384,0.28281279802322384,0.00633037978599564,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022431999444961548,0.016844799742102623,0.015471999999135733,0.0027624803153841917,0.0,0.0,0.0,0.0,0.0,0.3909119963645935,0.4160960018634796,0.39996159672737125,0.39996159672737125,0.007037174636804898,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014240000396966934,0.035071998834609985,0.018764800019562246,0.015583999920636415,0.006305594700523817,0.0,0.0,0.0,0.0,0.0,0.7383679747581482,0.7597119808197021,0.7430047929286957,0.7430047929286957,0.006192645960911466,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.028991999104619026,0.019401599932461978,0.01665600063279271,0.005434953793780546,0.0,0.0,0.0,0.0,0.0,1.427008032798767,1.4517120122909546,1.4361984014511109,1.4361984014511109,0.008802755821028187,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.02844800055027008,0.01820160010829568,0.014944000169634819,0.004942002036909087,0.0,0.0,0.0,0.0,0.0,0.08361600339412689,0.11027199774980545,0.09058240056037903,0.09058240056037903,0.00872938604423622,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.023104000836610794,0.01648960020393133,0.015632000286132097,0.0027628828340349578,0.0,0.0,0.0,0.0,0.0,0.5063040256500244,0.5497599840164185,0.5168287932872773,0.5168287932872773,0.013557229606313293,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01616000011563301,0.029055999591946602,0.018780799955129622,0.017215999774634838,0.003794434947431941,0.0,0.0,0.0,0.0,0.0,0.7439360022544861,0.7719680070877075,0.7502080142498017,0.7502080142498017,0.00839205598262567,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015968000516295433,0.03046399913728237,0.018822400271892546,0.01726400014013052,0.004212536831927695,0.0,0.0,0.0,0.0,0.0,1.4256000518798828,1.449504017829895,1.4325888037681578,1.4325888037681578,0.0065448028108711165,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.027712000533938408,0.01633920017629862,0.014944000169634819,0.0038488220071983326,0.0,0.0,0.0,0.0,0.0,2.798719882965088,3.0184640884399414,2.8293471813201903,2.8293471813201903,0.06375662767704417,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.021407999098300934,0.01637439979240298,0.014992000069469213,0.0024463594774459265,0.0,0.0,0.0,0.0,0.0,0.09347199648618698,0.10473600029945374,0.09820479974150656,0.09820479974150656,0.004061668684999473,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.04790399968624115,0.020179199893027543,0.016447999514639378,0.009635010283506967,0.0,0.0,0.0,0.0,0.0,0.6221439838409424,0.6444799900054932,0.6277984082698822,0.6277984082698822,0.007234140179397069,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.03276799991726875,0.01921279989182949,0.015232000034302473,0.0065506148511265076,0.0,0.0,0.0,0.0,0.0,0.9111359715461731,0.9307839870452881,0.9151648044586183,0.9151648044586183,0.005528712062927265,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.02707199938595295,0.016512000095099212,0.014688000082969666,0.0038989080209321414,0.0,0.0,0.0,0.0,0.0,1.7645119428634644,1.7849279642105103,1.7684095859527589,1.7684095859527589,0.005876963340184193,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014240000396966934,0.029184000566601753,0.016825600154697896,0.014752000104635954,0.00457910514148248,0.0,0.0,0.0,0.0,0.0,3.4781761169433594,3.5388801097869873,3.490892815589905,3.490892815589905,0.01818910600692522,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014976000413298607,0.03139200061559677,0.01898880014196038,0.016848000697791576,0.005083715524393418,0.13752702814163098,0.14117629917511842,0.13848196486571557,0.13848196486571557,0.0009880564964713527,0.523336967410432,0.5372236808930884,0.5269708253945994,0.5269708253945994,0.00375988994658546,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01833599992096424,0.02707199938595295,0.020995199866592883,0.01976000051945448,0.0028392656352050185,0.17817885890237703,0.18622915300900206,0.18109068484526028,0.18109068484526028,0.0027040006080457624,0.5449571488834201,0.5695788427872015,0.5538629212834322,0.5538629212834322,0.008270141985514729,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.03308799862861633,0.028883199393749236,0.02759999968111515,0.002285758578932753,0.4688266550410815,0.48301665772804186,0.4727750380198454,0.4727750380198454,0.004396396389258256,1.0430453981052807,1.0746153117238624,1.051829759343436,1.051829759343436,0.009781101336187197,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.04368000105023384,0.059328000992536545,0.04809600040316582,0.04617599956691265,0.005158792915002645,1.3509461459747514,1.3759066409666496,1.3589365122072,1.3589365122072,0.008008877716183384,0.9213738861449998,0.9383974724214122,0.9268234851606594,0.9268234851606594,0.005462224239661061,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.026944000273942947,0.040511999279260635,0.030291200056672095,0.02817599941045046,0.004386386489305873,0.5071830964059985,0.5142792136001758,0.5096392092471257,0.5096392092471257,0.0022059272685869546,2.175632932188972,2.2060727208328075,2.186168772243963,2.186168772243963,0.009462633998570079,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018303999677300453,0.03017600066959858,0.02039040010422468,0.018943999893963337,0.0034612051983613614,0.21341429693945616,0.21552776300295579,0.21393496609766816,0.21393496609766816,0.0005736773262009222,4.617401495144284,4.663128147488988,4.628666619290515,4.628666619290515,0.012412001359412127,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.02364799939095974,0.01809599995613098,0.016352000646293163,0.0035481127058959038,0.04822399839758873,0.08566399663686752,0.05961279980838299,0.05961279980838299,0.011445665413968877,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.020864000543951988,0.01668160008266568,0.015664000064134598,0.0025769063833097584,0.049695998430252075,0.08057600259780884,0.05882879942655563,0.05882879942655563,0.009515126108519331,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.028704000636935234,0.017430400010198355,0.015056000091135502,0.004349294613335555,0.049855999648571014,0.07366400212049484,0.05459520071744919,0.05459520071744919,0.0069610920757925574,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.03161599859595299,0.017788799852132796,0.015711999963968992,0.005005385723318659,0.06102399900555611,0.08179199695587158,0.06650560013949873,0.06650560013949873,0.006947995595801105,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.04416000097990036,0.019200000166893005,0.015488000120967627,0.008561241323364038,0.08054400235414505,0.09196799993515015,0.08607039973139763,0.08607039973139763,0.004145329035483754,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.023391999304294586,0.019667199812829494,0.018463999964296818,0.0022577669687832585,0.18729600310325623,0.20585599541664124,0.19546559900045393,0.19546559900045393,0.006339068824303663,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.026240000501275063,0.03446400165557861,0.028297600522637367,0.027088000439107418,0.0025148441522922374,0.5754240155220032,0.5889919996261597,0.5800191938877105,0.5800191938877105,0.003858869273829596,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.047200001776218414,0.043036799877882004,0.0423360001295805,0.0017073405772076728,2.063199996948242,2.0787200927734375,2.067151999473572,2.067151999473572,0.004959651271127835,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.05648000165820122,0.02270399993285537,0.017935999669134617,0.012377915150727689,0.049536000937223434,0.07196799665689468,0.056015999615192415,0.056015999615192415,0.0070476637552742884,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01408000010997057,0.021344000473618507,0.01611520005390048,0.014928000047802925,0.0025499884993961702,0.07072000205516815,0.2642880082130432,0.1588256008923054,0.1588256008923054,0.053220347086102376,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01360000018030405,0.03014400042593479,0.016975999902933837,0.015056000091135502,0.004715159697364111,0.08393599838018417,0.11036799848079681,0.09160000011324881,0.09160000011324881,0.007911669434472792,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.020767999812960625,0.016233599931001663,0.015392000321298838,0.002202615907995427,0.1998399943113327,0.22070400416851044,0.20855360180139543,0.20855360180139543,0.00689307200230667,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.033440001308918,0.018684800155460833,0.016048000194132328,0.00553991005639174,0.6043199896812439,0.635807991027832,0.6126143991947173,0.6126143991947173,0.008745933953408096,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013824000023305416,0.03359999880194664,0.017811199743300678,0.014944000169634819,0.005926140483565472,0.0,0.0,0.0,0.0,0.0,0.045471999794244766,0.07539200037717819,0.054758400097489356,0.054758400097489356,0.010253548506101549,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.02409599907696247,0.01641279999166727,0.01473599998280406,0.003347158638713987,0.0,0.0,0.0,0.0,0.0,0.0461760014295578,0.07529599964618683,0.05460800044238568,0.05460800044238568,0.009748937798340135,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.028416000306606293,0.018611199874430894,0.01598400017246604,0.005209875093863647,0.0,0.0,0.0,0.0,0.0,0.048128001391887665,0.07897599786520004,0.061353600397706036,0.061353600397706036,0.010488153655157845,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.025728000327944756,0.016092800162732603,0.01512000011280179,0.003300888123052605,0.0,0.0,0.0,0.0,0.0,0.04864000156521797,0.07648000121116638,0.05810560062527656,0.05810560062527656,0.009473544218404408,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03551999852061272,0.018588799890130757,0.016064000315964222,0.006071057437992447,0.0,0.0,0.0,0.0,0.0,0.04822399839758873,0.07862400263547897,0.05660480037331582,0.05660480037331582,0.009565394213730401,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.028831999748945236,0.01699519995599985,0.015024000313133001,0.004285000744868188,0.0,0.0,0.0,0.0,0.0,0.04854400083422661,0.06719999760389328,0.05778240002691746,0.05778240002691746,0.006852125554679805,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.024927999824285507,0.0161183999851346,0.01521599991247058,0.0029774808524673907,0.0,0.0,0.0,0.0,0.0,0.05379199981689453,0.08966399729251862,0.06270079985260964,0.06270079985260964,0.009983591277092696,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.03001599945127964,0.017648000083863736,0.01547200046479702,0.004585126309484848,0.0,0.0,0.0,0.0,0.0,0.061664000153541565,0.07692799717187881,0.06704320013523103,0.06704320013523103,0.005500191798490629,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.026367999613285065,0.016947199776768684,0.015039999969303608,0.0037951566103550205,0.0,0.0,0.0,0.0,0.0,0.08799999952316284,0.111455999314785,0.0964031994342804,0.0964031994342804,0.007397541615558088,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01369599997997284,0.021247999742627144,0.015359999984502793,0.014640000183135271,0.0020942770950814317,0.0,0.0,0.0,0.0,0.0,0.051711998879909515,0.07065600156784058,0.058054400235414506,0.058054400235414506,0.006633034815910025,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014208000153303146,0.0307839997112751,0.017148799914866685,0.015039999969303608,0.004882374686312284,0.0,0.0,0.0,0.0,0.0,0.061919998377561576,0.07843200117349625,0.06628479920327664,0.06628479920327664,0.004962852689801192,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013887999579310417,0.02969600073993206,0.017500799987465142,0.015232000034302473,0.00467273319705314,0.0,0.0,0.0,0.0,0.0,0.08819200098514557,0.11097600311040878,0.09493440166115762,0.09493440166115762,0.007509235042985577,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.022112000733613968,0.01751680001616478,0.016047999262809753,0.0030306621792915785,0.0,0.0,0.0,0.0,0.0,0.13065600395202637,0.15110400319099426,0.13857279866933822,0.13857279866933822,0.00750137841249771,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.04012800008058548,0.01892479993402958,0.01550400024279952,0.007459545267816961,0.0,0.0,0.0,0.0,0.0,0.06278400123119354,0.08259200304746628,0.0704512007534504,0.0704512007534504,0.005979055984382744,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.02236800082027912,0.016332800220698118,0.014928000047802925,0.002784044363186731,0.0,0.0,0.0,0.0,0.0,0.1003199964761734,0.1279360055923462,0.10921279862523078,0.10921279862523078,0.00862769617046716,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.0225600004196167,0.016128000058233737,0.014800000004470348,0.002958953332547708,0.0,0.0,0.0,0.0,0.0,0.13116799294948578,0.14812800288200378,0.13783999979496003,0.13783999979496003,0.005696384361148053,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014208000153303146,0.029983999207615852,0.017468800116330386,0.01508800033479929,0.0047379550962483065,0.0,0.0,0.0,0.0,0.0,0.217631995677948,0.2447360008955002,0.22715839892625808,0.22715839892625808,0.008831828221847138,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.02304000034928322,0.016512000095099212,0.014512000139802694,0.0035026774828624254,0.0,0.0,0.0,0.0,0.0,0.11020799726247787,0.12307199835777283,0.11600959971547126,0.11600959971547126,0.004667637266950902,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.042080000042915344,0.017510399967432023,0.014607999939471483,0.00823117883530167,0.0,0.0,0.0,0.0,0.0,0.15702399611473083,0.17587199807167053,0.16399359852075576,0.16399359852075576,0.006588299074676393,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.0208320003002882,0.015299199987202883,0.01462399959564209,0.001916768086505386,0.0,0.0,0.0,0.0,0.0,0.21779200434684753,0.2415360063314438,0.2260768011212349,0.2260768011212349,0.007251352080685236,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014015999622642994,0.021088000386953354,0.015619200188666582,0.01473599998280406,0.002013227452302141,0.0,0.0,0.0,0.0,0.0,0.3959999978542328,0.4152640104293823,0.40332479774951924,0.40332479774951924,0.006942401431914052,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.03379200026392937,0.020595200080424547,0.019600000232458115,0.006548754248881344,0.026192623739694512,0.040006882507168426,0.029155180178492036,0.029155180178492036,0.00408828748438241,0.024591375524545756,0.037561119537986146,0.027372820355088746,0.027372820355088746,0.0038383559348575944,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.020896000787615776,0.015318400040268899,0.014431999996304512,0.0019640312960926966,0.029349018208693862,0.06236262941356679,0.03714313592014963,0.03714313592014963,0.009618313791872569,0.028826981462526914,0.06125337308649041,0.036482463672774496,0.036482463672774496,0.009447230957011863,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.032127998769283295,0.017286399938166143,0.014479999896138906,0.005536495446636193,0.03273085874558354,0.04394578491937082,0.03563837490653034,0.03563837490653034,0.003429601730256567,0.03488514202593899,0.04683821345079978,0.037984025407085474,0.037984025407085474,0.0036553316361902684,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.021503999829292297,0.015078400075435639,0.01425600005313754,0.002238435975478849,0.042100813549974185,0.051784144690147756,0.045378692890289625,0.045378692890289625,0.003184109940650555,0.05111518843748309,0.06287185663927425,0.05509490773570219,0.05509490773570219,0.0038658725544299132,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.02687999978661537,0.017667199857532977,0.01566399959847331,0.003864811846387173,0.048475323773821306,0.05599956978723733,0.05123499252968345,0.05123499252968345,0.002427379820423941,0.08429268135885387,0.09737642835214408,0.0890914090616175,0.0890914090616175,0.0042209177332077005,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.03359999880194664,0.019804799742996693,0.016032000072300434,0.006750519908145474,0.07121508474579985,0.07292308367136396,0.07194410845270183,0.07194410845270183,0.0005500065915943844,0.14760091249712767,0.15114092373010238,0.14911189243564577,0.14911189243564577,0.0011399477384397005,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.021856000646948814,0.016835200227797033,0.015263999812304974,0.00283854448975065,0.08146338272142935,0.08560865714384096,0.0834932625520734,0.0834932625520734,0.0012259635905347874,0.2782486219401307,0.29240733788179385,0.2851819365989658,0.2851819365989658,0.004187435731481665,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.04495999962091446,0.02143679987639189,0.015856000129133463,0.009709987872683342,0.1194459208702178,0.12302524755181463,0.12053885718696096,0.12053885718696096,0.001005118119341339,0.5597220650459199,0.5764947443228802,0.5648435507167102,0.5648435507167102,0.004709970715400788,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018112000077962875,0.026496000587940216,0.019318400137126445,0.018432000651955605,0.0024249605112359905,0.19770253574610402,0.222811797868348,0.2054811520619412,0.2054811520619412,0.008000944254868043,0.13941746080159492,0.157124211777114,0.14490284788179197,0.14490284788179197,0.005642170080515992,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.02595200017094612,0.03155200183391571,0.02727359998971224,0.026575999334454536,0.0016260948764843166,0.6014684881116659,0.6095472988212,0.6046757700946376,0.6046757700946376,0.0023537435563177303,0.11325152264578045,0.11477269562841545,0.11385542721485654,0.11385542721485654,0.0004431903698023628,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.03359999880194664,0.01814719969406724,0.015375999733805656,0.005678506740662529,0.06774400174617767,0.07891199737787247,0.07312640026211739,0.07312640026211739,0.003826583051422731,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01788800023496151,0.028543999418616295,0.021340799890458582,0.019504000432789326,0.003718627037219618,0.09548799693584442,0.1327359974384308,0.10823359936475753,0.10823359936475753,0.013230635292043864,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02627200074493885,0.030368000268936157,0.027184000052511693,0.02643200010061264,0.0013545679205210022,0.14364799857139587,0.1597760021686554,0.15008639842271806,0.15008639842271806,0.005367723415171222,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04150399938225746,0.04726399853825569,0.042950399965047834,0.04224000126123428,0.001720147501765378,0.2433920055627823,0.28963199257850647,0.2549152016639709,0.2549152016639709,0.013450991019687407,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025887999683618546,0.03407999873161316,0.028303999826312064,0.026367999613285065,0.0028877495368841042,0.325439989566803,0.3441599905490875,0.33442879617214205,0.33442879617214205,0.005693597811441922,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04137599840760231,0.05084799975156784,0.044828799366950986,0.043087998405098915,0.003560363865797613,0.6110399961471558,0.6421759724617004,0.6204223990440368,0.6204223990440368,0.011214490370794758,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014816000126302242,0.02364799939095974,0.016668799985200166,0.01592000015079975,0.0025515238179941247,0.0,0.0,0.0,0.0,0.0,0.0544000007212162,0.09014400094747543,0.06511679962277411,0.06511679962277411,0.013005726711295521,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.0208320003002882,0.016604799963533878,0.01566399959847331,0.0021353594266203244,0.0,0.0,0.0,0.0,0.0,0.17871999740600586,0.1961279958486557,0.18568639904260634,0.18568639904260634,0.004566142885274747,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.02006400004029274,0.015619200002402068,0.01508800033479929,0.001572984783715635,0.0,0.0,0.0,0.0,0.0,0.2730880081653595,0.29721599817276,0.2815328001976013,0.2815328001976013,0.007016534727616923,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.02755199931561947,0.018678399827331306,0.015919999685138464,0.004323555372382858,0.0,0.0,0.0,0.0,0.0,0.3893119990825653,0.44041600823402405,0.40225600004196166,0.40225600004196166,0.013442998165884852,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.02659199945628643,0.016748800035566093,0.015343999955803156,0.0036412341868394082,0.0,0.0,0.0,0.0,0.0,0.7404800057411194,0.9689919948577881,0.8451807916164398,0.8451807916164398,0.08637455920036795,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.02160000056028366,0.016752000153064727,0.01521599991247058,0.0025167650350367246,0.0,0.0,0.0,0.0,0.0,0.06412799656391144,0.08956799656152725,0.07242240011692047,0.07242240011692047,0.009797120588901621,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022911999374628067,0.016137600038200618,0.015343999955803156,0.002330911555470513,0.0,0.0,0.0,0.0,0.0,0.3128319978713989,0.3282879889011383,0.3203647971153259,0.3203647971153259,0.004598568238907996,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03574400022625923,0.01919359974563122,0.01583999954164028,0.006642521913489215,0.0,0.0,0.0,0.0,0.0,0.5050879716873169,0.5311999917030334,0.511932796239853,0.511932796239853,0.007150929647317677,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02595200017094612,0.016992000117897987,0.015456000342965126,0.003390731937723684,0.0,0.0,0.0,0.0,0.0,0.7385600209236145,0.7681919932365417,0.7483008027076722,0.7483008027076722,0.010214373356208012,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.038975998759269714,0.01952639976516366,0.015327999833971262,0.007695927285621061,0.0,0.0,0.0,0.0,0.0,1.4228800535202026,1.5237760543823242,1.4435008168220522,1.4435008168220522,0.0342955291725742,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.03907199949026108,0.01793599994853139,0.015248000156134367,0.007167103363234667,0.0,0.0,0.0,0.0,0.0,0.06947200000286102,0.08857599645853043,0.07469440028071403,0.07469440028071403,0.00583249952929747,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.02175999991595745,0.016710399929434062,0.015327999833971262,0.002705383977079183,0.0,0.0,0.0,0.0,0.0,0.388480007648468,0.4073280096054077,0.3966591984033584,0.3966591984033584,0.004487315516095964,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.022048000246286392,0.017011200170964004,0.015392000321298838,0.002776899649845722,0.0,0.0,0.0,0.0,0.0,0.622048020362854,0.6347839832305908,0.6262047946453094,0.6262047946453094,0.004593360122958751,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.02937600016593933,0.01847040019929409,0.015344000421464443,0.005043809142122341,0.0,0.0,0.0,0.0,0.0,0.9105280041694641,0.9304640293121338,0.914108806848526,0.914108806848526,0.00563919268816644,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.039903998374938965,0.01791359977796674,0.015440000221133232,0.007367547649297217,0.0,0.0,0.0,0.0,0.0,1.764799952507019,1.7965760231018066,1.770739197731018,1.770739197731018,0.009082122770320404,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.03187200054526329,0.018553599901497363,0.016127999871969223,0.0050628774268006264,0.16963527081512533,0.1737871320906266,0.17088926838108623,0.17088926838108623,0.0011703114258121128,0.47362872483230506,0.48522089403242025,0.47712993814280913,0.47712993814280913,0.0032675581298664976,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.03868800029158592,0.01921919994056225,0.016959999687969685,0.006691309533030663,0.1589600576212269,0.16117782913137854,0.15969057520605917,0.15969057520605917,0.0006212984448748182,0.5199519263456005,0.5272061673552852,0.5223414198520004,0.5223414198520004,0.002032242112153396,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018400000408291817,0.024320000782608986,0.019910399988293647,0.0191040001809597,0.001841532763984352,0.25650752966102763,0.27066608538463055,0.259736894547936,0.259736894547936,0.00462927315947332,0.5278764825612624,0.5570139063374764,0.5345223138928448,0.5345223138928448,0.009526755161797178,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.026048000901937485,0.03728000074625015,0.028636799938976765,0.02705600019544363,0.003313953649011259,0.9270006318443208,0.9712965120641314,0.93480767601568,0.93480767601568,0.012612551450284608,0.8181833128578277,0.8572794566782395,0.8250739157811953,0.8250739157811953,0.01113200873299586,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018079999834299088,0.037696000188589096,0.022092800214886667,0.019024000503122807,0.005986278786633071,0.2839857165542317,0.2876110048757805,0.2849506939696605,0.2849506939696605,0.0009995178425916847,1.0871823008331585,1.101060989333509,1.0908765231323903,1.0908765231323903,0.0038264533900426207,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.025567999109625816,0.01726400014013052,0.015728000551462173,0.0031324465940990903,0.137270464802061,0.13799613818579368,0.13751499486424038,0.13751499486424038,0.00025221761530472904,2.3021855212208884,2.314355908780759,2.3062865750744197,2.3062865750744197,0.004229983070201486,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.022784000262618065,0.016435200069099664,0.015519999898970127,0.0023688301421469523,0.04879999905824661,0.09139200299978256,0.06054079942405224,0.06054079942405224,0.012152608702448775,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014816000126302242,0.02844800055027008,0.017008000146597625,0.015696000307798386,0.003941466294662679,0.047807998955249786,0.07356800138950348,0.05626560002565384,0.05626560002565384,0.00842179125412236,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.053408000618219376,0.019353600218892097,0.01532800029963255,0.01138302400841626,0.048448000103235245,0.0785600021481514,0.0556256003677845,0.0556256003677845,0.009348274502616908,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.03139200061559677,0.01823679991066456,0.016159999649971724,0.004860071238302512,0.055424001067876816,0.08505599945783615,0.0640383992344141,0.0640383992344141,0.00921448636178623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.026528000831604004,0.017324799951165915,0.01536000007763505,0.0036004822686428305,0.07660800218582153,0.0942080020904541,0.08209280073642732,0.08209280073642732,0.00515895587669752,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.021247999742627144,0.016403199825435876,0.01532800029963255,0.001995897022647934,0.11395200341939926,0.15113599598407745,0.12431039959192276,0.12431039959192276,0.011164431123683732,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017184000462293625,0.028672000393271446,0.019865600019693376,0.017823999747633934,0.0035798491315929977,0.3171840012073517,0.3341119885444641,0.3261695951223373,0.3261695951223373,0.005111046452878918,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024639999493956566,0.030400000512599945,0.02656640000641346,0.02556800004094839,0.0020189846603237303,1.0648640394210815,1.0828479528427124,1.0715327858924866,1.0715327858924866,0.005558639263049851,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014976000413298607,0.021695999428629875,0.017305599898099898,0.01593599934130907,0.0025718046517268054,0.04956800118088722,0.07932800054550171,0.06228480041027069,0.06228480041027069,0.01027160349757827,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01539199985563755,0.03580800071358681,0.019686400331556796,0.017136000096797943,0.005918387811304003,0.05158400163054466,0.10713600367307663,0.06364160068333148,0.06364160068333148,0.015832957809696766,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.02223999984562397,0.016672000009566545,0.015887999907135963,0.0020934945946034563,0.06521599739789963,0.08902399986982346,0.07520959973335266,0.07520959973335266,0.007913840684102929,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015231999568641186,0.04028800129890442,0.019971200078725816,0.01646399963647127,0.007351305886577286,0.14467200636863708,0.16844800114631653,0.15363519936800005,0.15363519936800005,0.008174435899956223,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015519999898970127,0.02304000034928322,0.01775679988786578,0.016784000210464,0.0025077243712082584,0.33740800619125366,0.35343998670578003,0.3445120006799698,0.3445120006799698,0.00445648463590358,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015647999942302704,0.02393599972128868,0.01809599995613098,0.01654400024563074,0.002998393102466254,0.0,0.0,0.0,0.0,0.0,0.0504320003092289,0.0843840017914772,0.060083200410008426,0.060083200410008426,0.00986959318572296,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01603199914097786,0.030047999694943428,0.019510399922728537,0.017680000513792038,0.004065185265963332,0.0,0.0,0.0,0.0,0.0,0.05004800111055374,0.07036799937486649,0.059315200522542,0.059315200522542,0.006537768821329647,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.02393599972128868,0.018035199772566558,0.016512000001966953,0.0030667689116777724,0.0,0.0,0.0,0.0,0.0,0.05100800096988678,0.06735999882221222,0.058387200161814694,0.058387200161814694,0.005832787739241381,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.026208000257611275,0.017500799987465142,0.01646399963647127,0.0030666103306165714,0.0,0.0,0.0,0.0,0.0,0.05023999884724617,0.07254400104284286,0.057254400476813315,0.057254400476813315,0.006890853653068151,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.027327999472618103,0.0179776000790298,0.01648000068962574,0.003783417447531392,0.0,0.0,0.0,0.0,0.0,0.04819199815392494,0.07100799679756165,0.05621119923889638,0.05621119923889638,0.007824391484124725,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015647999942302704,0.036448001861572266,0.019136000238358975,0.016704000532627106,0.006076530095624803,0.0,0.0,0.0,0.0,0.0,0.047488000243902206,0.08441600203514099,0.0588383998721838,0.0588383998721838,0.011275940726332522,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015135999768972397,0.033952001482248306,0.02119360016658902,0.018000000156462193,0.007136646614305304,0.0,0.0,0.0,0.0,0.0,0.04931199923157692,0.06992000341415405,0.05755840018391609,0.05755840018391609,0.007297587694315226,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.034272000193595886,0.0204415999352932,0.017311999574303627,0.00695404701803013,0.0,0.0,0.0,0.0,0.0,0.05215999856591225,0.0735040009021759,0.06117440015077591,0.06117440015077591,0.007381118436894922,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.03743999823927879,0.02012479966506362,0.01775999926030636,0.006181045509079057,0.0,0.0,0.0,0.0,0.0,0.06355199962854385,0.08508799970149994,0.07019200026988984,0.07019200026988984,0.0062246580442117845,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.032607998698949814,0.01959999995306134,0.017487999983131886,0.004882475068460749,0.0,0.0,0.0,0.0,0.0,0.04918399825692177,0.0865280032157898,0.05973760038614274,0.05973760038614274,0.011546847942113974,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.02675200067460537,0.017430400010198355,0.016560000367462635,0.003243135094239819,0.0,0.0,0.0,0.0,0.0,0.052671998739242554,0.08367999643087387,0.06238719932734965,0.06238719932734965,0.011207824780630104,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.03612799942493439,0.019327999837696553,0.01688000001013279,0.006058251425153085,0.0,0.0,0.0,0.0,0.0,0.06364800035953522,0.07878399640321732,0.06935679838061332,0.06935679838061332,0.004515588328677643,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03711999952793121,0.019670399930328132,0.017152000218629837,0.006165994646522264,0.0,0.0,0.0,0.0,0.0,0.09071999788284302,0.11507199704647064,0.10252480059862136,0.10252480059862136,0.008782544051535657,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.026528000831604004,0.018396800104528665,0.01601599995046854,0.004279788048986283,0.0,0.0,0.0,0.0,0.0,0.05331199988722801,0.07977599650621414,0.06076480001211167,0.06076480001211167,0.008761579122069606,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.02643200010061264,0.01726400004699826,0.015855999663472176,0.003210008417242029,0.0,0.0,0.0,0.0,0.0,0.062431998550891876,0.08246400207281113,0.06970879957079888,0.06970879957079888,0.007016341676068233,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.02223999984562397,0.0166015999391675,0.015887999907135963,0.00204913969129354,0.0,0.0,0.0,0.0,0.0,0.10127999633550644,0.1141119971871376,0.10621120035648347,0.10621120035648347,0.004029318311754605,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.03363199904561043,0.019305599946528675,0.016671999357640743,0.0055445582307981234,0.0,0.0,0.0,0.0,0.0,0.1319040060043335,0.15839999914169312,0.14040640145540234,0.14040640145540234,0.007998041073596942,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014368000440299511,0.04447999969124794,0.018908800091594458,0.0157279996201396,0.0086814937461721,0.0,0.0,0.0,0.0,0.0,0.06428799778223038,0.08982399851083755,0.07349760085344315,0.07349760085344315,0.008605284512197258,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.028672000393271446,0.01890560006722808,0.016080000437796116,0.004797584627815021,0.0,0.0,0.0,0.0,0.0,0.11123199760913849,0.14115199446678162,0.11942399889230729,0.11942399889230729,0.008513394706791027,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.024927999824285507,0.016915200091898442,0.015216000378131866,0.003374147499442635,0.0,0.0,0.0,0.0,0.0,0.15887999534606934,0.18111999332904816,0.16934399753808976,0.16934399753808976,0.007415181260761123,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.026944000273942947,0.017737600207328796,0.015199999790638685,0.00415598714375255,0.0,0.0,0.0,0.0,0.0,0.2192319929599762,0.23472000658512115,0.22809920012950893,0.22809920012950893,0.004730841376327335,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.05052800104022026,0.020320000313222408,0.015520000364631414,0.010604522177924678,0.024270629882498958,0.03340247625954076,0.028446344104128624,0.028446344104128624,0.0032330369099793834,0.023697370291069768,0.03261352722995356,0.027774456325453972,0.027774456325453972,0.0031566742680923386,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.033663999289274216,0.019987199828028678,0.018240000121295452,0.005522929139253732,0.0259194055660947,0.03719755183990719,0.029916029687899703,0.029916029687899703,0.003966666590554318,0.027104595853449518,0.03889844644729374,0.03128396954022015,0.03128396954022015,0.004148046317967881,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.03299200162291527,0.019840000104159115,0.016207999549806118,0.006546343008726814,0.02587869595769926,0.03763167265431482,0.030174939058162802,0.030174939058162802,0.0037303581782277364,0.026473304070195713,0.03849632587654989,0.03086826083864964,0.03086826083864964,0.003816069654529222,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013824000023305416,0.021023999899625778,0.016304000187665223,0.015584000386297703,0.0023236165806545476,0.031810621525966996,0.04156949936878106,0.0346749350032807,0.0346749350032807,0.003130209181143985,0.035677378270900374,0.04662250161636451,0.038889864871740294,0.038889864871740294,0.0035107033960828727,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.02175999991595745,0.01569600012153387,0.015008000191301107,0.0020882666168036863,0.038211712107062853,0.07212229256520057,0.04364509673334097,0.04364509673334097,0.009671953074025085,0.0476442859917874,0.08992570455184198,0.05441890342616107,0.05441890342616107,0.01205947791784038,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014368000440299511,0.033824000507593155,0.017628800217062236,0.014864000026136637,0.005791495403857738,0.05214261250030033,0.06266261508706669,0.05677791295527661,0.05677791295527661,0.0030298223079651514,0.08648138506878382,0.10392938682791132,0.09416928531647478,0.09416928531647478,0.005025126612204489,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.031199999153614044,0.017900799959897996,0.015343999955803156,0.004974475661316249,0.06411958891421111,0.07405276123263956,0.06793047918211377,0.06793047918211377,0.0033918658556700006,0.14058441263169497,0.16236323591492058,0.1489399211274489,0.1489399211274489,0.00743678300375353,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.03855999931693077,0.018495999928563833,0.015647999942302704,0.006918530056761627,0.09872985549401277,0.10683454583043753,0.10185316839884552,0.10185316839884552,0.0020802254037042074,0.2743261390166379,0.2968454508984596,0.2830044295482752,0.2830044295482752,0.005780016596065092,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.030400000512599945,0.01977920001372695,0.01726400014013052,0.005350883464206169,0.13918872472233365,0.16279522855335207,0.1501912864839173,0.1501912864839173,0.008992185529011513,0.11892328861766266,0.13909276049083735,0.1283239123428725,0.1283239123428725,0.007682951884956939,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01727999933063984,0.02223999984562397,0.01819519978016615,0.017680000513792038,0.0014397298270620873,0.3728044181625443,0.38295504353701676,0.3761264483787333,0.3761264483787333,0.0033660272332490977,0.07967557017401883,0.08184495665371805,0.08038555277807365,0.08038555277807365,0.0007193856241088455,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.034591998904943466,0.01941439984366298,0.01756799966096878,0.005541380584373771,0.05951999872922897,0.08268799632787704,0.06715519949793816,0.06715519949793816,0.006657802116288364,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03283200040459633,0.019180799927562477,0.016543999314308167,0.0052187911233635975,0.07199999690055847,0.08675199747085571,0.07749439924955369,0.07749439924955369,0.004849874741156772,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01724799908697605,0.02284800074994564,0.01928640007972717,0.018400000408291817,0.0020641390157565697,0.09676799923181534,0.12310399860143663,0.10618879944086074,0.10618879944086074,0.008982738207839057,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.03315199911594391,0.027436799928545953,0.027328000403940678,0.002679154874546328,0.14716799557209015,0.16412800550460815,0.15470399856567385,0.15470399856567385,0.005423454433987419,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01833599992096424,0.04806400090456009,0.026252799853682517,0.022672000341117382,0.00878438440429033,0.1844799965620041,0.2072959989309311,0.19359359890222552,0.19359359890222552,0.007663539820967659,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024480000138282776,0.04979199916124344,0.029081599973142146,0.025679999962449074,0.007227422044689197,0.34147199988365173,0.37968000769615173,0.3583200007677078,0.3583200007677078,0.011804422100726231,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.023744000121951103,0.017753600236028434,0.016368000768125057,0.003061040281616705,0.0,0.0,0.0,0.0,0.0,0.05686400085687637,0.1090880036354065,0.06629760004580021,0.06629760004580021,0.014803727026812366,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.021088000386953354,0.01647359998896718,0.015840000472962856,0.0017964402433206,0.0,0.0,0.0,0.0,0.0,0.08540800213813782,0.09961599856615067,0.09146559983491898,0.09146559983491898,0.00447423706371901,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.04156799986958504,0.018512000143527985,0.015584000386297703,0.007849701681877904,0.0,0.0,0.0,0.0,0.0,0.17948800325393677,0.194815993309021,0.1866239994764328,0.1866239994764328,0.004156328241374654,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.029632000252604485,0.019449600111693145,0.016736000776290894,0.005126811425136928,0.0,0.0,0.0,0.0,0.0,0.27529600262641907,0.2898879945278168,0.2825664013624191,0.2825664013624191,0.004542801609674093,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.031328000128269196,0.019759999960660933,0.016367999836802483,0.006266303740589241,0.0,0.0,0.0,0.0,0.0,0.39190399646759033,0.4079039990901947,0.3987520009279252,0.3987520009279252,0.0037367727509362725,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01539199985563755,0.040031999349594116,0.0212032001465559,0.01649599988013506,0.008068547381446682,0.0,0.0,0.0,0.0,0.0,0.06588800251483917,0.07897599786520004,0.0703904002904892,0.0703904002904892,0.004214826869769361,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022495999932289124,0.016006399970501663,0.01515199989080429,0.002265581884342885,0.0,0.0,0.0,0.0,0.0,0.1231679990887642,0.13526399433612823,0.1288223996758461,0.1288223996758461,0.004328976434897094,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.023231999948620796,0.01705600004643202,0.016048000194132328,0.002810904312746391,0.0,0.0,0.0,0.0,0.0,0.3158079981803894,0.33129599690437317,0.32348800003528594,0.32348800003528594,0.00417599076136664,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.021888000890612602,0.01708160014823079,0.01609600055962801,0.0025289234621475062,0.0,0.0,0.0,0.0,0.0,0.5103679895401001,0.5200319886207581,0.5132320046424866,0.5132320046424866,0.0035759433157749533,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.032416000962257385,0.019353600032627583,0.016176000237464905,0.0061458798960684425,0.0,0.0,0.0,0.0,0.0,0.7400320172309875,0.7516480088233948,0.7449311971664427,0.7449311971664427,0.004573461776168607,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.02006400004029274,0.01573119992390275,0.014992000069469213,0.001654446341262991,0.0,0.0,0.0,0.0,0.0,0.07097599655389786,0.09932799637317657,0.07828159928321837,0.07828159928321837,0.009464602895232642,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.02319999970495701,0.016883199848234654,0.015775999519973993,0.0026693906156048403,0.0,0.0,0.0,0.0,0.0,0.14176000654697418,0.1598079949617386,0.15063679963350293,0.15063679963350293,0.005741662969679433,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.03203200176358223,0.022224000189453363,0.02247999981045723,0.005996176183124981,0.0,0.0,0.0,0.0,0.0,0.39180800318717957,0.41046398878097534,0.4003200054168701,0.4003200054168701,0.005552433764307601,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.028192000463604927,0.017152000125497578,0.015536000020802021,0.003847486121602065,0.0,0.0,0.0,0.0,0.0,0.6239359974861145,0.6367359757423401,0.6285343945026398,0.6285343945026398,0.004006084286606002,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.026688000187277794,0.01718079997226596,0.015488000120967627,0.0037132536813398722,0.0,0.0,0.0,0.0,0.0,0.9141119718551636,0.9721279740333557,0.927455997467041,0.927455997467041,0.02148767779232235,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.047520000487565994,0.028710400220006704,0.0266720000654459,0.010774688401598903,0.06094816381288764,0.06816969726785131,0.0644060660218149,0.0644060660218149,0.002203106589073183,0.087051838094461,0.09736630405680231,0.09199073574791852,0.09199073574791852,0.0031466818046499644,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.02489599958062172,0.017427200078964235,0.01595200039446354,0.003071906489592143,0.19733789497223914,0.2016295616652644,0.19860388994013833,0.19860388994013833,0.001512389485439305,0.44861409134062713,0.45837046456077934,0.45149211526120153,0.45149211526120153,0.0034381598874302305,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.03046399913728237,0.020643199887126686,0.016944000497460365,0.006320148675595729,0.2157924314537054,0.22215709640166995,0.21769400765743155,0.21769400765743155,0.0019679921844645526,0.4905115822753901,0.5049789194903589,0.4948340005651485,0.4948340005651485,0.0044733865493072344,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.017152000218629837,0.02425600029528141,0.01865920014679432,0.018112000077962875,0.0019454553198986453,0.7430866512973927,0.7578834738720781,0.7455351005236347,0.7455351005236347,0.00416692487556653,0.7369773831645824,0.7516525540362471,0.7394057025273603,0.7394057025273603,0.004132666607961175,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015263999812304974,0.02470399998128414,0.018009600043296815,0.01657600048929453,0.0031544532774534346,0.25251173919752334,0.25566890792461106,0.2536729054481981,0.2536729054481981,0.0010279562245342983,1.0425282722084215,1.0555630628624373,1.0473223013846877,1.0473223013846877,0.004244053880724073,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.027807999402284622,0.01912960009649396,0.01643200032413006,0.004688164695934997,0.1251379565220268,0.14341821167748953,0.12742497577885914,0.12742497577885914,0.005353812167343766,1.1355340167064276,1.3014137556763312,1.1562870179154463,1.1562870179154463,0.048581869194943325,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.07446400076150894,0.08246400207281113,0.07736000046133995,0.07713599875569344,0.002301031358835964,11.935359954833984,12.102368354797363,11.96896333694458,11.96896333694458,0.05162549713370252,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07664000242948532,0.08118399977684021,0.07796800062060356,0.077504001557827,0.0013681081693640953,19.84774398803711,20.15795135498047,19.915702438354494,19.915702438354494,0.0912299324030902,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0753600001335144,0.08207999914884567,0.07715519964694977,0.07595199719071388,0.0024238358447475007,27.781503677368164,27.9836483001709,27.844886589050287,27.844886589050287,0.061307010154819624,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04339199885725975,0.050016000866889954,0.04504639990627766,0.04391999915242195,0.002339637813151782,5.1544318199157715,5.269279956817627,5.16938238143921,5.16938238143921,0.03363105774712194,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04291199892759323,0.058848001062870026,0.045657599717378615,0.04383999854326248,0.004522763233004815,9.256383895874023,9.27734375,9.261776161193849,9.261776161193849,0.0062232312957398745,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043296001851558685,0.04982399940490723,0.045123199746012685,0.04387199878692627,0.0023369398894319345,13.365216255187988,13.634464263916016,13.398719978332519,13.398719978332519,0.0787651922310212,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.0297279991209507,0.027692800015211107,0.02723200060427189,0.000851127720159845,2.387968063354492,2.4014720916748047,2.3920736074447637,2.3920736074447637,0.003552645719470337,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02691200003027916,0.029823999851942062,0.027689599990844728,0.027583999559283257,0.0007783369959809023,4.440767765045166,4.4521918296813965,4.444268751144409,4.444268751144409,0.004022325406840951,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.03888000175356865,0.030262400023639204,0.02801600005477667,0.0037877256209144718,6.500351905822754,6.600607872009277,6.524902391433716,6.524902391433716,0.03321667087212851,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07865600287914276,0.08668799698352814,0.08130879923701287,0.07993599772453308,0.0027016201268628523,35.717376708984375,35.843265533447266,35.771837615966795,35.771837615966795,0.04024815379149301,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04179200157523155,0.04864000156521797,0.0436256006360054,0.04267200082540512,0.002191476789190783,6.103871822357178,6.18287992477417,6.118390369415283,6.118390369415283,0.022924175047267112,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04163200035691261,0.05987200140953064,0.04625920057296753,0.04403200000524521,0.005544136325859629,10.207136154174805,11.073247909545898,10.326777648925782,10.326777648925782,0.25139335734700863,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04137599840760231,0.05027199909090996,0.04355199970304966,0.042399998754262924,0.0026572716942034787,14.303423881530762,14.343520164489746,14.311049747467042,14.311049747467042,0.01137511287661936,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.026016000658273697,0.03718400001525879,0.03212799951434135,0.03270399942994118,0.003953184738275612,2.6534719467163086,2.6875839233398438,2.661257576942444,2.661257576942444,0.009372641904054613,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025887999683618546,0.03347200155258179,0.027168000116944313,0.02649599965661764,0.0021492000999850562,4.70630407333374,4.7400641441345215,4.714492845535279,4.714492845535279,0.009349196197962609,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025631999596953392,0.030719999223947525,0.02736639976501465,0.02711999975144863,0.0013921874495504173,6.759712219238281,6.831999778747559,6.774675178527832,6.774675178527832,0.021507023842020512,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.024831999093294144,0.01991359982639551,0.018655999563634396,0.0023971416189370203,1.3446400165557861,1.3609600067138672,1.3508928060531615,1.3508928060531615,0.006186954712541735,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0180479995906353,0.022752000018954277,0.019507200084626676,0.01896000001579523,0.0015079573553847933,2.5208001136779785,2.5887041091918945,2.534086418151855,2.534086418151855,0.01891953046799999,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.018144000321626663,0.03222399950027466,0.02052800003439188,0.018864000216126442,0.004114364828004189,3.6922879219055176,3.760576009750366,3.706630396842957,3.706630396842957,0.019888657921309623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04169600084424019,0.04822399839758873,0.0439775999635458,0.04334400035440922,0.002101844179466219,18.41494369506836,18.502975463867188,18.445004844665526,18.445004844665526,0.026939913540867878,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023615999147295952,0.03267199918627739,0.026281599886715412,0.0248800003901124,0.003167969318232417,3.1188158988952637,3.1837120056152344,3.133536005020142,3.133536005020142,0.018365009771617643,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02380800060927868,0.03283200040459633,0.025670399703085423,0.024639999493956566,0.0026095917641781453,5.1729278564453125,5.243135929107666,5.192828798294067,5.192828798294067,0.01984737475315108,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024032000452280045,0.03167999908328056,0.02665280010551214,0.02550400048494339,0.002510131946267113,7.223167896270752,7.257152080535889,7.231532812118529,7.231532812118529,0.009418898846297825,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01696000061929226,0.023679999634623528,0.01866880003362894,0.017455999739468098,0.002277860345236006,1.478559970855713,1.496127963066101,1.48257919549942,1.48257919549942,0.005278358037488471,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017023999243974686,0.030688000842928886,0.02055360022932291,0.018240000121295452,0.004065815389665311,2.651711940765381,2.6691839694976807,2.657548785209656,2.657548785209656,0.005897264516465086,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016767999157309532,0.024351999163627625,0.01840319987386465,0.017167999409139156,0.0025234367788448957,3.823551893234253,3.8631999492645264,3.8335999727249135,3.8335999727249135,0.011858923917429712,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013952000066637993,0.03622400015592575,0.018214400112628936,0.014719999860972166,0.007277897466521675,0.7039039731025696,0.7163199782371521,0.7077983975410462,0.7077983975410462,0.004048424224033295,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03097599931061268,0.020870400313287973,0.02054399996995926,0.005422197456220526,1.2929600477218628,1.313088059425354,1.3006752014160157,1.3006752014160157,0.00790622446009414,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03232000023126602,0.019132800027728082,0.01673599984496832,0.0050857949387988245,1.8775999546051025,1.9125759601593018,1.8867840051651,1.8867840051651,0.01196043526228758,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023584000766277313,0.03126399964094162,0.02633600030094385,0.025200000032782555,0.0025710957466677864,9.283391952514648,9.48249626159668,9.331705665588379,9.331705665588379,0.06366970422148335,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07552000135183334,0.0851840004324913,0.07839040011167527,0.07703999802470207,0.0030805904904383677,43.62025451660156,43.89299011230469,43.68390693664551,43.68390693664551,0.08458163192147439,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,40960.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,40960.0,49152.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks48k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07718399912118912,0.0854400023818016,0.07895359992980958,0.07791999727487564,0.0024425064759109735,59.421791076660156,59.73017501831055,59.5021312713623,59.5021312713623,0.10367381308510447,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,57344.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,57344.0,65536.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks64k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07875200361013412,0.09055999666452408,0.08208959847688675,0.08087999746203423,0.0033614630055883482,75.2852783203125,75.55481719970703,75.38758392333985,75.38758392333985,0.08870109119325344,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,73728.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,73728.0,81920.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks80k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07788799703121185,0.09888000041246414,0.0819871999323368,0.08008000254631042,0.005835618489111142,91.13442993164062,91.45164489746094,91.24225158691405,91.24225158691405,0.1009018902177725,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,90112.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,90112.0,98304.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks96k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07596799731254578,0.08790399879217148,0.07967040091753005,0.07808000221848488,0.0036209252740976605,106.89055633544922,107.34063720703125,106.9560287475586,106.9560287475586,0.13032874360676439,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,106496.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,106496.0,114688.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks112k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07648000121116638,0.08534400165081024,0.07928640022873878,0.07787200063467026,0.0028305104107121735,122.71724700927734,122.9840316772461,122.8334243774414,122.8334243774414,0.09625274868603513,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,122880.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,122880.0,131072.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07664000242948532,0.08774399757385254,0.08081279993057251,0.07972799986600876,0.003487270970977775,130.6565399169922,131.09359741210938,130.79229431152345,130.79229431152345,0.13138911961272914,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,131072.0,139264.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks136k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01532800029963255,0.040063999593257904,0.019459199998527764,0.01601599995046854,0.007496165774486326,9.443936347961426,9.649087905883789,9.489968109130858,9.489968109130858,0.05961646816296781,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,512,130560.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,130560.0,131072.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02691200003027916,0.040832001715898514,0.02942080032080412,0.027888000011444092,0.004040048279091587,16.753759384155273,16.90662384033203,16.78766403198242,16.78766403198242,0.04236470233451937,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,2048,65536.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,65536.0,67584.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks66k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04416000097990036,0.04867200180888176,0.04565120078623295,0.04468800127506256,0.0018079971851529544,50.29715347290039,50.50300979614258,50.35054740905761,50.35054740905761,0.0724480602714254,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,4096,98304.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,98304.0,102400.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks100k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.060447998344898224,0.06684800237417221,0.06276160031557083,0.061824001371860504,0.0024045636532566625,96.07142639160156,96.53209686279297,96.1898666381836,96.1898666381836,0.1365794879996899,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,6144,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[6144],6144,6144,6144,6144.0,True,0.0,0.0,0.0,True,131072.0,137216.0,6144,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q6ks134k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04217600077390671,0.05100800096988678,0.04406719990074635,0.042847998440265656,0.002656672983576728,22.5166072845459,22.986656188964844,22.63068161010742,22.63068161010742,0.13840494695459715,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,40960.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,40960.0,49152.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks48k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.047200001776218414,0.043644800409674646,0.04262400045990944,0.001927741871473188,30.725727081298828,30.787456512451172,30.74285774230957,30.74285774230957,0.019110324860515015,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,57344.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,57344.0,65536.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks64k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04185599833726883,0.05049600079655647,0.043852799385786054,0.04289599880576134,0.0024573088797598033,38.930206298828125,38.98448181152344,38.94538269042969,38.94538269042969,0.017734743323340796,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,73728.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,73728.0,81920.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks80k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04227200150489807,0.046911999583244324,0.04412479996681214,0.04327999986708164,0.001811858548765738,47.13151931762695,47.245887756347656,47.148198699951166,47.148198699951166,0.033297799765613076,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,90112.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,90112.0,98304.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks96k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.06195199862122536,0.04529919996857643,0.043136000633239746,0.005751677082163394,55.338497161865234,55.64672088623047,55.42207336425782,55.42207336425782,0.11010194068839292,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,106496.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,106496.0,114688.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks112k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.054687999188899994,0.04418559968471527,0.042767999693751335,0.0036820249064621804,63.549087524414055,63.80697631835938,63.62084197998047,63.62084197998047,0.08872184973119365,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,122880.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,122880.0,131072.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0461760014295578,0.06566400080919266,0.05008000023663044,0.04843199998140335,0.0055350567765421,67.64147186279297,67.91651153564453,67.70619888305666,67.70619888305666,0.0993520482760623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,131072.0,139264.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks136k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023711999878287315,0.03859199956059456,0.02689919974654913,0.0244159996509552,0.004626699769228834,4.775519847869873,4.829855918884277,4.790303993225097,4.790303993225097,0.01597276061319242,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,512,130560.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,130560.0,131072.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.0453759990632534,0.027676799893379213,0.025200000032782555,0.006154880946840619,9.579392433166504,9.63871955871582,9.59802885055542,9.59802885055542,0.016811667352984866,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,2048,65536.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,65536.0,67584.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks66k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02630399912595749,0.03593600168824196,0.02780479993671179,0.027040000073611736,0.0027569987978884785,25.21686363220215,25.383039474487305,25.250255966186526,25.250255966186526,0.04761963064809279,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,4096,98304.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,98304.0,102400.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks100k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.034015998244285583,0.041728001087903976,0.03665280006825924,0.0352960005402565,0.002710617869728372,48.101280212402344,48.49884796142578,48.17219200134278,48.17219200134278,0.11195495368044632,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,6144,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[6144],6144,6144,6144,6144.0,True,0.0,0.0,0.0,True,131072.0,137216.0,6144,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q6ks134k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024191999807953835,0.039583999663591385,0.027286400087177753,0.025200000032782555,0.004620118437533604,11.32140827178955,11.496224403381348,11.354345512390138,11.354345512390138,0.050117766215439966,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,40960.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,40960.0,49152.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks48k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024032000452280045,0.03232000023126602,0.026406400091946124,0.02518399991095066,0.002792696117638337,15.42249584197998,15.639776229858397,15.451993656158448,15.451993656158448,0.06304729308178388,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,57344.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,57344.0,65536.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks64k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.03488000109791756,0.026147199980914592,0.02459200005978346,0.003303059079404486,19.5251522064209,19.70569610595703,19.54985942840576,19.54985942840576,0.052248914965338324,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,73728.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,73728.0,81920.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks80k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024159999564290047,0.03046399913728237,0.0255103999748826,0.024656000547111034,0.00195717586616911,23.630016326904297,23.927040100097656,23.672822570800783,23.672822570800783,0.08715594728873713,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,90112.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,90112.0,98304.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks96k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02380800060927868,0.03574400022625923,0.02678720001131296,0.024720000103116035,0.003678231234048846,27.736671447753906,27.79840087890625,27.747158622741697,27.747158622741697,0.018952179343788657,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,106496.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,106496.0,114688.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks112k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.042080000042915344,0.02844479978084564,0.02527999971061945,0.006562862750709003,31.83427238464356,32.08835220336914,31.86994876861572,31.86994876861572,0.07444245241186317,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,122880.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,122880.0,131072.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02377600036561489,0.03139200061559677,0.026396799832582474,0.025071999989449978,0.0025962810796740263,33.88313674926758,33.96985626220703,33.89708137512208,33.89708137512208,0.02518493598134421,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,131072.0,139264.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks136k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014944000169634819,0.022624000906944275,0.017174400109797715,0.015536000020802021,0.0028331860349340154,3.179744005203247,3.192960023880005,3.182867193222046,3.182867193222046,0.0035239767128429386,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,512,130560.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,130560.0,131072.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02985600009560585,0.017318400088697672,0.015568000264465809,0.004479309643844885,4.811935901641846,4.826848030090332,4.8173023700714115,4.8173023700714115,0.00594674080057053,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,2048,65536.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,65536.0,67584.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks66k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017184000462293625,0.03916800022125244,0.021356799826025962,0.01935999933630228,0.0062801287963799276,14.374591827392578,14.635007858276367,14.41227512359619,14.41227512359619,0.07569270440104603,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,4096,98304.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,98304.0,102400.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks100k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.020479999482631683,0.033215999603271484,0.023340800032019614,0.02092800009995699,0.004304974035052475,24.088096618652344,24.18492889404297,24.11243553161621,24.11243553161621,0.02797845886790172,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,6144,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[6144],6144,6144,6144,6144.0,True,0.0,0.0,0.0,True,131072.0,137216.0,6144,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q6ks134k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/linear_op.csv

@@ -0,0 +1,37 @@
+time_stats.emb.min,time_stats.emb.max,time_stats.emb.mean,time_stats.emb.median,time_stats.emb.std,time_stats.input_layernorm.min,time_stats.input_layernorm.max,time_stats.input_layernorm.mean,time_stats.input_layernorm.median,time_stats.input_layernorm.std,time_stats.attn_pre_proj.min,time_stats.attn_pre_proj.max,time_stats.attn_pre_proj.mean,time_stats.attn_pre_proj.median,time_stats.attn_pre_proj.std,time_stats.attn_rope.min,time_stats.attn_rope.max,time_stats.attn_rope.mean,time_stats.attn_rope.median,time_stats.attn_rope.std,time_stats.attn_post_proj.min,time_stats.attn_post_proj.max,time_stats.attn_post_proj.mean,time_stats.attn_post_proj.median,time_stats.attn_post_proj.std,time_stats.post_attention_layernorm.min,time_stats.post_attention_layernorm.max,time_stats.post_attention_layernorm.mean,time_stats.post_attention_layernorm.median,time_stats.post_attention_layernorm.std,n_head,n_kv_head,n_embd,n_expanded_embd,vocab_size,use_gated_mlp,use_qk_norm,attn_output_gate,num_tokens,num_tensor_parallel_workers,padded_n_embd,padded_n_expanded_embd,model_arch,is_step2_mini,share_expert_dim,share_q_dim,measurement_type,profiling_precision,quant_signature
+0.029184000566601753,0.06780800223350525,0.03157280012965202,0.030736000277101994,0.005874173435341216,0.033215999603271484,0.04825599864125252,0.03443359974771738,0.0337119996547699,0.0031825678429048183,1.438431978225708,1.505568027496338,1.446228802204132,1.4429279565811157,0.014128607642643025,0.538752019405365,0.5440319776535034,0.5416463971138,0.5420799851417542,0.0016099306164432847,1.0073280334472656,1.0163840055465698,1.0098415970802308,1.0081279873847961,0.0032980457197329728,0.04022400081157684,0.041728001087903976,0.04091359991580248,0.04081599973142147,0.0004728505416767937,32,4,2048,768,151936,True,True,False,8192,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.01360000018030405,0.06784000247716904,0.017755200061947106,0.0179840000346303,0.00837681421401443,0.01836800016462803,0.03651199862360954,0.019606399815529585,0.018719999119639397,0.0038821958848767424,0.7512000203132629,0.8208960294723511,0.7566704005002975,0.75382399559021,0.014793092586577971,0.28995200991630554,0.29337599873542786,0.29135999977588656,0.29150401055812836,0.000998381071258815,0.5149760246276855,0.5169600248336792,0.5160208016633987,0.5158880054950714,0.0005038652783291977,0.02051199972629547,0.021503999829292297,0.021067200042307378,0.021104000508785248,0.00023542855306902367,32,4,2048,768,151936,True,True,False,4096,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.0080960001796484,0.04281599819660187,0.011092799971811474,0.011039999779313803,0.005489135752949302,0.012223999947309494,0.026335999369621277,0.013187199970707298,0.01247999956831336,0.0030236510562153375,0.3928639888763428,0.45372799038887024,0.3976895987987518,0.39528000354766846,0.012905176716136006,0.1547199934720993,0.15884800255298615,0.15712319910526276,0.1573439985513687,0.001165121945135037,0.26633599400520325,0.268095999956131,0.26719200164079665,0.2671840041875839,0.0004242740199415328,0.013024000450968742,0.013887999579310417,0.013489600038155913,0.013520000036805868,0.0002382817554057738,32,4,2048,768,151936,True,True,False,2048,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.00825599953532219,0.04755200073122978,0.02398160002194345,0.01961600035429001,0.00874683840888523,0.018880000337958336,0.03868800029158592,0.021590400114655496,0.0208320003002882,0.004057015751746236,0.24316799640655518,0.2710399925708771,0.2521967992186546,0.25065599381923676,0.007998418528894075,0.09644799679517746,0.19120000302791595,0.10407840013504029,0.09963199868798256,0.020043722414992166,0.13600000739097595,0.18892799317836761,0.15760480016469955,0.15760000050067902,0.0112223677907762,0.009664000011980534,0.010015999898314476,0.009836799977347255,0.009824000298976898,9.016971952948177e-05,32,4,2048,768,151936,True,True,False,1024,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017376000061631203,0.044704001396894455,0.026318399980664254,0.027312000282108784,0.007565344034680866,0.01836800016462803,0.03014400042593479,0.021276800055056812,0.020655999891459942,0.002632977855097951,0.1438719928264618,0.17132799327373505,0.152497598528862,0.15012799948453903,0.007947150319625347,0.1430719941854477,0.19305600225925446,0.16630879789590836,0.1685439944267273,0.014628024163894684,0.08899199962615967,0.10467199981212616,0.0943599995225668,0.09374399855732918,0.003472669494074113,0.007615999784320593,0.007935999892652035,0.007769599952735007,0.0077760000713169575,7.680004540222077e-05,32,4,2048,768,151936,True,True,False,512,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.016992000862956047,0.0544000007212162,0.02573199989274144,0.026016000658273697,0.00803323401720434,0.018432000651955605,0.02348800003528595,0.020648000109940768,0.02062400057911873,0.0013939985047930988,0.10220800340175629,0.1361600011587143,0.11850560046732425,0.11684799939393997,0.010637754898360304,0.16710400581359863,0.21110400557518005,0.19078560024499894,0.19409599900245667,0.013714352646558832,0.06265600025653839,0.0740479975938797,0.06842879951000214,0.0690080001950264,0.0031292240446560557,0.00979200005531311,0.033440001308918,0.01864320016466081,0.017280000261962414,0.006957998188876383,32,4,2048,768,151936,True,True,False,256,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017152000218629837,0.04396799951791763,0.025907999789342284,0.02598400041460991,0.007714554737915267,0.018400000408291817,0.03667199984192848,0.024132800102233887,0.02112000063061714,0.006120348733354326,0.10678400099277496,0.1363839954137802,0.1193264003843069,0.11583999916911125,0.009838647443214228,0.17017599940299988,0.22748799622058868,0.18853759989142418,0.18433599919080734,0.015728078443174653,0.04569600149989128,0.06652799993753433,0.05192639995366335,0.05151999928057194,0.004516605694976449,0.021856000646948814,0.026335999369621277,0.02384479995816946,0.023599999956786633,0.0011301153013314744,32,4,2048,768,151936,True,True,False,128,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017343999817967415,1.0683200359344482,0.05748240072280168,0.029504000209271908,0.16366824814254827,0.018688000738620758,0.3317759931087494,0.03685439983382821,0.021151999942958355,0.06767422466731164,0.10255999863147736,0.9434880018234253,0.16412640027701855,0.11956800147891045,0.17964606281728834,0.1714559942483902,2.1306240558624268,0.3011296011507511,0.1926399990916252,0.42310127734378766,0.03574400022625923,0.6859520077705383,0.08389280084520578,0.04279999993741512,0.14321647071615612,0.020479999482631683,0.1831360012292862,0.033024000097066165,0.023856000043451786,0.03470987082429836,32,4,2048,768,151936,True,True,False,64,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.016095999628305435,0.05142400041222572,0.026363200135529043,0.02676799986511469,0.008637725852473854,0.01849599927663803,0.03577600046992302,0.022193600237369538,0.020848000422120094,0.004615266181181815,0.10540799796581268,0.15014399588108063,0.12211520001292228,0.11896000057458878,0.012772013396624768,0.17315199971199036,0.21478399634361267,0.1881632000207901,0.1873439997434616,0.011761164657572015,0.03481600061058998,0.058079998940229416,0.042200000025331974,0.03969600051641464,0.006461281521769989,0.02143999934196472,0.038816001266241074,0.02466559996828437,0.023856000043451786,0.0035418033748569927,32,4,2048,768,151936,True,True,False,32,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.015904000028967857,0.03753599897027016,0.023127999808639287,0.02527999971061945,0.0054230027890305385,0.018400000408291817,0.03001599945127964,0.020488000102341176,0.020096000283956528,0.002585688394137697,0.10355199873447418,0.1438400000333786,0.11536479964852334,0.11124800145626068,0.011136028294919255,0.16502399742603302,0.2072959989309311,0.18646399974822997,0.19075199961662292,0.013296437761247597,0.03142400085926056,0.05215999856591225,0.03888959977775812,0.03750399872660637,0.0057399302067536314,0.020128000527620316,0.04064000025391579,0.023937600292265417,0.023648000322282314,0.004144197400898248,32,4,2048,768,151936,True,True,False,16,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.015200000256299973,0.037151999771595,0.02144480012357235,0.02195199951529503,0.005576364091933697,0.017952000722289085,0.0226879995316267,0.019934400077909233,0.019952000118792057,0.0011812499470458758,0.10063999891281128,0.13468800485134125,0.11595199964940547,0.1207519993185997,0.011974301621092394,0.16332800686359406,0.20748800039291382,0.18162400051951408,0.1767839938402176,0.01474120743720334,0.03222399950027466,0.043455999344587326,0.03829439990222454,0.03859200142323971,0.0031378131240041122,0.020959999412298203,0.03587200120091438,0.023795200139284135,0.023423999547958374,0.0031339489695198443,32,4,2048,768,151936,True,True,False,8,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.014944000169634819,0.04022400081157684,0.02162720002233982,0.022463999688625336,0.006017219317026815,0.018464000895619392,0.026528000831604004,0.021264000236988066,0.020896000787615776,0.001934095017586082,0.10156799852848053,0.1703999936580658,0.12565439902245998,0.1244799979031086,0.01641776722785409,0.1653759926557541,0.23865599930286407,0.1969360001385212,0.19223999977111816,0.02058903050274278,0.03254399821162224,0.06752000004053116,0.0443536002188921,0.041519999504089355,0.00984829071098989,0.020096000283956528,0.040031999349594116,0.026934400014579297,0.02478400059044361,0.00562071702648593,32,4,2048,768,151936,True,True,False,1,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.7531200051307678,0.8461440205574036,0.7618160009384155,0.7576479911804199,0.019464233617982506,0.2922559976577759,0.2985599935054779,0.2953856036067009,0.29576000571250916,0.001575901077689139,0.510047972202301,0.5140479803085327,0.5121696025133133,0.5123839974403381,0.0012263137313476844,,,,,,32,4,2048,768,151936,True,True,False,8192,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.39529600739479065,0.47279998660087585,0.40216960161924364,0.39825600385665894,0.0162749796240819,0.15625600516796112,0.16211199760437012,0.15959519892930984,0.15988799929618835,0.0011493418942396922,0.2635200023651123,0.2642880082130432,0.2639120012521744,0.26392000913619995,0.0001903593401384135,,,,,,32,4,2048,768,151936,True,True,False,4096,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.2531839907169342,0.32419198751449585,0.26924319565296173,0.26049599051475525,0.01657194262368116,0.10051199793815613,0.2375359982252121,0.12411200068891048,0.10311999917030334,0.039466165428540506,0.15881599485874176,0.19289599359035492,0.16896959990262986,0.1640480011701584,0.009162416086418127,,,,,,32,4,2048,768,151936,True,True,False,2048,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.15302400290966034,0.1780800074338913,0.16284480094909667,0.16113600134849548,0.00740355661356144,0.14521600306034088,0.20233599841594696,0.17807039842009545,0.1796799972653389,0.013908448560094403,0.0907519981265068,0.09750399738550186,0.09460479989647866,0.09478399902582169,0.0017965487667361475,,,,,,32,4,2048,768,151936,True,True,False,1024,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11740799993276596,0.17958399653434753,0.13262080028653145,0.12878400087356567,0.014146060532423056,0.17468799650669098,0.21161599457263947,0.1910431995987892,0.18966399878263474,0.01270903647700971,0.05926400050520897,0.07577600330114365,0.06530559975653887,0.06404799968004227,0.00479458462514625,,,,,,32,4,2048,768,151936,True,True,False,512,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11260800063610077,0.17209599912166595,0.13610880002379416,0.13809599727392197,0.017675922458993677,0.18729600310325623,0.26822400093078613,0.20815680101513861,0.2078079953789711,0.01700718593658771,0.04499199986457825,0.06537599861621857,0.0507551996037364,0.04787199944257736,0.005727123232692158,,,,,,32,4,2048,768,151936,True,True,False,256,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11257600039243698,0.23452800512313843,0.13866880126297473,0.13964799791574478,0.026590047697062115,0.1828799992799759,0.24751999974250793,0.20735519900918006,0.20670399814844131,0.01500438131586336,0.03654399886727333,0.05593600124120712,0.0408239996060729,0.03892800025641918,0.004864409450710354,,,,,,32,4,2048,768,151936,True,True,False,128,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11734399944543839,0.18726399540901184,0.13890240006148816,0.13308800011873245,0.019086167340006257,0.16991999745368958,0.2443840056657791,0.19185120090842248,0.19257599860429764,0.018017536159219673,0.03033600002527237,0.04956800118088722,0.038387199863791466,0.03750400058925152,0.00500897018702887,,,,,,32,4,2048,768,151936,True,True,False,64,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11420799791812897,0.18115200102329254,0.13938880078494548,0.1393439993262291,0.018141313962922536,0.1693439930677414,0.221343994140625,0.19187839925289155,0.19438399374485016,0.01430752121272052,0.03017600066959858,0.06019200012087822,0.03866560012102127,0.0364960003644228,0.0074199790031205266,,,,,,32,4,2048,768,151936,True,True,False,32,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11276800185441971,0.16223999857902527,0.1331360016018152,0.13305599987506866,0.01432493740801146,0.17187200486660004,0.23625600337982178,0.19287680014967917,0.1913280040025711,0.01754628831589704,0.029823999851942062,0.07574400305747986,0.03900959976017475,0.036927999928593636,0.009277465083962879,,,,,,32,4,2048,768,151936,True,True,False,16,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1130559965968132,0.19814400374889374,0.13379519879817964,0.12531199678778648,0.020074427302248836,0.16841599345207214,0.21139200031757355,0.1893615983426571,0.19092799723148346,0.012418720676762711,0.029503999277949333,0.07558400183916092,0.040144000016152856,0.03728000074625015,0.010223239196498205,,,,,,32,4,2048,768,151936,True,True,False,8,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.115167997777462,0.2640640139579773,0.13805920109152794,0.13118399679660797,0.03132849683515479,0.17103999853134155,0.2977280020713806,0.19899839907884598,0.1976960003376007,0.028928046763067948,0.0297279991209507,0.05990400165319443,0.03834720011800528,0.0363520011305809,0.0070885330713495905,,,,,,32,4,2048,768,151936,True,True,False,1,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.3940800130367279,0.46483200788497925,0.39923040121793746,0.3957759886980057,0.015108903906233569,0.16022400557994843,0.1634880006313324,0.16203359961509706,0.16228799521923065,0.0010220720912414007,0.26073598861694336,0.2627840042114258,0.26138080209493636,0.2613760083913803,0.00041640363494172775,,,,,,32,4,2048,768,151936,True,True,False,8192,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.2536959946155548,0.27161601185798645,0.25960480123758317,0.2577280104160309,0.005142017404245015,0.09849599748849869,0.10281600058078766,0.10057279989123344,0.10063999891281128,0.0008836232237268523,0.13913600146770477,0.17606399953365326,0.15875840038061143,0.15988799929618835,0.010058952112389172,,,,,,32,4,2048,768,151936,True,True,False,4096,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.15142400562763214,0.1780479997396469,0.16139679849147798,0.1602879986166954,0.0070656055731385115,0.14601600170135498,0.23343999683856964,0.17913119941949845,0.179967999458313,0.023065274309176566,0.09388799965381622,0.12108799815177917,0.10317599996924401,0.10073599964380264,0.007364345618470178,,,,,,32,4,2048,768,151936,True,True,False,2048,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12198399752378464,0.18406400084495544,0.13770400024950505,0.13232000172138214,0.01623164885687898,0.1773120015859604,0.20688000321388245,0.1870912007987499,0.18535999953746796,0.00825628058448234,0.05766399949789047,0.06739199906587601,0.06187200043350458,0.061216000467538834,0.003004312467037863,,,,,,32,4,2048,768,151936,True,True,False,1024,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11737599968910217,0.18614399433135986,0.13109439946711063,0.12771200388669968,0.01372168725934724,0.17587199807167053,0.2699519991874695,0.19926720038056372,0.19075199961662292,0.02327478982490538,0.041728001087903976,0.06224000081419945,0.05000480003654957,0.049375999718904495,0.006185850944273961,,,,,,32,4,2048,768,151936,True,True,False,512,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1159679964184761,0.1610880047082901,0.12988320142030715,0.12494400516152382,0.011865375783184065,0.1767680048942566,0.27379199862480164,0.21035519987344742,0.21275199949741364,0.023206964017550125,0.037087999284267426,0.062144000083208084,0.04439679980278015,0.04283200018107891,0.006155226392511492,,,,,,32,4,2048,768,151936,True,True,False,256,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12015999853610992,0.158720001578331,0.13573280088603495,0.1343199983239174,0.010666830836014414,0.1737920045852661,0.22127999365329742,0.20261440128087999,0.20321600139141083,0.011153965849067301,0.03587200120091438,0.04944000020623207,0.038265600241720675,0.037328001111745834,0.0030431580113575636,,,,,,32,4,2048,768,151936,True,True,False,128,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1212799996137619,0.15139199793338776,0.13183839991688728,0.13014400005340576,0.00851207547022104,0.17257599532604218,0.2250880002975464,0.1872655987739563,0.18193599581718445,0.013638284975248818,0.02969600073993206,0.0525440014898777,0.03840640028938651,0.03444799967110157,0.007598511754254174,,,,,,32,4,2048,768,151936,True,True,False,64,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11270400136709213,0.1624639928340912,0.13508000001311302,0.13180799782276154,0.014243564451606775,0.17052799463272095,0.23715199530124664,0.1953311987221241,0.1966560035943985,0.01500670718978398,0.030239999294281006,0.05270399898290634,0.03807039987295866,0.03742399998009205,0.004906165602021684,,,,,,32,4,2048,768,151936,True,True,False,32,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12319999933242798,0.17919999361038208,0.14059039913117885,0.14156799763441086,0.016217662982107223,0.1701119989156723,0.21987199783325195,0.18903039917349815,0.1870879977941513,0.014117854507841239,0.030368000268936157,0.06406400352716446,0.04045119984075427,0.03710399940609932,0.00878942205983628,,,,,,32,4,2048,768,151936,True,True,False,16,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11296000331640244,0.16761599481105804,0.13843199908733367,0.13814399391412735,0.014925286935582404,0.1711679995059967,0.21561600267887115,0.1906527981162071,0.1876479983329773,0.011803992622137974,0.0306560005992651,0.09216000139713287,0.041129599791020155,0.036847999319434166,0.013739721468154687,,,,,,32,4,2048,768,151936,True,True,False,8,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11929599940776825,0.2008959949016571,0.14535359852015972,0.13971199840307236,0.024769473061576282,0.16803200542926788,0.25123199820518494,0.19573760256171227,0.19366399943828583,0.023196011081705884,0.03049599938094616,0.08089599758386612,0.04739360017701984,0.0453919991850853,0.01286389846889463,,,,,,32,4,2048,768,151936,True,True,False,1,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/manifest.json

@@ -0,0 +1,21 @@
+{
+  "schema": "frontier-profile-v6-code-longctx-v1",
+  "base": "runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/attention.csv",
+  "base_sha256": "ff32c38975e68565c85770d85b92bc310ab1d8e34fd02327bef7fb305a9ffae3",
+  "raw_inputs": {
+    "runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/raw/flashattn-code-longctx-tp1.json": "3937c623adcea3d379b82bb7ab63d3b293f917fa59fa6fd6a147581d6a252cb6",
+    "runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/raw/flashattn-code-longctx-tp2.json": "e48aa6e6d039bdb68759701c9976f78ad818d6778b6c40b14823a0a4e852c52d",
+    "runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/raw/flashattn-code-longctx-tp4.json": "759d33decc09d229d07818df8161ed41180cab575304f335ad4d5e7a917cbc93"
+  },
+  "appended_rows": 33,
+  "max_model_len": 147456,
+  "anchor_checks": [
+    "anchor q1ks8k/TP1: v4=1.1445ms new=1.1426ms rel_diff=0.2%",
+    "anchor q512s4k/TP1: v4=0.3391ms new=0.3367ms rel_diff=0.7%",
+    "anchor q1ks8k/TP2: v4=0.6126ms new=0.6065ms rel_diff=1.0%",
+    "anchor q512s4k/TP2: v4=0.2086ms new=0.2079ms rel_diff=0.3%",
+    "anchor q1ks8k/TP4: v4=0.3445ms new=0.3470ms rel_diff=0.7%",
+    "anchor q512s4k/TP4: v4=0.1536ms new=0.1510ms rel_diff=1.7%"
+  ],
+  "output_sha256": "fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550"
+}

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/moe.csv

@@ -0,0 +1,73 @@
+time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature,router_median_nonadditivity_ratio,projection_policy
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.33926400542259216,0.405023992061615,0.36780479848384856,0.36507199704647064,0.01690507644474779,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,1.1943039894104004,1.286784052848816,1.228384006023407,1.2273280024528503,0.02832547242381263,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,64,0.375,0.5,0.5,0.3984375,0.0,1.346291201783626,4.0,5.59375,0.661865234375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,1.679744005203247,1.766144037246704,1.7095808148384095,1.7015680074691772,0.02438921262998535,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,128,0.375,1.0,1.0,0.625,0.0,1.015504800579495,5.0,6.15516433212955,0.529052734375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,2.1576640605926514,2.2921600341796875,2.2097824096679686,2.188944101333618,0.045572321842012986,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,256,0.375,2.0,2.0,0.875,0.0,0.6343057228182637,2.5,6.64370748444639,0.35369873046875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,2.422368049621582,2.5130879878997803,2.4516672134399413,2.434159994125366,0.03278900287381846,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,512,0.375,4.0,4.0,0.984375,0.0,0.4921254921257382,2.25,6.817190042344769,0.272369384765625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,2.2217600345611572,2.289599895477295,2.2571327924728393,2.263375997543335,0.021660416089449488,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,1024,0.375,8.0,8.0,1.0,0.125,0.3486861500690843,1.875,6.908192310183997,0.197662353515625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,2.18668794631958,2.318079948425293,2.2218016147613526,2.211087942123413,0.035380897213135316,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,2048,0.375,16.0,16.0,1.0,0.4375,0.2525504668006971,1.875,6.953347743053017,0.1410369873046875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,2.2291839122772217,2.3929600715637207,2.2908096313476562,2.2804640531539917,0.04479348924786221,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,4096,0.375,32.0,32.0,1.0,0.65625,0.15765965680164504,1.5625,6.98229848728205,0.08779525756835938,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,2.0678720474243164,2.1297600269317627,2.0837119817733765,2.0779199600219727,0.017880044357986735,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8192,0.375,64.0,64.0,1.0,0.625,0.12169081635504074,1.3125,6.9892029662356325,0.06879425048828125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,2.916032075881958,3.0819520950317383,2.9805248022079467,2.9656319618225098,0.05482195799019572,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,16384,0.375,128.0,128.0,1.0,0.796875,0.07935434147688751,1.1796875,6.9954297964750305,0.044734954833984375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,4.421599864959717,4.535359859466553,4.486294317245483,4.497056007385254,0.036990243787549344,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,32768,0.375,256.0,256.0,1.0,0.8203125,0.060849326483103046,1.17578125,6.9973188375859685,0.033740997314453125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,7.302591800689697,7.402751922607422,7.354758310317993,7.3464319705963135,0.032142662400335566,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,65536,0.375,512.0,512.0,1.0,0.890625,0.0412323087266341,1.08984375,6.998772433185578,0.02334284782409668,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.35280001163482666,0.39692801237106323,0.37662720382213594,0.37196800112724304,0.013401318050665304,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.4692479968070984,0.5523840188980103,0.5134752035140991,0.5100640058517456,0.02291433464135784,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.34652799367904663,0.4119040071964264,0.3789471983909607,0.38550400733947754,0.02073945105803335,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.31462401151657104,0.7456960082054138,0.38617280423641204,0.34545600414276123,0.12230201266253077,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.32521599531173706,0.419871985912323,0.36325119733810424,0.34968000650405884,0.03161798848223672,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.289792001247406,0.4663360118865967,0.4091839998960495,0.41655999422073364,0.0446001986506615,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.3761279881000519,0.4416320025920868,0.40686399936676027,0.40540799498558044,0.02257778769899645,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.7172480225563049,0.8663039803504944,0.7723807990550995,0.7591840028762817,0.04164772379451242,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,1.0195519924163818,1.2216639518737793,1.1253888130187988,1.1453600525856018,0.06548005322243594,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.7490559816360474,1.9644800424575806,1.8529024004936219,1.814303994178772,0.08042565617327288,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,3.2479360103607178,3.385279893875122,3.296070408821106,3.2800960540771484,0.04525529026442046,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,6.344799995422363,6.517856121063232,6.464438438415527,6.478623867034912,0.05116674443145098,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.2648000121116638,0.325439989566803,0.28852800130844114,0.28390398621559143,0.01933778377635077,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.7347840070724487,0.862496018409729,0.7769344031810761,0.769216001033783,0.03290485328796285,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,64,0.375,0.5,0.5,0.421875,0.0,1.346291201783626,6.0,5.652114648336087,0.636962890625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.9198399782180786,0.9646080136299133,0.9412063956260681,0.9411839842796326,0.014939365085478117,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,128,0.375,1.0,1.0,0.5703125,0.0,1.118033988749895,5.0,6.008641773518898,0.580810546875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,1.2796800136566162,1.3484159708023071,1.3006976008415223,1.2929120063781738,0.020807998177176254,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,256,0.375,2.0,2.0,0.8828125,0.0,0.6959705453537527,3.0,6.60872850615583,0.38055419921875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,1.3630399703979492,1.4430400133132935,1.3909215927124023,1.3892319798469543,0.022335744492366926,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,512,0.375,4.0,4.0,0.984375,0.0,0.5201036555341637,3.0,6.798826509158851,0.28302001953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,1.27948796749115,1.3904000520706177,1.3176063895225525,1.309440016746521,0.038060887827312775,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,1024,0.375,8.0,8.0,1.0,0.25,0.3511282039725661,1.875,6.91002266305238,0.1970977783203125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,1.264415979385376,1.3145920038223267,1.2791999936103822,1.2753440141677856,0.014130605249568332,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,2048,0.375,16.0,16.0,1.0,0.375,0.24692938483248605,1.6875,6.955481130775285,0.13909912109375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,1.3081920146942139,1.347648024559021,1.3292255997657776,1.329967975616455,0.014558863679016933,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,4096,0.375,32.0,32.0,1.0,0.625,0.17143053326165383,1.5625,6.9786675275754035,0.09520339965820312,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,1.242751955986023,1.3112000226974487,1.2747935891151427,1.266207993030548,0.021093073517695057,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8192,0.375,64.0,64.0,1.0,0.78125,0.11000099875256815,1.296875,6.991308871213679,0.062183380126953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.7388160228729248,1.8077759742736816,1.772764801979065,1.772704005241394,0.021056644077284283,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,16384,0.375,128.0,128.0,1.0,0.78125,0.0864630150197678,1.1796875,6.994552526394139,0.048796653747558594,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,2.6563520431518555,2.7063679695129395,2.6785055875778196,2.6791679859161377,0.01639963463052944,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,32768,0.375,256.0,256.0,1.0,0.8671875,0.06127686514721937,1.16015625,6.997291583027146,0.03497934341430664,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,4.386879920959473,4.452256202697754,4.4108480453491214,4.406303882598877,0.019768161791937636,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,65536,0.375,512.0,512.0,1.0,0.884765625,0.041723768525324195,1.1171875,6.998746434318934,0.02298593521118164,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.24208000302314758,0.4028480052947998,0.3041536003351212,0.277103990316391,0.05660721484881584,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.2447039932012558,0.30502399802207947,0.26446720361709597,0.26265600323677063,0.016744548364435372,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.2337920069694519,0.2881599962711334,0.26074880361557007,0.264384001493454,0.016469850143940968,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.23369599878787994,0.28591999411582947,0.25465920120477675,0.25385600328445435,0.01593204652619194,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.2295999974012375,0.26556798815727234,0.24674240052700042,0.2497600018978119,0.010345732066199003,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.21721599996089935,0.29020801186561584,0.2394208014011383,0.2346400022506714,0.018747330357768585,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.2717759907245636,0.305184006690979,0.28813759982585907,0.28809599578380585,0.01183422600545854,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.3917759954929352,0.43772798776626587,0.41130879521369934,0.4131519943475723,0.012992473640805227,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.6176639795303345,0.7009919881820679,0.642767995595932,0.6330719888210297,0.024074084919938756,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.0820800065994263,1.1674879789352417,1.1034304022789,1.0977439880371094,0.0233067292981566,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.9809919595718384,2.0415360927581787,2.003715181350708,1.992751955986023,0.022066076434645737,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,3.790112018585205,3.8651199340820312,3.829139161109924,3.8230879306793213,0.025177464160110564,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.212351992726326,0.24383999407291412,0.22760000079870224,0.22723200172185898,0.01050568575837594,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.47494399547576904,0.5184000134468079,0.4920704007148743,0.49169600009918213,0.011991064701471855,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,64,0.375,0.5,0.5,0.3984375,0.0,1.3919410907075054,6.0,5.570159765557392,0.667236328125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.6360960006713867,0.7004479765892029,0.6608384013175964,0.6572319865226746,0.020416877242438597,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,128,0.375,1.0,1.0,0.625,0.0,1.0307764064044151,4.0,6.138251855282827,0.5382080078125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.780896008014679,0.8301439881324768,0.80346559882164,0.8030399978160858,0.016801230312128875,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,256,0.375,2.0,2.0,0.859375,0.0,0.6903350635742038,3.0,6.5943747091218174,0.38067626953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.8607040047645569,0.9195200204849243,0.8783008038997651,0.8751039803028107,0.01719059253115595,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,512,0.375,4.0,4.0,0.9765625,0.0,0.49410588440130926,2.75,6.814452474347134,0.271270751953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.833952009677887,0.894752025604248,0.8619967997074127,0.863215982913971,0.018716378851797198,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,1024,0.375,8.0,8.0,1.0,0.25,0.33693529145074724,2.125,6.9186075263155535,0.1867218017578125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.834879994392395,0.8871039748191833,0.8651552021503448,0.8638879954814911,0.015262430894400969,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,2048,0.375,16.0,16.0,1.0,0.375,0.25567294018677456,1.8125,6.952441049154937,0.14349365234375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.8518080115318298,0.9097599983215332,0.8810272097587586,0.8751040101051331,0.017005819633271906,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,4096,0.375,32.0,32.0,1.0,0.625,0.1747801353218523,1.53125,6.978069554482723,0.09820938110351562,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.8470079898834229,0.9010239839553833,0.8694015920162201,0.8716959953308105,0.016138881171190216,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8192,0.375,64.0,64.0,1.0,0.65625,0.1158122428154187,1.3125,6.9901908183358845,0.06445503234863281,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.1698240041732788,1.2311359643936157,1.1888479948043824,1.1890720129013062,0.017978797794492758,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,16384,0.375,128.0,128.0,1.0,0.78125,0.08347181893108634,1.1796875,6.994921772573154,0.046871185302734375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.7702720165252686,1.8097599744796753,1.7919103980064393,1.7956640124320984,0.012641295676021557,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,32768,0.375,256.0,256.0,1.0,0.78125,0.06866734477822484,1.20703125,6.996602562938728,0.03801727294921875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,2.968672037124634,3.0278079509735107,2.9899007797241213,2.9824799299240112,0.01816414122631667,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,65536,0.375,512.0,512.0,1.0,0.8984375,0.04399546833977376,1.126953125,6.998607314922362,0.024699926376342773,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.19574399292469025,0.2512960135936737,0.21939200013875962,0.2199999988079071,0.017532156418212565,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.20483200252056122,0.24006399512290955,0.22215040028095245,0.22433599829673767,0.00969639786132892,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.20559999346733093,0.24726399779319763,0.22126719802618028,0.22207999974489212,0.01328093478485499,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.20003199577331543,0.2301120012998581,0.21453119963407516,0.21598400175571442,0.010402239855151332,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.19551999866962433,0.22972799837589264,0.21238719969987868,0.21488000452518463,0.010706095835489097,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.19420799612998962,0.2903999984264374,0.2211231991648674,0.21476799994707108,0.025955008886196004,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.2290560007095337,0.289247989654541,0.2543327987194061,0.24939200282096863,0.01663236753503115,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.30831998586654663,0.36953601241111755,0.3324000000953674,0.3288639932870865,0.018617999572156707,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.462911993265152,0.5497919917106628,0.4893856018781662,0.4816960096359253,0.023348152887178286,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,0.7662079930305481,0.8717759847640991,0.788454395532608,0.7744799852371216,0.03174746482604812,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.363935947418213,1.4143040180206299,1.3812703967094422,1.3798720240592957,0.014770450075530007,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,2.5507519245147705,2.680704116821289,2.579859209060669,2.566223978996277,0.03673283558365955,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+3937c623adcea3d379b82bb7ab63d3b293f917fa59fa6fd6a147581d6a252cb6  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/raw/flashattn-code-longctx-tp1.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/raw/flashattn-code-longctx-tp1.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0851840004324913,
+        "mean_ms": 0.07839040011167527,
+        "median_ms": 0.07703999802470207,
+        "min_ms": 0.07552000135183334,
+        "std_ms": 0.0030805904904383677
+      },
+      "max_time": 0.043892990112304686,
+      "mean_time": 0.04368390693664551,
+      "memory_allocated_mb": 240.0849609375,
+      "memory_reserved_mb": 246.0,
+      "min_time": 0.043620254516601564,
+      "std_time": 8.458163192147439e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 187529.01410308387
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0854400023818016,
+        "mean_ms": 0.07895359992980958,
+        "median_ms": 0.07791999727487564,
+        "min_ms": 0.07718399912118912,
+        "std_ms": 0.0024425064759109735
+      },
+      "max_time": 0.059730175018310544,
+      "mean_time": 0.0595021312713623,
+      "memory_allocated_mb": 272.0888671875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.05942179107666016,
+      "std_time": 0.00010367381308510448,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 137675.74076699864
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.09055999666452408,
+        "mean_ms": 0.08208959847688675,
+        "median_ms": 0.08087999746203423,
+        "min_ms": 0.07875200361013412,
+        "std_ms": 0.0033614630055883482
+      },
+      "max_time": 0.07555481719970703,
+      "mean_time": 0.07538758392333984,
+      "memory_allocated_mb": 304.0927734375,
+      "memory_reserved_mb": 534.0,
+      "min_time": 0.0752852783203125,
+      "std_time": 8.870109119325343e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 108665.10867797918
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.09888000041246414,
+        "mean_ms": 0.0819871999323368,
+        "median_ms": 0.08008000254631042,
+        "min_ms": 0.07788799703121185,
+        "std_ms": 0.005835618489111142
+      },
+      "max_time": 0.09145164489746094,
+      "mean_time": 0.09124225158691406,
+      "memory_allocated_mb": 336.0966796875,
+      "memory_reserved_mb": 726.0,
+      "min_time": 0.09113442993164063,
+      "std_time": 0.0001009018902177725,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 89782.96630697016
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08790399879217148,
+        "mean_ms": 0.07967040091753005,
+        "median_ms": 0.07808000221848488,
+        "min_ms": 0.07596799731254578,
+        "std_ms": 0.0036209252740976605
+      },
+      "max_time": 0.10734063720703126,
+      "mean_time": 0.1069560287475586,
+      "memory_allocated_mb": 368.1005859375,
+      "memory_reserved_mb": 950.0,
+      "min_time": 0.10689055633544922,
+      "std_time": 0.00013032874360676438,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 76592.22295299545
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08534400165081024,
+        "mean_ms": 0.07928640022873878,
+        "median_ms": 0.07787200063467026,
+        "min_ms": 0.07648000121116638,
+        "std_ms": 0.0028305104107121735
+      },
+      "max_time": 0.1229840316772461,
+      "mean_time": 0.12283342437744141,
+      "memory_allocated_mb": 400.1044921875,
+      "memory_reserved_mb": 1206.0,
+      "min_time": 0.12271724700927734,
+      "std_time": 9.625274868603512e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 66691.94514049937
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08774399757385254,
+        "mean_ms": 0.08081279993057251,
+        "median_ms": 0.07972799986600876,
+        "min_ms": 0.07664000242948532,
+        "std_ms": 0.003487270970977775
+      },
+      "max_time": 0.13109359741210938,
+      "mean_time": 0.13079229431152345,
+      "memory_allocated_mb": 416.1064453125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.1306565399169922,
+      "std_time": 0.00013138911961272913,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 62633.65929255852
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.040063999593257904,
+        "mean_ms": 0.019459199998527764,
+        "median_ms": 0.01601599995046854,
+        "min_ms": 0.01532800029963255,
+        "std_ms": 0.007496165774486326
+      },
+      "max_time": 0.009649087905883789,
+      "mean_time": 0.009489968109130859,
+      "memory_allocated_mb": 265.0458984375,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.009443936347961425,
+      "std_time": 5.961646816296781e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 53951.70922728124
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.040832001715898514,
+        "mean_ms": 0.02942080032080412,
+        "median_ms": 0.027888000011444092,
+        "min_ms": 0.02691200003027916,
+        "std_ms": 0.004040048279091587
+      },
+      "max_time": 0.016906623840332032,
+      "mean_time": 0.01678766403198242,
+      "memory_allocated_mb": 168.04248046875,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.016753759384155274,
+      "std_time": 4.236470233451937e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 121994.34037387963
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04867200180888176,
+        "mean_ms": 0.04565120078623295,
+        "median_ms": 0.04468800127506256,
+        "min_ms": 0.04416000097990036,
+        "std_ms": 0.0018079971851529544
+      },
+      "max_time": 0.05050300979614258,
+      "mean_time": 0.05035054740905761,
+      "memory_allocated_mb": 272.06640625,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.05029715347290039,
+      "std_time": 7.24480602714254e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 81349.66173700758
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.06684800237417221,
+        "mean_ms": 0.06276160031557083,
+        "median_ms": 0.061824001371860504,
+        "min_ms": 0.060447998344898224,
+        "std_ms": 0.0024045636532566625
+      },
+      "max_time": 0.09653209686279297,
+      "mean_time": 0.0961898666381836,
+      "memory_allocated_mb": 376.09033203125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.09607142639160156,
+      "std_time": 0.0001365794879996899,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 63873.67208970714
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0326399989426136,
+        "mean_ms": 0.021955199912190436,
+        "median_ms": 0.020560000091791153,
+        "min_ms": 0.018559999763965607,
+        "std_ms": 0.00393975597463501
+      },
+      "max_time": 0.0011526399850845337,
+      "mean_time": 0.0011425888061523438,
+      "memory_allocated_mb": 34.0205078125,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.0011403199434280396,
+      "std_time": 3.498447348453724e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 896210.4253833097
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02783999964594841,
+        "mean_ms": 0.017366399988532066,
+        "median_ms": 0.01611199975013733,
+        "min_ms": 0.015424000099301338,
+        "std_ms": 0.003547472261377056
+      },
+      "max_time": 0.00034601598978042605,
+      "mean_time": 0.0003366623997688293,
+      "memory_allocated_mb": 17.015625,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.0003308480083942413,
+      "std_time": 4.813544996936389e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 1520811.3539010207
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+f113ab5a05167f8ac4c1938c4a04fadc11d5dc2bb8c536a69db23638e6ef73d4  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/raw/flashattn-code-longctx-tp1.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/raw/flashattn-code-longctx-tp1.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08799999952316284,
+        "mean_ms": 0.08035840019583702,
+        "median_ms": 0.08003199845552444,
+        "min_ms": 0.07583999633789062,
+        "std_ms": 0.003823532942228132
+      },
+      "max_time": 0.04384675216674805,
+      "mean_time": 0.043662386703491214,
+      "memory_allocated_mb": 240.0849609375,
+      "memory_reserved_mb": 246.0,
+      "min_time": 0.04360464096069336,
+      "std_time": 7.18445670309503e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 187621.4430427591
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08611200004816055,
+        "mean_ms": 0.07973119989037514,
+        "median_ms": 0.07807999849319458,
+        "min_ms": 0.07692799717187881,
+        "std_ms": 0.0030488875049308785
+      },
+      "max_time": 0.05954601669311523,
+      "mean_time": 0.0594769702911377,
+      "memory_allocated_mb": 272.0888671875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.05941872024536133,
+      "std_time": 4.5624842215119615e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 137733.98274828802
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.09087999910116196,
+        "mean_ms": 0.08214399963617325,
+        "median_ms": 0.08115199953317642,
+        "min_ms": 0.07846400141716003,
+        "std_ms": 0.003409163930931844
+      },
+      "max_time": 0.07549359893798828,
+      "mean_time": 0.07529304885864258,
+      "memory_allocated_mb": 304.0927734375,
+      "memory_reserved_mb": 534.0,
+      "min_time": 0.07523104095458985,
+      "std_time": 8.131664708395529e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 108801.5444211843
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08684799820184708,
+        "mean_ms": 0.08159359991550445,
+        "median_ms": 0.08057600259780884,
+        "min_ms": 0.0790719985961914,
+        "std_ms": 0.0025743110833498623
+      },
+      "max_time": 0.09123193359375,
+      "mean_time": 0.09110229187011719,
+      "memory_allocated_mb": 336.0966796875,
+      "memory_reserved_mb": 726.0,
+      "min_time": 0.09105142211914062,
+      "std_time": 5.304259171893706e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 89920.89915453696
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08508799970149994,
+        "mean_ms": 0.07919999957084656,
+        "median_ms": 0.0780319981276989,
+        "min_ms": 0.0764480009675026,
+        "std_ms": 0.002612344815356456
+      },
+      "max_time": 0.10705843353271484,
+      "mean_time": 0.10692193984985351,
+      "memory_allocated_mb": 368.1005859375,
+      "memory_reserved_mb": 950.0,
+      "min_time": 0.10688256072998047,
+      "std_time": 5.1132907623503757e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 76616.64211763947
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08479999750852585,
+        "mean_ms": 0.07948800027370453,
+        "median_ms": 0.07824000343680382,
+        "min_ms": 0.0764160007238388,
+        "std_ms": 0.0028062370889137406
+      },
+      "max_time": 0.12304061126708984,
+      "mean_time": 0.12278123931884766,
+      "memory_allocated_mb": 400.1044921875,
+      "memory_reserved_mb": 1206.0,
+      "min_time": 0.122716796875,
+      "std_time": 9.14188387039285e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 66720.29086403332
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08329600095748901,
+        "mean_ms": 0.07985600009560585,
+        "median_ms": 0.07945600152015686,
+        "min_ms": 0.07705599814653397,
+        "std_ms": 0.0021785520776957702
+      },
+      "max_time": 0.13100553894042968,
+      "mean_time": 0.1307180953979492,
+      "memory_allocated_mb": 416.1064453125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.1306175994873047,
+      "std_time": 0.000107931256378197,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 62669.21174961153
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02659199945628643,
+        "mean_ms": 0.01834559990093112,
+        "median_ms": 0.01649599988013506,
+        "min_ms": 0.015135999768972397,
+        "std_ms": 0.00375367086613964
+      },
+      "max_time": 0.009480863571166993,
+      "mean_time": 0.009440905380249024,
+      "memory_allocated_mb": 265.0458984375,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.009432319641113282,
+      "std_time": 1.4279626944428254e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 54232.08679446535
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0331839993596077,
+        "mean_ms": 0.028758399933576585,
+        "median_ms": 0.027872000820934772,
+        "min_ms": 0.026623999699950218,
+        "std_ms": 0.002088649186504863
+      },
+      "max_time": 0.01684105682373047,
+      "mean_time": 0.016775878524780276,
+      "memory_allocated_mb": 168.04248046875,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.016756032943725584,
+      "std_time": 2.3830672818836446e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 122080.04468885626
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.052480001002550125,
+        "mean_ms": 0.045238399878144264,
+        "median_ms": 0.04420800134539604,
+        "min_ms": 0.04303999990224838,
+        "std_ms": 0.0027615853312914604
+      },
+      "max_time": 0.050535968780517575,
+      "mean_time": 0.0503309886932373,
+      "memory_allocated_mb": 272.06640625,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.05029235076904297,
+      "std_time": 6.98791171184278e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 81381.27436686649
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0740479975938797,
+        "mean_ms": 0.06334079951047897,
+        "median_ms": 0.06135999970138073,
+        "min_ms": 0.060095999389886856,
+        "std_ms": 0.004142020833679573
+      },
+      "max_time": 0.09652816009521484,
+      "mean_time": 0.09613609313964844,
+      "memory_allocated_mb": 376.09033203125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.09607071685791016,
+      "std_time": 0.0001328159156966631,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 63909.39967859056
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.028192000463604927,
+        "mean_ms": 0.021113600023090838,
+        "median_ms": 0.01923200022429228,
+        "min_ms": 0.018624000251293182,
+        "std_ms": 0.0034253398003137136
+      },
+      "max_time": 0.0011505279541015624,
+      "mean_time": 0.001141315186023712,
+      "memory_allocated_mb": 34.0205078125,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.0011371200084686279,
+      "std_time": 4.555471006662272e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 897210.527415803
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.021727999672293663,
+        "mean_ms": 0.017212800029665232,
+        "median_ms": 0.0161920003592968,
+        "min_ms": 0.015231999568641186,
+        "std_ms": 0.002282587014061408
+      },
+      "max_time": 0.0003445119857788086,
+      "mean_time": 0.0003361823946237564,
+      "memory_allocated_mb": 17.015625,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.00032927998900413513,
+      "std_time": 4.115851563011574e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 1522982.7860944727
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+e48aa6e6d039bdb68759701c9976f78ad818d6778b6c40b14823a0a4e852c52d  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/raw/flashattn-code-longctx-tp2.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/raw/flashattn-code-longctx-tp2.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05100800096988678,
+        "mean_ms": 0.04406719990074635,
+        "median_ms": 0.042847998440265656,
+        "min_ms": 0.04217600077390671,
+        "std_ms": 0.002656672983576728
+      },
+      "max_time": 0.022986656188964845,
+      "mean_time": 0.02263068161010742,
+      "memory_allocated_mb": 120.0849609375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.022516607284545898,
+      "std_time": 0.00013840494695459716,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 361986.4457083454
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.047200001776218414,
+        "mean_ms": 0.043644800409674646,
+        "median_ms": 0.04262400045990944,
+        "min_ms": 0.04182400181889534,
+        "std_ms": 0.001927741871473188
+      },
+      "max_time": 0.030787456512451173,
+      "mean_time": 0.03074285774230957,
+      "memory_allocated_mb": 136.0888671875,
+      "memory_reserved_mb": 198.0,
+      "min_time": 0.030725727081298827,
+      "std_time": 1.9110324860515016e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 266468.39629114367
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05049600079655647,
+        "mean_ms": 0.043852799385786054,
+        "median_ms": 0.04289599880576134,
+        "min_ms": 0.04185599833726883,
+        "std_ms": 0.0024573088797598033
+      },
+      "max_time": 0.03898448181152344,
+      "mean_time": 0.038945382690429686,
+      "memory_allocated_mb": 152.0927734375,
+      "memory_reserved_mb": 278.0,
+      "min_time": 0.038930206298828124,
+      "std_time": 1.7734743323340796e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 210345.85961362437
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.046911999583244324,
+        "mean_ms": 0.04412479996681214,
+        "median_ms": 0.04327999986708164,
+        "min_ms": 0.04227200150489807,
+        "std_ms": 0.001811858548765738
+      },
+      "max_time": 0.04724588775634766,
+      "mean_time": 0.04714819869995117,
+      "memory_allocated_mb": 168.0966796875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.047131519317626956,
+      "std_time": 3.329779976561307e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 173750.01009335453
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.06195199862122536,
+        "mean_ms": 0.04529919996857643,
+        "median_ms": 0.043136000633239746,
+        "min_ms": 0.04182400181889534,
+        "std_ms": 0.005751677082163394
+      },
+      "max_time": 0.05564672088623047,
+      "mean_time": 0.05542207336425782,
+      "memory_allocated_mb": 184.1005859375,
+      "memory_reserved_mb": 486.0,
+      "min_time": 0.05533849716186524,
+      "std_time": 0.00011010194068839293,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 147811.14279429129
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.054687999188899994,
+        "mean_ms": 0.04418559968471527,
+        "median_ms": 0.042767999693751335,
+        "min_ms": 0.04182400181889534,
+        "std_ms": 0.0036820249064621804
+      },
+      "max_time": 0.06380697631835938,
+      "mean_time": 0.06362084197998047,
+      "memory_allocated_mb": 200.1044921875,
+      "memory_reserved_mb": 614.0,
+      "min_time": 0.06354908752441406,
+      "std_time": 8.872184973119366e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 128762.83533905087
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.06566400080919266,
+        "mean_ms": 0.05008000023663044,
+        "median_ms": 0.04843199998140335,
+        "min_ms": 0.0461760014295578,
+        "std_ms": 0.0055350567765421
+      },
+      "max_time": 0.06791651153564453,
+      "mean_time": 0.06770619888305665,
+      "memory_allocated_mb": 208.1064453125,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.06764147186279297,
+      "std_time": 9.93520482760623e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 120993.35267881997
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03859199956059456,
+        "mean_ms": 0.02689919974654913,
+        "median_ms": 0.0244159996509552,
+        "min_ms": 0.023711999878287315,
+        "std_ms": 0.004626699769228834
+      },
+      "max_time": 0.004829855918884277,
+      "mean_time": 0.004790303993225097,
+      "memory_allocated_mb": 132.5458984375,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.004775519847869873,
+      "std_time": 1.597276061319242e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 106882.56960813323
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0453759990632534,
+        "mean_ms": 0.027676799893379213,
+        "median_ms": 0.025200000032782555,
+        "min_ms": 0.024224000051617622,
+        "std_ms": 0.006154880946840619
+      },
+      "max_time": 0.00963871955871582,
+      "mean_time": 0.00959802885055542,
+      "memory_allocated_mb": 84.04248046875,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.009579392433166503,
+      "std_time": 1.6811667352984867e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 213377.14565022237
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03593600168824196,
+        "mean_ms": 0.02780479993671179,
+        "median_ms": 0.027040000073611736,
+        "min_ms": 0.02630399912595749,
+        "std_ms": 0.0027569987978884785
+      },
+      "max_time": 0.025383039474487303,
+      "mean_time": 0.025250255966186526,
+      "memory_allocated_mb": 136.06640625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.025216863632202147,
+      "std_time": 4.761963064809279e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 162216.17735222535
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.041728001087903976,
+        "mean_ms": 0.03665280006825924,
+        "median_ms": 0.0352960005402565,
+        "min_ms": 0.034015998244285583,
+        "std_ms": 0.002710617869728372
+      },
+      "max_time": 0.048498847961425784,
+      "mean_time": 0.048172192001342776,
+      "memory_allocated_mb": 188.09033203125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.04810128021240234,
+      "std_time": 0.00011195495368044632,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 127542.46266868527
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02191999927163124,
+        "mean_ms": 0.016563199926167727,
+        "median_ms": 0.015584000386297703,
+        "min_ms": 0.014879999682307243,
+        "std_ms": 0.002306362959594964
+      },
+      "max_time": 0.0006153280138969421,
+      "mean_time": 0.0006064576089382171,
+      "memory_allocated_mb": 17.0205078125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.0006022400259971619,
+      "std_time": 4.680714919723724e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 1688493.9440248988
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03843199834227562,
+        "mean_ms": 0.019180799927562477,
+        "median_ms": 0.01601599995046854,
+        "min_ms": 0.014911999925971031,
+        "std_ms": 0.006856655009795906
+      },
+      "max_time": 0.00022172799706459046,
+      "mean_time": 0.00020788159817457198,
+      "memory_allocated_mb": 8.515625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.00019884799420833587,
+      "std_time": 7.012280869814824e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 2462940.464648726
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+e89bb293d5d8d6d4ec226de86ce6e4cf46daaf6eefb5b3f2809a7d079b948ecf  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/raw/flashattn-code-longctx-tp2.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/raw/flashattn-code-longctx-tp2.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.047520000487565994,
+        "mean_ms": 0.04390720054507256,
+        "median_ms": 0.043088000267744064,
+        "min_ms": 0.041760001331567764,
+        "std_ms": 0.0020753894539536763
+      },
+      "max_time": 0.0226856632232666,
+      "mean_time": 0.022550320053100585,
+      "memory_allocated_mb": 120.0849609375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.02251683235168457,
+      "std_time": 5.03848780842368e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 363276.440454495
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05392000079154968,
+        "mean_ms": 0.04477120004594326,
+        "median_ms": 0.04323200136423111,
+        "min_ms": 0.04150399938225746,
+        "std_ms": 0.003602053060896233
+      },
+      "max_time": 0.03088857650756836,
+      "mean_time": 0.030766559982299803,
+      "memory_allocated_mb": 136.0888671875,
+      "memory_reserved_mb": 198.0,
+      "min_time": 0.030721696853637695,
+      "std_time": 5.739627345668801e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 266263.11179127306
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04835199937224388,
+        "mean_ms": 0.04382400028407574,
+        "median_ms": 0.0427200011909008,
+        "min_ms": 0.041600000113248825,
+        "std_ms": 0.002316465883417635
+      },
+      "max_time": 0.03911993789672852,
+      "mean_time": 0.03901147232055664,
+      "memory_allocated_mb": 152.0927734375,
+      "memory_reserved_mb": 278.0,
+      "min_time": 0.03894742584228516,
+      "std_time": 5.231496369987919e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 209989.51110295113
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04902400076389313,
+        "mean_ms": 0.04363839998841286,
+        "median_ms": 0.04273599945008755,
+        "min_ms": 0.04153599962592125,
+        "std_ms": 0.002234876899495284
+      },
+      "max_time": 0.047580448150634766,
+      "mean_time": 0.0472187198638916,
+      "memory_allocated_mb": 168.0966796875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.04714566421508789,
+      "std_time": 0.00012505866815108255,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 173490.51443185066
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04848000034689903,
+        "mean_ms": 0.04378239996731281,
+        "median_ms": 0.042527999728918076,
+        "min_ms": 0.04163200035691261,
+        "std_ms": 0.002414242667964844
+      },
+      "max_time": 0.05583651351928711,
+      "mean_time": 0.055455712127685554,
+      "memory_allocated_mb": 184.1005859375,
+      "memory_reserved_mb": 486.0,
+      "min_time": 0.05535052871704101,
+      "std_time": 0.00014272069779462445,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 147721.48234501254
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04851200059056282,
+        "mean_ms": 0.04361280016601086,
+        "median_ms": 0.042128000408411026,
+        "min_ms": 0.04179200157523155,
+        "std_ms": 0.0023557503035821635
+      },
+      "max_time": 0.06379235076904297,
+      "mean_time": 0.06362125053405762,
+      "memory_allocated_mb": 200.1044921875,
+      "memory_reserved_mb": 614.0,
+      "min_time": 0.06355436706542969,
+      "std_time": 7.081883106781061e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 128762.00846782589
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05718399956822395,
+        "mean_ms": 0.04485439993441105,
+        "median_ms": 0.042527999728918076,
+        "min_ms": 0.04156799986958504,
+        "std_ms": 0.004775190073319638
+      },
+      "max_time": 0.06783235168457032,
+      "mean_time": 0.06771669235229492,
+      "memory_allocated_mb": 208.1064453125,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.06765017700195312,
+      "std_time": 5.8650788028093655e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 120974.603387024
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.019807999953627586,
+        "mean_ms": 0.015823999978601934,
+        "median_ms": 0.015039999969303608,
+        "min_ms": 0.014303999952971935,
+        "std_ms": 0.001913656346272671
+      },
+      "max_time": 0.004766176223754883,
+      "mean_time": 0.0047575551986694335,
+      "memory_allocated_mb": 132.5458984375,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.004751776218414307,
+      "std_time": 4.190866412679079e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 107618.29944573072
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02687999978661537,
+        "mean_ms": 0.01938559990376234,
+        "median_ms": 0.018383999355137348,
+        "min_ms": 0.018015999346971512,
+        "std_ms": 0.0025744710494052777
+      },
+      "max_time": 0.009582624435424805,
+      "mean_time": 0.009561939239501951,
+      "memory_allocated_mb": 84.04248046875,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.009554431915283204,
+      "std_time": 8.291579854680458e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 214182.49464913702
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03206399828195572,
+        "mean_ms": 0.028105599991977214,
+        "median_ms": 0.02675200067460537,
+        "min_ms": 0.025728000327944756,
+        "std_ms": 0.0022390414558466644
+      },
+      "max_time": 0.02538035202026367,
+      "mean_time": 0.0252856897354126,
+      "memory_allocated_mb": 136.06640625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.025221471786499024,
+      "std_time": 4.676405035080222e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 161988.85784252718
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04089599847793579,
+        "mean_ms": 0.03524160049855709,
+        "median_ms": 0.03411199897527695,
+        "min_ms": 0.03385600075125694,
+        "std_ms": 0.0022532652248236193
+      },
+      "max_time": 0.048259136199951175,
+      "mean_time": 0.048143408584594725,
+      "memory_allocated_mb": 188.09033203125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.048105857849121095,
+      "std_time": 5.1879732806325464e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 127618.7162611083
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.018783999606966972,
+        "mean_ms": 0.015311999991536141,
+        "median_ms": 0.014943999703973532,
+        "min_ms": 0.014271999709308147,
+        "std_ms": 0.0012243293501672883
+      },
+      "max_time": 0.0008861759901046753,
+      "mean_time": 0.0006353183984756471,
+      "memory_allocated_mb": 17.0205078125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.0006019840240478516,
+      "std_time": 8.386037831062328e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 1611790.2495141604
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.019840000197291374,
+        "mean_ms": 0.015875199995934963,
+        "median_ms": 0.01497599994763732,
+        "min_ms": 0.014431999996304512,
+        "std_ms": 0.001896142444948042
+      },
+      "max_time": 0.00022230400145053863,
+      "mean_time": 0.00020609600096940998,
+      "memory_allocated_mb": 8.515625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.00019760000705718993,
+      "std_time": 7.520956759601396e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 2484279.1591865686
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+759d33decc09d229d07818df8161ed41180cab575304f335ad4d5e7a917cbc93  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/raw/flashattn-code-longctx-tp4.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/raw/flashattn-code-longctx-tp4.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.039583999663591385,
+        "mean_ms": 0.027286400087177753,
+        "median_ms": 0.025200000032782555,
+        "min_ms": 0.024191999807953835,
+        "std_ms": 0.004620118437533604
+      },
+      "max_time": 0.011496224403381348,
+      "mean_time": 0.011354345512390137,
+      "memory_allocated_mb": 60.0849609375,
+      "memory_reserved_mb": 62.0,
+      "min_time": 0.01132140827178955,
+      "std_time": 5.011776621543997e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 721485.8831854897
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03232000023126602,
+        "mean_ms": 0.026406400091946124,
+        "median_ms": 0.02518399991095066,
+        "min_ms": 0.024032000452280045,
+        "std_ms": 0.002792696117638337
+      },
+      "max_time": 0.015639776229858397,
+      "mean_time": 0.015451993656158448,
+      "memory_allocated_mb": 68.0888671875,
+      "memory_reserved_mb": 94.0,
+      "min_time": 0.01542249584197998,
+      "std_time": 6.304729308178388e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 530158.1260185833
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03488000109791756,
+        "mean_ms": 0.026147199980914592,
+        "median_ms": 0.02459200005978346,
+        "min_ms": 0.024224000051617622,
+        "std_ms": 0.003303059079404486
+      },
+      "max_time": 0.01970569610595703,
+      "mean_time": 0.01954985942840576,
+      "memory_allocated_mb": 76.0927734375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.019525152206420898,
+      "std_time": 5.224891496533832e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 419031.1459783236
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03046399913728237,
+        "mean_ms": 0.0255103999748826,
+        "median_ms": 0.024656000547111034,
+        "min_ms": 0.024159999564290047,
+        "std_ms": 0.00195717586616911
+      },
+      "max_time": 0.023927040100097656,
+      "mean_time": 0.02367282257080078,
+      "memory_allocated_mb": 84.0966796875,
+      "memory_reserved_mb": 182.0,
+      "min_time": 0.023630016326904296,
+      "std_time": 8.715594728873713e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 346050.83426360885
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03574400022625923,
+        "mean_ms": 0.02678720001131296,
+        "median_ms": 0.024720000103116035,
+        "min_ms": 0.02380800060927868,
+        "std_ms": 0.003678231234048846
+      },
+      "max_time": 0.02779840087890625,
+      "mean_time": 0.027747158622741696,
+      "memory_allocated_mb": 92.1005859375,
+      "memory_reserved_mb": 238.0,
+      "min_time": 0.027736671447753908,
+      "std_time": 1.8952179343788658e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 295237.4371509809
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.042080000042915344,
+        "mean_ms": 0.02844479978084564,
+        "median_ms": 0.02527999971061945,
+        "min_ms": 0.024224000051617622,
+        "std_ms": 0.006562862750709003
+      },
+      "max_time": 0.03208835220336914,
+      "mean_time": 0.03186994876861572,
+      "memory_allocated_mb": 100.1044921875,
+      "memory_reserved_mb": 302.0,
+      "min_time": 0.03183427238464356,
+      "std_time": 7.444245241186317e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 257044.65543625728
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03139200061559677,
+        "mean_ms": 0.026396799832582474,
+        "median_ms": 0.025071999989449978,
+        "min_ms": 0.02377600036561489,
+        "std_ms": 0.0025962810796740263
+      },
+      "max_time": 0.033969856262207034,
+      "mean_time": 0.033897081375122075,
+      "memory_allocated_mb": 104.1064453125,
+      "memory_reserved_mb": 370.0,
+      "min_time": 0.03388313674926758,
+      "std_time": 2.518493598134421e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 241672.72424853416
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.022624000906944275,
+        "mean_ms": 0.017174400109797715,
+        "median_ms": 0.015536000020802021,
+        "min_ms": 0.014944000169634819,
+        "std_ms": 0.0028331860349340154
+      },
+      "max_time": 0.003192960023880005,
+      "mean_time": 0.003182867193222046,
+      "memory_allocated_mb": 66.2958984375,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.003179744005203247,
+      "std_time": 3.5239767128429387e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 160861.25148115202
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02985600009560585,
+        "mean_ms": 0.017318400088697672,
+        "median_ms": 0.015568000264465809,
+        "min_ms": 0.014655999839305878,
+        "std_ms": 0.004479309643844885
+      },
+      "max_time": 0.004826848030090332,
+      "mean_time": 0.004817302370071412,
+      "memory_allocated_mb": 42.04248046875,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.004811935901641846,
+      "std_time": 5.94674080057053e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 425134.20222979283
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03916800022125244,
+        "mean_ms": 0.021356799826025962,
+        "median_ms": 0.01935999933630228,
+        "min_ms": 0.017184000462293625,
+        "std_ms": 0.0062801287963799276
+      },
+      "max_time": 0.014635007858276367,
+      "mean_time": 0.014412275123596191,
+      "memory_allocated_mb": 68.06640625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.014374591827392579,
+      "std_time": 7.569270440104603e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 284202.17938345566
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.033215999603271484,
+        "mean_ms": 0.023340800032019614,
+        "median_ms": 0.02092800009995699,
+        "min_ms": 0.020479999482631683,
+        "std_ms": 0.004304974035052475
+      },
+      "max_time": 0.02418492889404297,
+      "mean_time": 0.02411243553161621,
+      "memory_allocated_mb": 96.09033203125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.024088096618652344,
+      "std_time": 2.797845886790172e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 254806.28001862322
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04992000013589859,
+        "mean_ms": 0.019958400167524815,
+        "median_ms": 0.015552000608295202,
+        "min_ms": 0.014688000082969666,
+        "std_ms": 0.010290953685254347
+      },
+      "max_time": 0.00036675199866294863,
+      "mean_time": 0.0003469599992036819,
+      "memory_allocated_mb": 8.5205078125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.0003364480137825012,
+      "std_time": 7.875187259592915e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 2951348.865431786
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.022336000576615334,
+        "mean_ms": 0.016252800077199935,
+        "median_ms": 0.014895999804139137,
+        "min_ms": 0.014271999709308147,
+        "std_ms": 0.0028271219653727385
+      },
+      "max_time": 0.00016867199540138245,
+      "mean_time": 0.00015103680044412614,
+      "memory_allocated_mb": 4.265625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.00014329600334167482,
+      "std_time": 7.872199103975806e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 3389902.3184710997
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+6ee8fdf15bef211e7d53fef0c5daf02b17429bc7d88a49cb967c2b2f6134ebe2  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/raw/flashattn-code-longctx-tp4.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/raw/flashattn-code-longctx-tp4.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.040991999208927155,
+        "mean_ms": 0.02782719973474741,
+        "median_ms": 0.025295999832451344,
+        "min_ms": 0.02412799932062626,
+        "std_ms": 0.005434033125981124
+      },
+      "max_time": 0.011621503829956055,
+      "mean_time": 0.011469862461090087,
+      "memory_allocated_mb": 60.0849609375,
+      "memory_reserved_mb": 62.0,
+      "min_time": 0.01142630386352539,
+      "std_time": 5.2644635131708985e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 714219.5495185945
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03811199963092804,
+        "mean_ms": 0.02837119996547699,
+        "median_ms": 0.026176000013947487,
+        "min_ms": 0.024159999564290047,
+        "std_ms": 0.004902838828525061
+      },
+      "max_time": 0.015688991546630858,
+      "mean_time": 0.015603910255432129,
+      "memory_allocated_mb": 68.0888671875,
+      "memory_reserved_mb": 94.0,
+      "min_time": 0.01557091236114502,
+      "std_time": 3.431861179521254e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 524996.6108429873
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.029759999364614487,
+        "mean_ms": 0.02612799983471632,
+        "median_ms": 0.025695999152958393,
+        "min_ms": 0.024000000208616257,
+        "std_ms": 0.0019206532069484194
+      },
+      "max_time": 0.020114784240722657,
+      "mean_time": 0.019786701011657717,
+      "memory_allocated_mb": 76.0927734375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.019717824935913085,
+      "std_time": 0.00011854565766692264,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 414015.4538734641
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.032896000891923904,
+        "mean_ms": 0.026428800262510776,
+        "median_ms": 0.02483200002461672,
+        "min_ms": 0.024159999564290047,
+        "std_ms": 0.0028081875041272375
+      },
+      "max_time": 0.023987648010253906,
+      "mean_time": 0.023885836791992184,
+      "memory_allocated_mb": 84.0966796875,
+      "memory_reserved_mb": 182.0,
+      "min_time": 0.023849760055541992,
+      "std_time": 4.082290087440645e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 342964.7481618228
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03388800099492073,
+        "mean_ms": 0.02606080025434494,
+        "median_ms": 0.02440000046044588,
+        "min_ms": 0.023840000852942467,
+        "std_ms": 0.003071273491702287
+      },
+      "max_time": 0.02814064025878906,
+      "mean_time": 0.028014582252502435,
+      "memory_allocated_mb": 92.1005859375,
+      "memory_reserved_mb": 238.0,
+      "min_time": 0.027969343185424805,
+      "std_time": 4.739888751775852e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 292419.138224638
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02953599952161312,
+        "mean_ms": 0.025788800045847892,
+        "median_ms": 0.024800000712275505,
+        "min_ms": 0.024191999807953835,
+        "std_ms": 0.002060583463186555
+      },
+      "max_time": 0.03218783950805664,
+      "mean_time": 0.03215217628479004,
+      "memory_allocated_mb": 100.1044921875,
+      "memory_reserved_mb": 302.0,
+      "min_time": 0.03213116836547852,
+      "std_time": 1.9548439899050275e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 254788.35172583075
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.032735999673604965,
+        "mean_ms": 0.02950399983674288,
+        "median_ms": 0.028351999819278717,
+        "min_ms": 0.027904000133275986,
+        "std_ms": 0.0017488521169760844
+      },
+      "max_time": 0.03428476715087891,
+      "mean_time": 0.03422174110412597,
+      "memory_allocated_mb": 104.1064453125,
+      "memory_reserved_mb": 370.0,
+      "min_time": 0.034186080932617186,
+      "std_time": 3.324003475365458e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 239379.98873506542
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03446400165557861,
+        "mean_ms": 0.0248607998713851,
+        "median_ms": 0.023519999347627163,
+        "min_ms": 0.0226879995316267,
+        "std_ms": 0.0035075699446123015
+      },
+      "max_time": 0.003272864103317261,
+      "mean_time": 0.003240902423858643,
+      "memory_allocated_mb": 66.2958984375,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.003228480100631714,
+      "std_time": 1.4700871477091224e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 157980.68964705482
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03484800085425377,
+        "mean_ms": 0.02619840018451214,
+        "median_ms": 0.024639999493956566,
+        "min_ms": 0.02284800074994564,
+        "std_ms": 0.0037605668537262792
+      },
+      "max_time": 0.005165311813354492,
+      "mean_time": 0.004916048002243043,
+      "memory_allocated_mb": 42.04248046875,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.004875135898590088,
+      "std_time": 8.365277131314812e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 416594.79302593466
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.023615999147295952,
+        "mean_ms": 0.018966399878263474,
+        "median_ms": 0.01788800023496151,
+        "min_ms": 0.01679999940097332,
+        "std_ms": 0.0021688374822942504
+      },
+      "max_time": 0.0145830717086792,
+      "mean_time": 0.01451537265777588,
+      "memory_allocated_mb": 68.06640625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.014502431869506836,
+      "std_time": 2.2952325515836057e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 282183.5922900522
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.028192000463604927,
+        "mean_ms": 0.021894400380551814,
+        "median_ms": 0.021088000386953354,
+        "min_ms": 0.020640000700950623,
+        "std_ms": 0.0021379161656161685
+      },
+      "max_time": 0.024349727630615235,
+      "mean_time": 0.024299632072448733,
+      "memory_allocated_mb": 96.09033203125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.024284608840942384,
+      "std_time": 2.0842542550131525e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 252843.3344867865
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0261439997702837,
+        "mean_ms": 0.016019200067967178,
+        "median_ms": 0.01512000011280179,
+        "min_ms": 0.014368000440299511,
+        "std_ms": 0.0034013214299104646
+      },
+      "max_time": 0.0003514559864997864,
+      "mean_time": 0.00034397439956665037,
+      "memory_allocated_mb": 8.5205078125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.0003375680148601532,
+      "std_time": 3.760143706773439e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 2976965.731432534
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.021023999899625778,
+        "mean_ms": 0.015327999833971262,
+        "median_ms": 0.014639999717473984,
+        "min_ms": 0.013856000266969204,
+        "std_ms": 0.0020222886267191407
+      },
+      "max_time": 0.0001605760008096695,
+      "mean_time": 0.0001493696004152298,
+      "memory_allocated_mb": 4.265625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.00014313599467277526,
+      "std_time": 5.429351067501872e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 3427738.9681481416
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/repeat-check.json

@@ -0,0 +1,320 @@
+{
+  "comparisons": [
+    {
+      "batch_spec": "q1ks8k",
+      "first_mean_s": 0.0011425888061523438,
+      "pass": true,
+      "relative_difference": 0.0011153009347107706,
+      "second_mean_s": 0.001141315186023712,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q2ks66k",
+      "first_mean_s": 0.01678766403198242,
+      "pass": true,
+      "relative_difference": 0.0007022802901221452,
+      "second_mean_s": 0.016775878524780276,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q4ks100k",
+      "first_mean_s": 0.05035054740905761,
+      "pass": true,
+      "relative_difference": 0.000388526368934946,
+      "second_mean_s": 0.0503309886932373,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q512s128k",
+      "first_mean_s": 0.009489968109130859,
+      "pass": true,
+      "relative_difference": 0.005183356056904404,
+      "second_mean_s": 0.009440905380249024,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q512s4k",
+      "first_mean_s": 0.0003366623997688293,
+      "pass": true,
+      "relative_difference": 0.0014267930704769689,
+      "second_mean_s": 0.0003361823946237564,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q6ks134k",
+      "first_mean_s": 0.0961898666381836,
+      "pass": true,
+      "relative_difference": 0.0005591912667149224,
+      "second_mean_s": 0.09613609313964844,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks112k",
+      "first_mean_s": 0.1069560287475586,
+      "pass": true,
+      "relative_difference": 0.00031876960426212737,
+      "second_mean_s": 0.10692193984985351,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks128k",
+      "first_mean_s": 0.12283342437744141,
+      "pass": true,
+      "relative_difference": 0.0004249343895711351,
+      "second_mean_s": 0.12278123931884766,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks136k",
+      "first_mean_s": 0.13079229431152345,
+      "pass": true,
+      "relative_difference": 0.0005674643646600315,
+      "second_mean_s": 0.1307180953979492,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks48k",
+      "first_mean_s": 0.04368390693664551,
+      "pass": true,
+      "relative_difference": 0.0004927566415802605,
+      "second_mean_s": 0.043662386703491214,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks64k",
+      "first_mean_s": 0.0595021312713623,
+      "pass": true,
+      "relative_difference": 0.0004229478941120101,
+      "second_mean_s": 0.0594769702911377,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks80k",
+      "first_mean_s": 0.07538758392333984,
+      "pass": true,
+      "relative_difference": 0.0012547739275033378,
+      "second_mean_s": 0.07529304885864258,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks96k",
+      "first_mean_s": 0.09124225158691406,
+      "pass": true,
+      "relative_difference": 0.0015351127502189047,
+      "second_mean_s": 0.09110229187011719,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q1ks8k",
+      "first_mean_s": 0.0006064576089382171,
+      "pass": true,
+      "relative_difference": 0.04648308449369336,
+      "second_mean_s": 0.0006353183984756471,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q2ks66k",
+      "first_mean_s": 0.00959802885055542,
+      "pass": true,
+      "relative_difference": 0.003767189056248553,
+      "second_mean_s": 0.009561939239501951,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q4ks100k",
+      "first_mean_s": 0.025250255966186526,
+      "pass": true,
+      "relative_difference": 0.001402319427652538,
+      "second_mean_s": 0.0252856897354126,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q512s128k",
+      "first_mean_s": 0.004790303993225097,
+      "pass": true,
+      "relative_difference": 0.006859924072500961,
+      "second_mean_s": 0.0047575551986694335,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q512s4k",
+      "first_mean_s": 0.00020788159817457198,
+      "pass": true,
+      "relative_difference": 0.008626540222728182,
+      "second_mean_s": 0.00020609600096940998,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q6ks134k",
+      "first_mean_s": 0.048172192001342776,
+      "pass": true,
+      "relative_difference": 0.0005976896073522018,
+      "second_mean_s": 0.048143408584594725,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks112k",
+      "first_mean_s": 0.05542207336425782,
+      "pass": true,
+      "relative_difference": 0.0006067719206058191,
+      "second_mean_s": 0.055455712127685554,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks128k",
+      "first_mean_s": 0.06362084197998047,
+      "pass": true,
+      "relative_difference": 6.421681207537411e-06,
+      "second_mean_s": 0.06362125053405762,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks136k",
+      "first_mean_s": 0.06770619888305665,
+      "pass": true,
+      "relative_difference": 0.00015497334523802088,
+      "second_mean_s": 0.06771669235229492,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks48k",
+      "first_mean_s": 0.02263068161010742,
+      "pass": true,
+      "relative_difference": 0.003557316307676039,
+      "second_mean_s": 0.022550320053100585,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks64k",
+      "first_mean_s": 0.03074285774230957,
+      "pass": true,
+      "relative_difference": 0.0007706865344215391,
+      "second_mean_s": 0.030766559982299803,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks80k",
+      "first_mean_s": 0.038945382690429686,
+      "pass": true,
+      "relative_difference": 0.001695543775275234,
+      "second_mean_s": 0.03901147232055664,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks96k",
+      "first_mean_s": 0.04714819869995117,
+      "pass": true,
+      "relative_difference": 0.0014946162280953233,
+      "second_mean_s": 0.0472187198638916,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q1ks8k",
+      "first_mean_s": 0.0003469599992036819,
+      "pass": true,
+      "relative_difference": 0.008642208702722183,
+      "second_mean_s": 0.00034397439956665037,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q2ks66k",
+      "first_mean_s": 0.004817302370071412,
+      "pass": true,
+      "relative_difference": 0.020290162871872575,
+      "second_mean_s": 0.004916048002243043,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q4ks100k",
+      "first_mean_s": 0.014412275123596191,
+      "pass": true,
+      "relative_difference": 0.007127958343441791,
+      "second_mean_s": 0.01451537265777588,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q512s128k",
+      "first_mean_s": 0.003182867193222046,
+      "pass": true,
+      "relative_difference": 0.018068901625077752,
+      "second_mean_s": 0.003240902423858643,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q512s4k",
+      "first_mean_s": 0.00015103680044412614,
+      "pass": true,
+      "relative_difference": 0.011099630528025208,
+      "second_mean_s": 0.0001493696004152298,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q6ks134k",
+      "first_mean_s": 0.02411243553161621,
+      "pass": true,
+      "relative_difference": 0.007733466059061839,
+      "second_mean_s": 0.024299632072448733,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks112k",
+      "first_mean_s": 0.027747158622741696,
+      "pass": true,
+      "relative_difference": 0.009591652827304903,
+      "second_mean_s": 0.028014582252502435,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks128k",
+      "first_mean_s": 0.03186994876861572,
+      "pass": true,
+      "relative_difference": 0.008816561960069082,
+      "second_mean_s": 0.03215217628479004,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks136k",
+      "first_mean_s": 0.033897081375122075,
+      "pass": true,
+      "relative_difference": 0.009532159165047292,
+      "second_mean_s": 0.03422174110412597,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks48k",
+      "first_mean_s": 0.011354345512390137,
+      "pass": true,
+      "relative_difference": 0.010122318271387163,
+      "second_mean_s": 0.011469862461090087,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks64k",
+      "first_mean_s": 0.015451993656158448,
+      "pass": true,
+      "relative_difference": 0.00978342795663941,
+      "second_mean_s": 0.015603910255432129,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks80k",
+      "first_mean_s": 0.01954985942840576,
+      "pass": true,
+      "relative_difference": 0.012041804397861736,
+      "second_mean_s": 0.019786701011657717,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks96k",
+      "first_mean_s": 0.02367282257080078,
+      "pass": true,
+      "relative_difference": 0.008957957353947301,
+      "second_mean_s": 0.023885836791992184,
+      "tp": 4
+    }
+  ],
+  "maximum_relative_difference": 0.04648308449369336,
+  "schema": "frontier-code-longctx-repeat-check-v1",
+  "status": "PASS",
+  "threshold": 0.05
+}

runs/frontier-code-trace-v0/results/calibration-summary.json

@@ -0,0 +1,634 @@
+{
+  "attention_profile_sha256": "fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550",
+  "cell_version": "v3",
+  "cells": [
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 5525
+        },
+        "max": 1,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 5525
+      },
+      "drain_fraction": 0.00044126232317770095,
+      "drain_tail_s": 1.6150201028303854,
+      "offered_load": {
+        "decode_tokens_per_s": 1.5183060109289617,
+        "prefill_tokens_per_s_after_prefix": 189.91639344262296,
+        "prefill_tokens_per_s_raw": 201.38743169398907,
+        "requests_per_s": 0.008743169398907104
+      },
+      "prefix_cache_hit_ratio": 0.05698154180238871,
+      "requests": 32,
+      "rho": 5e-05,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 32,
+        "max": 6.06516500761245,
+        "mean": 5.267661756941212,
+        "min": 4.642776716764274,
+        "p50": 5.27961358181982,
+        "p90": 6.065165007589712,
+        "p95": 6.06516500761245,
+        "p99": 6.06516500761245
+      },
+      "trace_sha256": "bdc54edafacf3e3f353de7a54d5e43a7e4323eb1e8b3dc55ce333a5e0b347ce5",
+      "ttft_ms": {
+        "count": 32,
+        "max": 9026.404118319988,
+        "mean": 1316.2539772416199,
+        "min": 76.98844366404956,
+        "p50": 401.828087941567,
+        "p90": 3363.622890573971,
+        "p95": 6721.082190563591,
+        "p99": 8741.382807755395
+      },
+      "waiting_ms": {
+        "count": 32,
+        "max": 0.0,
+        "mean": 0.0,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 33249,
+          "2": 294,
+          "3": 76
+        },
+        "max": 3,
+        "share_gt_1": 0.011005681311163331,
+        "share_gt_4": 0.0,
+        "stages": 33619
+      },
+      "drain_fraction": 0.0006573023445132039,
+      "drain_tail_s": 2.4057265809183264,
+      "offered_load": {
+        "decode_tokens_per_s": 9.343989071038251,
+        "prefill_tokens_per_s_after_prefix": 491.8571038251366,
+        "prefill_tokens_per_s_raw": 1477.1773224043716,
+        "requests_per_s": 0.03224043715846994
+      },
+      "prefix_cache_hit_ratio": 0.667142227102801,
+      "requests": 118,
+      "rho": 0.0001,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 118,
+        "max": 109.2514556116484,
+        "mean": 6.521761912064516,
+        "min": 4.642776716750063,
+        "p50": 5.7134254039277,
+        "p90": 6.06516500761245,
+        "p95": 6.582261050425584,
+        "p99": 14.049063410907634
+      },
+      "trace_sha256": "8d4ac5646f46e84560c17c44013af2bcb7cfe91d4a2be3476d8f3c09565cf55f",
+      "ttft_ms": {
+        "count": 118,
+        "max": 9696.373176562247,
+        "mean": 1203.6180593647337,
+        "min": 72.1560867923472,
+        "p50": 592.0461311603731,
+        "p90": 3231.0798294979118,
+        "p95": 6885.471905257904,
+        "p99": 8918.025104477005
+      },
+      "waiting_ms": {
+        "count": 118,
+        "max": 7151.845512636555,
+        "mean": 98.59009397263883,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.21863999056675887,
+        "p99": 2087.1951396751865
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 71338,
+          "2": 4093,
+          "3": 1021,
+          "4": 33
+        },
+        "max": 4,
+        "share_gt_1": 0.067294240700791,
+        "share_gt_4": 0.0,
+        "stages": 76485
+      },
+      "drain_fraction": 0.001483881651363999,
+      "drain_tail_s": 5.431006843992236,
+      "offered_load": {
+        "decode_tokens_per_s": 22.691803278688525,
+        "prefill_tokens_per_s_after_prefix": 884.2945355191257,
+        "prefill_tokens_per_s_raw": 2459.7437158469947,
+        "requests_per_s": 0.0633879781420765
+      },
+      "prefix_cache_hit_ratio": 0.6406200282639031,
+      "requests": 232,
+      "rho": 0.0002,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 231,
+        "max": 151.0151050020596,
+        "mean": 8.197384891414327,
+        "min": 4.790955988028145,
+        "p50": 6.377284733844135,
+        "p90": 9.066414456096654,
+        "p95": 13.235755136549969,
+        "p99": 45.72584194314243
+      },
+      "trace_sha256": "188f9e52674ec268d70d763995f2cdf84ccbcbab86420214b127994569324e9b",
+      "ttft_ms": {
+        "count": 232,
+        "max": 22394.384315265255,
+        "mean": 2036.2309970727358,
+        "min": 90.95485497891787,
+        "p50": 876.6052006358223,
+        "p90": 5676.270318256946,
+        "p95": 9955.05948414866,
+        "p99": 15641.83197487735
+      },
+      "waiting_ms": {
+        "count": 232,
+        "max": 13155.607757982125,
+        "mean": 338.8558697712575,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 5.941389865392926,
+        "p95": 1644.4218195787325,
+        "p99": 8012.825782587187
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 72379,
+          "2": 4138,
+          "3": 696
+        },
+        "max": 3,
+        "share_gt_1": 0.06260603784336835,
+        "share_gt_4": 0.0,
+        "stages": 77213
+      },
+      "drain_fraction": 0.0010192168060329308,
+      "drain_tail_s": 3.730333510080527,
+      "offered_load": {
+        "decode_tokens_per_s": 22.691803278688525,
+        "prefill_tokens_per_s_after_prefix": 880.7972677595628,
+        "prefill_tokens_per_s_raw": 2459.7437158469947,
+        "requests_per_s": 0.0633879781420765
+      },
+      "prefix_cache_hit_ratio": 0.6420421114379927,
+      "requests": 232,
+      "rho": 0.0002,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 231,
+        "max": 109.2514556116484,
+        "mean": 6.682483450713786,
+        "min": 4.6427767165369005,
+        "p50": 5.757631515507455,
+        "p90": 6.873691941109428,
+        "p95": 8.022302513342353,
+        "p99": 28.619774468019706
+      },
+      "trace_sha256": "188f9e52674ec268d70d763995f2cdf84ccbcbab86420214b127994569324e9b",
+      "ttft_ms": {
+        "count": 232,
+        "max": 12080.695954866542,
+        "mean": 1109.6819359234096,
+        "min": 72.1560867923472,
+        "p50": 527.4056752296019,
+        "p90": 2603.3383651315494,
+        "p95": 5585.788369509169,
+        "p99": 8828.771798959253
+      },
+      "waiting_ms": {
+        "count": 232,
+        "max": 7151.845512636555,
+        "mean": 100.5913696184332,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 2.71668797665825,
+        "p95": 337.7136230834021,
+        "p99": 2261.809117635907
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 163554,
+          "10": 18,
+          "11": 5,
+          "12": 10,
+          "2": 48104,
+          "3": 15894,
+          "4": 2673,
+          "5": 1305,
+          "6": 332,
+          "7": 181,
+          "8": 46,
+          "9": 15
+        },
+        "max": 12,
+        "share_gt_1": 0.2954419157652593,
+        "share_gt_4": 0.008236515505929689,
+        "stages": 232137
+      },
+      "drain_fraction": 0.001932022033012745,
+      "drain_tail_s": 7.071200640826646,
+      "offered_load": {
+        "decode_tokens_per_s": 90.30054644808743,
+        "prefill_tokens_per_s_after_prefix": 1805.1767759562842,
+        "prefill_tokens_per_s_raw": 5620.918852459016,
+        "requests_per_s": 0.13633879781420766
+      },
+      "prefix_cache_hit_ratio": 0.6789729811301968,
+      "requests": 499,
+      "rho": 0.0004,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 497,
+        "max": 556.5990343567938,
+        "mean": 18.61788167095804,
+        "min": 4.790955988028145,
+        "p50": 7.918935117459114,
+        "p90": 26.775158132602897,
+        "p95": 56.43467403678585,
+        "p99": 256.82570583636533
+      },
+      "trace_sha256": "2751c373bb91eccf42b94e1b28245870d6ffa2f91e1b5dabafe2435bc46da274",
+      "ttft_ms": {
+        "count": 499,
+        "max": 28488.652032648133,
+        "mean": 2601.050597929721,
+        "min": 90.96449664048123,
+        "p50": 942.4448686800133,
+        "p90": 7645.075071658986,
+        "p95": 12524.265102993284,
+        "p99": 19300.755134457228
+      },
+      "waiting_ms": {
+        "count": 499,
+        "max": 25277.458416518584,
+        "mean": 926.4084886277059,
+        "min": 0.0,
+        "p50": 2.7291107814733095,
+        "p90": 2483.439910098784,
+        "p95": 6352.661668917902,
+        "p99": 13692.62653031437
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 184431,
+          "2": 52296,
+          "3": 10583,
+          "4": 1254,
+          "5": 675
+        },
+        "max": 5,
+        "share_gt_1": 0.2600235115692167,
+        "share_gt_4": 0.002708243894414598,
+        "stages": 249239
+      },
+      "drain_fraction": 0.001114813580178276,
+      "drain_tail_s": 4.08021770345249,
+      "offered_load": {
+        "decode_tokens_per_s": 90.30054644808743,
+        "prefill_tokens_per_s_after_prefix": 1738.4489071038251,
+        "prefill_tokens_per_s_raw": 5620.918852459016,
+        "requests_per_s": 0.13633879781420766
+      },
+      "prefix_cache_hit_ratio": 0.6908465352465023,
+      "requests": 499,
+      "rho": 0.0004,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 497,
+        "max": 191.36693232402044,
+        "mean": 9.29907499058627,
+        "min": 4.6427767165369005,
+        "p50": 6.554626532726112,
+        "p90": 10.321323503714908,
+        "p95": 18.835821286381098,
+        "p99": 73.17188748931419
+      },
+      "trace_sha256": "2751c373bb91eccf42b94e1b28245870d6ffa2f91e1b5dabafe2435bc46da274",
+      "ttft_ms": {
+        "count": 499,
+        "max": 14944.866787426236,
+        "mean": 1252.450784907019,
+        "min": 72.1560867923472,
+        "p50": 526.7327146962089,
+        "p90": 3295.008250609953,
+        "p95": 5487.844406213842,
+        "p99": 10840.596450182773
+      },
+      "waiting_ms": {
+        "count": 499,
+        "max": 12922.960017376226,
+        "mean": 274.43276731097365,
+        "min": 0.0,
+        "p50": 0.6520015415389935,
+        "p90": 164.9360333363801,
+        "p95": 2029.720098907419,
+        "p99": 6246.634668026156
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 123423,
+          "10": 262,
+          "11": 84,
+          "12": 37,
+          "13": 42,
+          "14": 108,
+          "15": 59,
+          "16": 39,
+          "2": 64628,
+          "3": 34551,
+          "4": 17977,
+          "5": 5215,
+          "6": 2255,
+          "7": 1043,
+          "8": 349,
+          "9": 406
+        },
+        "max": 16,
+        "share_gt_1": 0.5072501377366475,
+        "share_gt_4": 0.039520436924600166,
+        "stages": 250478
+      },
+      "drain_fraction": 0.0006878766017147268,
+      "drain_tail_s": 2.5176283622759,
+      "offered_load": {
+        "decode_tokens_per_s": 135.20737704918034,
+        "prefill_tokens_per_s_after_prefix": 3107.11174863388,
+        "prefill_tokens_per_s_raw": 7722.316120218579,
+        "requests_per_s": 0.23743169398907105
+      },
+      "prefix_cache_hit_ratio": 0.5977800565206239,
+      "requests": 869,
+      "rho": 0.0008,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 864,
+        "max": 1048.199917263105,
+        "mean": 35.531282784807445,
+        "min": 4.790955988028145,
+        "p50": 10.56810791578755,
+        "p90": 58.679343047855866,
+        "p95": 113.21830549665293,
+        "p99": 497.8300148864786
+      },
+      "trace_sha256": "2788a21c6bb8be6d03fb5151f2c51efa09c0eb5e1f941af60c00a7cc11d41206",
+      "ttft_ms": {
+        "count": 869,
+        "max": 35023.43713844812,
+        "mean": 3519.677566968775,
+        "min": 80.48812343014333,
+        "p50": 944.5946905507299,
+        "p90": 11539.678186253443,
+        "p95": 16395.861888973763,
+        "p99": 23283.982779742393
+      },
+      "waiting_ms": {
+        "count": 869,
+        "max": 33885.372635082604,
+        "mean": 1944.5601581730987,
+        "min": 0.0,
+        "p50": 4.989910657513974,
+        "p90": 7660.373377703809,
+        "p95": 12268.558603129362,
+        "p99": 19104.247337156932
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 185761,
+          "2": 81136,
+          "3": 28184,
+          "4": 10328,
+          "5": 2183,
+          "6": 759,
+          "7": 199,
+          "8": 89,
+          "9": 18
+        },
+        "max": 9,
+        "share_gt_1": 0.3981636573931581,
+        "share_gt_4": 0.010523007739983218,
+        "stages": 308657
+      },
+      "drain_fraction": 0.0005799250803688739,
+      "drain_tail_s": 2.1225257941500786,
+      "offered_load": {
+        "decode_tokens_per_s": 135.20737704918034,
+        "prefill_tokens_per_s_after_prefix": 2943.3609289617484,
+        "prefill_tokens_per_s_raw": 7722.316120218579,
+        "requests_per_s": 0.23743169398907105
+      },
+      "prefix_cache_hit_ratio": 0.6189897292366545,
+      "requests": 869,
+      "rho": 0.0008,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 864,
+        "max": 556.1359991102521,
+        "mean": 12.147948538437653,
+        "min": 4.6427767165369005,
+        "p50": 6.843951976861717,
+        "p90": 14.306706048821074,
+        "p95": 30.43280397112956,
+        "p99": 137.6170979713351
+      },
+      "trace_sha256": "2788a21c6bb8be6d03fb5151f2c51efa09c0eb5e1f941af60c00a7cc11d41206",
+      "ttft_ms": {
+        "count": 869,
+        "max": 17076.806859823137,
+        "mean": 1363.8187082248587,
+        "min": 67.46058986817616,
+        "p50": 457.5338581378219,
+        "p90": 4063.5681271006715,
+        "p95": 6757.6291299215145,
+        "p99": 11100.99635793714
+      },
+      "waiting_ms": {
+        "count": 869,
+        "max": 16033.766590147934,
+        "mean": 459.6734459851503,
+        "min": 0.0,
+        "p50": 2.5305451699750847,
+        "p90": 796.2202857544993,
+        "p95": 3149.129404067843,
+        "p99": 9085.03332971697
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 8883,
+          "10": 102,
+          "11": 206,
+          "12": 48,
+          "13": 104,
+          "14": 98,
+          "15": 112,
+          "16": 42432,
+          "2": 8887,
+          "3": 5320,
+          "4": 1084,
+          "5": 391,
+          "6": 1238,
+          "7": 983,
+          "8": 600,
+          "9": 185
+        },
+        "max": 16,
+        "share_gt_1": 0.8743084346214255,
+        "share_gt_4": 0.6579457501450342,
+        "stages": 70673
+      },
+      "drain_fraction": 0.26690162220748465,
+      "drain_tail_s": 976.8599372793938,
+      "offered_load": {
+        "decode_tokens_per_s": 222.2658469945355,
+        "prefill_tokens_per_s_after_prefix": 6478.3401639344265,
+        "prefill_tokens_per_s_raw": 14510.624316939891,
+        "requests_per_s": 0.46939890710382515
+      },
+      "prefix_cache_hit_ratio": 0.5536777047530368,
+      "requests": 1718,
+      "rho": 0.0016,
+      "subcritical_gate": false,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 1710,
+        "max": 1075.3968454229532,
+        "mean": 82.12803809595945,
+        "min": 5.044100516215622,
+        "p50": 55.434284109098826,
+        "p90": 148.45522338371572,
+        "p95": 240.8300400647424,
+        "p99": 469.60790088235683
+      },
+      "trace_sha256": "d785665c86a562c9d9a7492ad710e6393bb883bf8c3ad552a34ff18f22925043",
+      "ttft_ms": {
+        "count": 1718,
+        "max": 899333.5996373689,
+        "mean": 363823.5998174686,
+        "min": 95.16785391630833,
+        "p50": 333469.8464378732,
+        "p90": 786932.9549504423,
+        "p95": 836208.7308378813,
+        "p99": 879481.7276833938
+      },
+      "waiting_ms": {
+        "count": 1718,
+        "max": 898008.7571403392,
+        "mean": 362060.3850241827,
+        "min": 0.0,
+        "p50": 332965.98011225695,
+        "p90": 785791.1926569697,
+        "p95": 832975.7759936375,
+        "p99": 872501.6632742453
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 82578,
+          "10": 992,
+          "11": 686,
+          "12": 564,
+          "13": 388,
+          "14": 207,
+          "15": 120,
+          "16": 556,
+          "2": 76732,
+          "3": 49252,
+          "4": 30080,
+          "5": 19950,
+          "6": 9869,
+          "7": 6575,
+          "8": 3735,
+          "9": 1733
+        },
+        "max": 16,
+        "share_gt_1": 0.7092497984275589,
+        "share_gt_4": 0.15976156356837795,
+        "stages": 284017
+      },
+      "drain_fraction": 0.0057571275011030374,
+      "drain_tail_s": 21.071086654037117,
+      "offered_load": {
+        "decode_tokens_per_s": 222.2658469945355,
+        "prefill_tokens_per_s_after_prefix": 6066.187158469946,
+        "prefill_tokens_per_s_raw": 14510.624316939891,
+        "requests_per_s": 0.46939890710382515
+      },
+      "prefix_cache_hit_ratio": 0.5820880455385098,
+      "requests": 1718,
+      "rho": 0.0016,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 1710,
+        "max": 779.2730316123988,
+        "mean": 31.087273162821237,
+        "min": 4.6427767165369005,
+        "p50": 9.443696854706767,
+        "p90": 67.85056779351899,
+        "p95": 145.53426807885694,
+        "p99": 346.8968267376913
+      },
+      "trace_sha256": "d785665c86a562c9d9a7492ad710e6393bb883bf8c3ad552a34ff18f22925043",
+      "ttft_ms": {
+        "count": 1718,
+        "max": 24498.655361823694,
+        "mean": 2344.154809416248,
+        "min": 72.15684780067022,
+        "p50": 646.7976478797937,
+        "p90": 7657.698621058489,
+        "p95": 10053.03089890951,
+        "p99": 16706.358496869783
+      },
+      "waiting_ms": {
+        "count": 1718,
+        "max": 20607.142196811765,
+        "mean": 1395.7184472333554,
+        "min": 0.0,
+        "p50": 5.521385635063325,
+        "p90": 5253.222863654173,
+        "p95": 8173.9598212075725,
+        "p99": 14808.95660872061
+      }
+    }
+  ],
+  "schema": "frontier-code-trace-calibration-summary-v1",
+  "subcritical_rule": "drain tail <= 10% of the 3660s arrival window",
+  "workload_mode": "prefill_decode"
+}

runs/frontier-code-trace-v0/results/canary-analysis-tp2-v1.json

@@ -0,0 +1,319 @@
+{
+  "drain": {
+    "interpretation": "Report the max-completion request explicitly; a response that arrived well before the cutoff can create a long drain tail without implying queue accumulation.",
+    "real_per_trial": [
+      {
+        "last_arrival_s": 597.0069999999832,
+        "last_completion_s": 714.3421455998905,
+        "tail_after_last_arrival_s": 117.33514559990726,
+        "tail_driver": {
+          "admission_lag_ms": 12.391218915581703,
+          "arrival_before_cutoff_s": 62.03099999995902,
+          "arrival_s": 534.9760000000242,
+          "e2e_ms": 179353.7543809507,
+          "input_tokens": 61976,
+          "output_tokens": 21361,
+          "request_index": 40
+        }
+      }
+    ],
+    "sim": {
+      "last_arrival_s": 597.007,
+      "last_completion_s": 713.4565521329489,
+      "tail_after_last_arrival_s": 116.44955213294895,
+      "tail_driver": {
+        "arrival_before_cutoff_s": 62.03099999999995,
+        "arrival_s": 534.976,
+        "e2e_ms": 178480.5521329489,
+        "input_tokens": 61976,
+        "output_tokens": 21361,
+        "request_index": 40,
+        "waiting_ms": 0.0
+      }
+    }
+  },
+  "input": {
+    "frontier_csv_sha256": "624bc983d3c9fba58cbfa3bb5126304b99b19883da4c78f63cd8622b856dbc1f",
+    "manifest": "runs/frontier-code-trace-v0/results/canary-input-r0p0004-v1/manifest.json",
+    "paired_row_vector_sha256": "72b09903851f5684838da23498a068efc409e6481d1e84efeeefddd8e431c3e7"
+  },
+  "latency_ms": {
+    "e2e": {
+      "real": {
+        "count": 53,
+        "max": 179353.7543809507,
+        "mean": 15936.508002247274,
+        "p50": 3162.7415760885924,
+        "p90": 31124.16882226244,
+        "p95": 62989.56617997021,
+        "p99": 165236.24419869846
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 179353.7543809507,
+          "mean": 15936.508002247274,
+          "p50": 3162.7415760885924,
+          "p90": 31124.16882226244,
+          "p95": 62989.56617997021,
+          "p99": 165236.24419869846
+        }
+      ],
+      "relative_bias_percent": {
+        "mean": -8.269543527339733,
+        "p50": -8.321761801888824,
+        "p90": -18.368943324171262,
+        "p95": -23.79834859601531,
+        "p99": -2.75020118151507
+      },
+      "sim": {
+        "count": 53,
+        "max": 178480.5521329489,
+        "mean": 14618.631536263456,
+        "p50": 2899.5457557171953,
+        "p90": 25406.98789118167,
+        "p95": 47999.08964134313,
+        "p99": 160691.91505845473
+      }
+    },
+    "tpot": {
+      "real": {
+        "count": 53,
+        "max": 641.5267530052612,
+        "mean": 27.41422879865018,
+        "p50": 8.106854443332919,
+        "p90": 23.880988038291846,
+        "p95": 92.03784805296183,
+        "p99": 397.1900480028069
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 641.5267530052612,
+          "mean": 27.41422879865018,
+          "p50": 8.106854443332919,
+          "p90": 23.880988038291846,
+          "p95": 92.03784805296183,
+          "p99": 397.1900480028069
+        }
+      ],
+      "relative_bias_percent": {
+        "mean": -9.878641617364694,
+        "p50": -2.3180300975258694,
+        "p90": 26.54184469239794,
+        "p95": -24.72214442719661,
+        "p99": -13.722225303935645
+      },
+      "sim": {
+        "count": 53,
+        "max": 556.599034356792,
+        "mean": 24.706075383467144,
+        "p50": 7.918935117373849,
+        "p90": 30.219442794425397,
+        "p95": 69.28411832962485,
+        "p99": 342.6867347310516
+      }
+    },
+    "ttft": {
+      "real": {
+        "count": 53,
+        "max": 23546.024559997022,
+        "mean": 3438.4982450513767,
+        "p50": 918.7854011543095,
+        "p90": 10432.362293498598,
+        "p95": 13642.840061988678,
+        "p99": 22228.129854062565
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 23546.024559997022,
+          "mean": 3438.4982450513767,
+          "p50": 918.7854011543095,
+          "p90": 10432.362293498598,
+          "p95": 13642.840061988678,
+          "p99": 22228.129854062565
+        }
+      ],
+      "relative_bias_percent": {
+        "mean": -21.116133763518285,
+        "p50": 13.780393777591794,
+        "p90": -32.09252788469729,
+        "p95": -13.833140262007847,
+        "p99": -14.006169043239403
+      },
+      "sim": {
+        "count": 53,
+        "max": 19564.258977976864,
+        "mean": 2712.4203561700992,
+        "p50": 1045.3976474043998,
+        "p90": 7084.353515424915,
+        "p95": 11755.606860492386,
+        "p99": 19114.8204115518
+      }
+    }
+  },
+  "prefix_cache": {
+    "real_hit_ratio_mean": 0.2399084848137731,
+    "real_per_trial": [
+      {
+        "hit_ratio": 0.2399084848137731,
+        "hit_tokens": 309968.0,
+        "query_tokens": 1292026.0
+      }
+    ],
+    "sim": {
+      "block_size_tokens": 16,
+      "hit_ratio": 0.23997274866840085,
+      "mean_cached_prefill_tokens": 5848.452830188679,
+      "requests": 53,
+      "requests_with_hits": 9,
+      "total_cached_prefill_tokens": 309968,
+      "total_hit_blocks": 19373,
+      "total_query_blocks": 80730
+    }
+  },
+  "real_artifacts": [
+    {
+      "result_sha256": "54ebf30bb66fd845d0d6bbdb15674784989070ec12aa8b785f1dd19886707269",
+      "root": "runs/frontier-code-trace-v0/results/real-canary/r0p0004-tp2-t1-v3"
+    }
+  ],
+  "real_decode_batch_per_trial": [
+    {
+      "histogram": {
+        "1": 53281,
+        "2": 5701,
+        "3": 2919,
+        "4": 476,
+        "5": 110,
+        "6": 62
+      },
+      "max": 6,
+      "mixed_steps_excluded": 152,
+      "share_gt_1": 0.14817183328270636,
+      "steps": 62549
+    }
+  ],
+  "requests_per_trial": 53,
+  "schema": "frontier-code-trace-canary-analysis-v1",
+  "sim_decode_batch": {
+    "histogram": {
+      "1": 53233,
+      "2": 6466,
+      "3": 2626,
+      "4": 395,
+      "5": 103,
+      "6": 29
+    },
+    "max": 6,
+    "share_gt_1": 0.1530420670782155,
+    "share_gt_4": 0.0021001718322408197,
+    "stages": 62852
+  },
+  "slo": {
+    "definition": {
+      "target_pass_rate": 0.95,
+      "tpot_ms": 150,
+      "ttft_ms": "1000 + 1000 * input_tokens / 8000"
+    },
+    "feasibility_flip": false,
+    "real_per_trial": [
+      {
+        "feasible": false,
+        "pass_rate": 0.7735849056603774,
+        "passed": 41
+      }
+    ],
+    "sim": {
+      "feasible": false,
+      "pass_rate": 0.8679245283018868,
+      "passed": 46
+    }
+  },
+  "topology": "tp2_mns16",
+  "tpot_by_min_output_tokens": {
+    "2": {
+      "real": {
+        "count": 53,
+        "max": 641.5267530052612,
+        "mean": 27.41422879865018,
+        "p50": 8.106854443332919,
+        "p90": 23.880988038291846,
+        "p95": 92.03784805296183,
+        "p99": 397.1900480028069
+      },
+      "relative_bias_percent": {
+        "mean": -9.878641617364694,
+        "p50": -2.3180300975258694,
+        "p90": 26.54184469239794,
+        "p95": -24.72214442719661,
+        "p99": -13.722225303935645
+      },
+      "sim": {
+        "count": 53,
+        "max": 556.599034356792,
+        "mean": 24.706075383467144,
+        "p50": 7.918935117373849,
+        "p90": 30.219442794425397,
+        "p95": 69.28411832962485,
+        "p99": 342.6867347310516
+      }
+    },
+    "32": {
+      "real": {
+        "count": 45,
+        "max": 171.6484741543905,
+        "mean": 16.593114914019644,
+        "p50": 8.106854443332919,
+        "p90": 19.44677900381304,
+        "p95": 66.47742398097495,
+        "p99": 148.17240555253153
+      },
+      "relative_bias_percent": {
+        "mean": -9.940572863733157,
+        "p50": -2.3180300975258694,
+        "p90": 10.853794302071227,
+        "p95": -22.75922601837412,
+        "p99": -18.755468241526987
+      },
+      "sim": {
+        "count": 45,
+        "max": 145.22922738421678,
+        "mean": 14.943664235628548,
+        "p50": 7.918935117373849,
+        "p90": 21.557492395265285,
+        "p95": 51.34767680595202,
+        "p99": 120.38197708641991
+      }
+    },
+    "8": {
+      "real": {
+        "count": 52,
+        "max": 171.6484741543905,
+        "mean": 15.604372563907663,
+        "p50": 8.031664372446695,
+        "p90": 21.733123898935485,
+        "p95": 52.378594044159485,
+        "p99": 144.4375764567812
+      },
+      "relative_bias_percent": {
+        "mean": -7.222385426340451,
+        "p50": -1.403560331275464,
+        "p90": 10.95444148849469,
+        "p95": -10.968821104857895,
+        "p99": -19.391471177881424
+      },
+      "sim": {
+        "count": 52,
+        "max": 145.22922738421678,
+        "mean": 14.477364633980129,
+        "p50": 7.918935117373849,
+        "p90": 24.11386624006643,
+        "p95": 46.63327976621588,
+        "p99": 116.42900544813402
+      }
+    }
+  },
+  "trials": 1
+}

runs/frontier-code-trace-v0/results/canary-analysis-tp4-v2.json

@@ -0,0 +1,484 @@
+{
+  "drain": {
+    "interpretation": "Report the max-completion request explicitly; a response that arrived well before the cutoff can create a long drain tail without implying queue accumulation.",
+    "real_per_trial": [
+      {
+        "last_arrival_s": 597.0069999999832,
+        "last_completion_s": 689.6167901020963,
+        "tail_after_last_arrival_s": 92.60979010211304,
+        "tail_driver": {
+          "admission_lag_ms": 11.249033967033029,
+          "arrival_before_cutoff_s": 62.03099999995902,
+          "arrival_s": 534.9760000000242,
+          "e2e_ms": 154629.54106810503,
+          "input_tokens": 61976,
+          "output_tokens": 21361,
+          "request_index": 40
+        }
+      },
+      {
+        "last_arrival_s": 597.0069999999832,
+        "last_completion_s": 689.344073610846,
+        "tail_after_last_arrival_s": 92.33707361086272,
+        "tail_driver": {
+          "admission_lag_ms": 12.91605201549828,
+          "arrival_before_cutoff_s": 62.03099999995902,
+          "arrival_s": 534.9760000000242,
+          "e2e_ms": 154355.15755880624,
+          "input_tokens": 61976,
+          "output_tokens": 21361,
+          "request_index": 40
+        }
+      }
+    ],
+    "sim": {
+      "last_arrival_s": 597.007,
+      "last_completion_s": 688.377327361797,
+      "tail_after_last_arrival_s": 91.37032736179708,
+      "tail_driver": {
+        "arrival_before_cutoff_s": 62.03099999999995,
+        "arrival_s": 534.976,
+        "e2e_ms": 153401.32736179704,
+        "input_tokens": 61976,
+        "output_tokens": 21361,
+        "request_index": 40,
+        "waiting_ms": 0.0
+      }
+    }
+  },
+  "input": {
+    "frontier_csv_sha256": "624bc983d3c9fba58cbfa3bb5126304b99b19883da4c78f63cd8622b856dbc1f",
+    "manifest": "runs/frontier-code-trace-v0/results/canary-input-r0p0004-v1/manifest.json",
+    "paired_row_vector_sha256": "72b09903851f5684838da23498a068efc409e6481d1e84efeeefddd8e431c3e7"
+  },
+  "latency_ms": {
+    "e2e": {
+      "real": {
+        "count": 106,
+        "max": 154629.54106810503,
+        "mean": 10734.692601765997,
+        "p50": 1698.6070264829323,
+        "p90": 16955.372447962873,
+        "p95": 40098.011117079295,
+        "p99": 152916.82229990847
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 154629.54106810503,
+          "mean": 10752.865282098219,
+          "p50": 1699.7638561297208,
+          "p90": 16719.75081679412,
+          "p95": 32787.88763261397,
+          "p99": 139528.17495073183
+        },
+        {
+          "count": 53,
+          "max": 154355.15755880624,
+          "mean": 10716.519921433777,
+          "p50": 1697.8129579219967,
+          "p90": 16614.017988741405,
+          "p95": 32681.116299051762,
+          "p99": 139096.4567810855
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 10734.692601765997,
+        "p50": 1698.7884070258588,
+        "p90": 16666.884402767762,
+        "p95": 32734.501965832867,
+        "p99": 139312.31586590866
+      },
+      "relative_bias_percent": {
+        "mean": 2.2014558826783013,
+        "p50": 6.0982380145415895,
+        "p90": 11.04630840126803,
+        "p95": 1.3549410756255433,
+        "p99": -1.3257901787372948
+      },
+      "sim": {
+        "count": 53,
+        "max": 153401.32736179704,
+        "mean": 10971.012123535007,
+        "p50": 1802.3845674497352,
+        "p90": 18507.95985478033,
+        "p95": 33178.03517886939,
+        "p99": 137465.32686438697
+      }
+    },
+    "tpot": {
+      "real": {
+        "count": 106,
+        "max": 212.7724236343056,
+        "mean": 10.903347021010111,
+        "p50": 6.172303046257078,
+        "p90": 8.493485802941574,
+        "p95": 9.056036081054664,
+        "p99": 203.35920991970733
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 212.7724236343056,
+          "mean": 10.931995250979348,
+          "p50": 6.17668606558597,
+          "p90": 8.407062460850472,
+          "p95": 8.930405185071265,
+          "p99": 137.66455429146993
+        },
+        {
+          "count": 53,
+          "max": 210.46578868602714,
+          "mean": 10.874698791040872,
+          "p50": 6.167920026928186,
+          "p90": 8.391951717689471,
+          "p95": 8.880314963829257,
+          "p99": 136.53404073389595
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 10.90334702101011,
+        "p50": 6.172303046257078,
+        "p90": 8.39950708926997,
+        "p95": 8.905360074450261,
+        "p99": 137.09929751268294
+      },
+      "relative_bias_percent": {
+        "mean": -29.893478835305263,
+        "p50": 8.372972857395876,
+        "p90": -4.849833918394976,
+        "p95": 9.343177342624926,
+        "p99": -71.54426929141353
+      },
+      "sim": {
+        "count": 53,
+        "max": 68.73522813087318,
+        "mean": 7.643957286944565,
+        "p50": 6.6891083049964015,
+        "p90": 7.992144945476565,
+        "p95": 9.737403659205464,
+        "p99": 39.01260690357286
+      }
+    },
+    "ttft": {
+      "real": {
+        "count": 106,
+        "max": 12213.839336996898,
+        "mean": 1499.2377276352238,
+        "p50": 523.7399585312232,
+        "p90": 3843.0782224750146,
+        "p95": 7372.6646098075435,
+        "p99": 12137.850578338843
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 12213.839336996898,
+          "mean": 1501.9183055506492,
+          "p50": 524.1345730610192,
+          "p90": 3807.3078704066584,
+          "p95": 6591.913985600691,
+          "p99": 11615.677419919517
+        },
+        {
+          "count": 53,
+          "max": 12194.393874146044,
+          "mean": 1496.5571497197984,
+          "p50": 523.3453440014273,
+          "p90": 3795.468537742273,
+          "p95": 6574.768227571618,
+          "p99": 11596.839222898703
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 1499.2377276352238,
+        "p50": 523.7399585312232,
+        "p90": 3801.3882040744656,
+        "p95": 6583.341106586155,
+        "p99": 11606.25832140911
+      },
+      "relative_bias_percent": {
+        "mean": -8.33138601964073,
+        "p50": -4.13255547614917,
+        "p90": -8.820129670118508,
+        "p95": -13.08726396978494,
+        "p99": -6.311340671626045
+      },
+      "sim": {
+        "count": 53,
+        "max": 11454.943556960472,
+        "mean": 1374.3304451938434,
+        "p50": 502.0961141941598,
+        "p90": 3466.1008352105086,
+        "p95": 5721.761877935864,
+        "p99": 10873.747819516035
+      }
+    }
+  },
+  "prefix_cache": {
+    "real_hit_ratio_mean": 0.2399084848137731,
+    "real_per_trial": [
+      {
+        "hit_ratio": 0.2399084848137731,
+        "hit_tokens": 309968.0,
+        "query_tokens": 1292026.0
+      },
+      {
+        "hit_ratio": 0.2399084848137731,
+        "hit_tokens": 309968.0,
+        "query_tokens": 1292026.0
+      }
+    ],
+    "sim": {
+      "block_size_tokens": 16,
+      "hit_ratio": 0.23997274866840085,
+      "mean_cached_prefill_tokens": 5848.452830188679,
+      "requests": 53,
+      "requests_with_hits": 9,
+      "total_cached_prefill_tokens": 309968,
+      "total_hit_blocks": 19373,
+      "total_query_blocks": 80730
+    }
+  },
+  "real_artifacts": [
+    {
+      "result_sha256": "c1105d6ba182be86fcd85f4eaf6889d1e788d48f923779b742ab95b4d3bd46ca",
+      "root": "runs/frontier-code-trace-v0/results/real-canary/r0p0004-tp4-t1-v2"
+    },
+    {
+      "result_sha256": "8e9092ac94b9d84d13aca41bd41d33db67d9917c8cedc8b61854aa2963a124ce",
+      "root": "runs/frontier-code-trace-v0/results/real-canary/r0p0004-tp4-t2-v3"
+    }
+  ],
+  "real_decode_batch_per_trial": [
+    {
+      "histogram": {
+        "1": 54699,
+        "2": 9377,
+        "3": 867,
+        "4": 77
+      },
+      "max": 4,
+      "mixed_steps_excluded": 155,
+      "share_gt_1": 0.15873577360812058,
+      "steps": 65020
+    },
+    {
+      "histogram": {
+        "1": 54893,
+        "2": 9265,
+        "3": 877,
+        "4": 77
+      },
+      "max": 4,
+      "mixed_steps_excluded": 155,
+      "share_gt_1": 0.15694495638284803,
+      "steps": 65112
+    }
+  ],
+  "requests_per_trial": 53,
+  "schema": "frontier-code-trace-canary-analysis-v1",
+  "sim_decode_batch": {
+    "histogram": {
+      "1": 54192,
+      "2": 9673,
+      "3": 840,
+      "4": 77
+    },
+    "max": 4,
+    "share_gt_1": 0.1634713346299898,
+    "share_gt_4": 0.0,
+    "stages": 64782
+  },
+  "slo": {
+    "definition": {
+      "target_pass_rate": 0.95,
+      "tpot_ms": 150,
+      "ttft_ms": "1000 + 1000 * input_tokens / 8000"
+    },
+    "feasibility_flip": true,
+    "real_per_trial": [
+      {
+        "feasible": false,
+        "pass_rate": 0.9433962264150944,
+        "passed": 50
+      },
+      {
+        "feasible": false,
+        "pass_rate": 0.9433962264150944,
+        "passed": 50
+      }
+    ],
+    "sim": {
+      "feasible": true,
+      "pass_rate": 0.9811320754716981,
+      "passed": 52
+    }
+  },
+  "topology": "tp4_mns16",
+  "tpot_by_min_output_tokens": {
+    "2": {
+      "real": {
+        "count": 106,
+        "max": 212.7724236343056,
+        "mean": 10.903347021010111,
+        "p50": 6.172303046257078,
+        "p90": 8.493485802941574,
+        "p95": 9.056036081054664,
+        "p99": 203.35920991970733
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 212.7724236343056,
+          "mean": 10.931995250979348,
+          "p50": 6.17668606558597,
+          "p90": 8.407062460850472,
+          "p95": 8.930405185071265,
+          "p99": 137.66455429146993
+        },
+        {
+          "count": 53,
+          "max": 210.46578868602714,
+          "mean": 10.874698791040872,
+          "p50": 6.167920026928186,
+          "p90": 8.391951717689471,
+          "p95": 8.880314963829257,
+          "p99": 136.53404073389595
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 10.90334702101011,
+        "p50": 6.172303046257078,
+        "p90": 8.39950708926997,
+        "p95": 8.905360074450261,
+        "p99": 137.09929751268294
+      },
+      "relative_bias_percent": {
+        "mean": -29.893478835305263,
+        "p50": 8.372972857395876,
+        "p90": -4.849833918394976,
+        "p95": 9.343177342624926,
+        "p99": -71.54426929141353
+      },
+      "sim": {
+        "count": 53,
+        "max": 68.73522813087318,
+        "mean": 7.643957286944565,
+        "p50": 6.6891083049964015,
+        "p90": 7.992144945476565,
+        "p95": 9.737403659205464,
+        "p99": 39.01260690357286
+      }
+    },
+    "32": {
+      "real": {
+        "count": 90,
+        "max": 68.33421335962247,
+        "mean": 7.270237487352443,
+        "p50": 6.167292870073156,
+        "p90": 8.490015433923487,
+        "p95": 8.695471711641535,
+        "p99": 68.2942852512835
+      },
+      "real_per_trial": [
+        {
+          "count": 45,
+          "max": 68.33421335962247,
+          "mean": 7.2794157345844095,
+          "p50": 6.17668606558597,
+          "p90": 8.316301157486755,
+          "p95": 8.667796532745909,
+          "p99": 42.34591449575732
+        },
+        {
+          "count": 45,
+          "max": 68.28935031654497,
+          "mean": 7.261059240120478,
+          "p50": 6.157899674560342,
+          "p90": 8.294755593709974,
+          "p95": 8.647785231561759,
+          "p99": 42.27831274721521
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 7.270237487352444,
+        "p50": 6.167292870073156,
+        "p90": 8.305528375598364,
+        "p95": 8.657790882153833,
+        "p99": 42.31211362148626
+      },
+      "relative_bias_percent": {
+        "mean": 8.827198903080898,
+        "p50": 10.602910786790718,
+        "p90": 3.6523081450671953,
+        "p95": 13.443742144167812,
+        "p99": 3.009077380352242
+      },
+      "sim": {
+        "count": 45,
+        "max": 68.73522813087318,
+        "mean": 7.911995811087395,
+        "p50": 6.821205431047118,
+        "p90": 8.60887186495121,
+        "p95": 9.821721963731866,
+        "p99": 43.585317861619345
+      }
+    },
+    "8": {
+      "real": {
+        "count": 104,
+        "max": 68.33421335962247,
+        "mean": 7.043428576026337,
+        "p50": 6.162909850744264,
+        "p90": 8.031762239708375,
+        "p95": 8.660612540767117,
+        "p99": 66.51876673984641
+      },
+      "real_per_trial": [
+        {
+          "count": 52,
+          "max": 68.33421335962247,
+          "mean": 7.050448551299998,
+          "p50": 6.167180339419747,
+          "p90": 8.00607090381441,
+          "p95": 8.604306061843479,
+          "p99": 38.21141240377873
+        },
+        {
+          "count": 52,
+          "max": 68.28935031654497,
+          "mean": 7.036408600752676,
+          "p50": 6.162909850744264,
+          "p90": 7.971146045984491,
+          "p95": 8.582739223220587,
+          "p99": 38.14019313391272
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 7.043428576026336,
+        "p50": 6.1650450950820055,
+        "p90": 7.988608474899451,
+        "p95": 8.593522642532033,
+        "p99": 38.17580276884573
+      },
+      "relative_bias_percent": {
+        "mean": 8.736391650572934,
+        "p50": 8.20225561474644,
+        "p90": 0.591743534737434,
+        "p95": 13.433615674969445,
+        "p99": 3.6892295703930555
+      },
+      "sim": {
+        "count": 52,
+        "max": 68.73522813087318,
+        "mean": 7.658770082056369,
+        "p50": 6.670717852545019,
+        "p90": 8.035880549065155,
+        "p95": 9.747943447271265,
+        "p99": 39.584195773328915
+      }
+    }
+  },
+  "trials": 2
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/provenance/gpus.after.csv

@@ -0,0 +1,8 @@
+0, NVIDIA H20, 0, 0
+1, NVIDIA H20, 0, 0
+2, NVIDIA H20, 0, 0
+3, NVIDIA H20, 0, 0
+4, NVIDIA H20, 0, 0
+5, NVIDIA H20, 0, 0
+6, NVIDIA H20, 0, 0
+7, NVIDIA H20, 0, 0

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/provenance/gpus.before.csv

@@ -0,0 +1,8 @@
+0, NVIDIA H20, 0, 0, 0
+1, NVIDIA H20, 0, 0, 0
+2, NVIDIA H20, 0, 0, 0
+3, NVIDIA H20, 0, 0, 0
+4, NVIDIA H20, 0, 0, 0
+5, NVIDIA H20, 0, 0, 0
+6, NVIDIA H20, 0, 0, 0
+7, NVIDIA H20, 0, 0, 0

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/provenance/sources.sha256

@@ -0,0 +1,3 @@
+9857f3ee2565e0ae5bf414c769ea7404295eb7b15fc4a4e8c0f85726312e27a5  runs/frontier-code-trace-v0/run_max_model_len_smoke.sh
+0b6a6d1d8c71f57b8e11b4eb89a9887e182a327223206577086605930da3c028  /home/admin/cpfs/wjh/aituner/aituner-code-smoke-da480a8/runs/frontier-s3-real-v0/qwen30_prefill_client.py
+2850ddb3bf7aecad20b611e2d44f3077fc8193f4827c93beddd4c02ad63c2297  /home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B/config.json

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/code-candidate-max.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.09795790538191795,
+      "admitted_s": 9.795790538191795e-05,
+      "completion_tokens": 242,
+      "e2e_ms": 5778.945273021236,
+      "prompt_tokens": 136774,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": true,
+      "streamed_token_count": 242,
+      "success": true,
+      "tpot_ms": 8.126009058562799,
+      "ttft_ms": 3820.549983996898
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.09795790538191795,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": true,
+      "pass_rate": 1.0,
+      "passed": 1,
+      "tpot_threshold_ms": 150.0,
+      "ttft_threshold_ms": 18096.75
+    },
+    "tpot_max_ms": 8.126009058562799,
+    "tpot_p50_ms": 8.126009058562799,
+    "tpot_p95_ms": 8.126009058562799,
+    "ttft_max_ms": 3820.549983996898,
+    "ttft_p50_ms": 3820.549983996898,
+    "ttft_p95_ms": 3820.549983996898
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 136774,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 242,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "40852e3d159377ae557b58933b077bee1e565bc51a4b7d4e3ea0070709e63999",
+    "request_count": 1
+  }
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/code-candidate-p99.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.10300683788955212,
+      "admitted_s": 0.00010300683788955212,
+      "completion_tokens": 68,
+      "e2e_ms": 13182.361921994016,
+      "prompt_tokens": 119702,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": true,
+      "streamed_token_count": 68,
+      "success": true,
+      "tpot_ms": 7.21338267243509,
+      "ttft_ms": 12699.044595938176
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.10300683788955212,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": true,
+      "pass_rate": 1.0,
+      "passed": 1,
+      "tpot_threshold_ms": 150.0,
+      "ttft_threshold_ms": 15962.75
+    },
+    "tpot_max_ms": 7.21338267243509,
+    "tpot_p50_ms": 7.21338267243509,
+    "tpot_p95_ms": 7.21338267243509,
+    "ttft_max_ms": 12699.044595938176,
+    "ttft_p50_ms": 12699.044595938176,
+    "ttft_p95_ms": 12699.044595938176
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 119702,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 68,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "aaa2e7077defd41a1abb6227d9d22ec928a5d56e18fb3135783a6044e8365af7",
+    "request_count": 1
+  }
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/code-p50.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.08747284300625324,
+      "admitted_s": 8.747284300625324e-05,
+      "completion_tokens": 78,
+      "e2e_ms": 2135.5441550258547,
+      "prompt_tokens": 20051,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": true,
+      "streamed_token_count": 78,
+      "success": true,
+      "tpot_ms": 16.65233903830605,
+      "ttft_ms": 853.2876779790968
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.08747284300625324,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": true,
+      "pass_rate": 1.0,
+      "passed": 1,
+      "tpot_threshold_ms": 150.0,
+      "ttft_threshold_ms": 3506.375
+    },
+    "tpot_max_ms": 16.65233903830605,
+    "tpot_p50_ms": 16.65233903830605,
+    "tpot_p95_ms": 16.65233903830605,
+    "ttft_max_ms": 853.2876779790968,
+    "ttft_p50_ms": 853.2876779790968,
+    "ttft_p95_ms": 853.2876779790968
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 20051,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 78,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "a814b02092178b7e949e4d14d15aa271c9e93cead6552d91413d8f9abf2c9287",
+    "request_count": 1
+  }
+}